Can a biased estimator be consistent?

Yes. For example, adding 1 / n to a consistent sample mean creates a finite-sample bias that vanishes as n grows.

Does consistency give a rate of convergence?

Not by itself. Rates require additional results and assumptions.

Does consistency require independent data?

No. Consistency can hold for dependent data under suitable conditions, but an independent-data argument cannot simply be reused without checking those conditions.

AI Glossary / Probabilistic machine learning / Intermediate

Consistent estimator

Definition

A consistent estimator is an estimation rule whose output converges in probability to the quantity being estimated as the sample size grows, under stated assumptions. For any fixed positive error tolerance, the probability that its error exceeds that tolerance tends to zero.

What it means

An estimator is a rule applied to data, such as taking a sample average. Its result changes from sample to sample. Consistency concerns a sequence of these rules or results as more data become available, rather than the accuracy of one observed estimate.
The property says the method eventually concentrates near the correct target in the specified statistical setting. It does not identify a universal sample size that is “large enough,” guarantee improvement after every additional observation, or establish that the assumptions hold in a particular business dataset.

How it works

First define the target, such as a population mean or a model parameter. Then state the data assumptions and estimation rule for each sample size n. To establish ordinary consistency, show that P(|estimate_n − target| > epsilon) approaches zero for every epsilon greater than zero.
For independent, identically distributed observations with finite variance, the sample mean has variance equal to the population variance divided by n. Combined with its zero bias, this yields consistency through a probability bound.
Other estimators require different arguments. Strong consistency uses almost-sure convergence, a stronger property. Mean-square convergence also implies convergence in probability, but ordinary consistency does not require every estimator to have a vanishing mean squared error.

Three practical examples

Service measurement: estimate the mean duration of a stable process from an appropriate sample, using the sample mean when its assumptions are reasonable.
Conversion analysis: estimate a fixed success probability by a sample proportion under a suitable repeated-trial model, distinguishing that target from a changing live rate.
Model fitting: assess whether a parameter estimator approaches the intended parameter under identification and regularity assumptions, rather than assuming any optimization result is statistically consistent.

Think of it this way

Think of locating a landmark from repeated noisy readings. A suitable averaging method can concentrate around its true position as more readings arrive. The analogy fails if every reading shares an uncorrected offset or the landmark is moving. Likewise, more data cannot automatically remove systematic measurement error or make an ill-defined statistical target stable.

Worked example

Consider an original example with independent, identically distributed binary observations having success probability p = 0.3. The sample proportion p_hat has expectation 0.3 and variance p(1 − p) / n = 0.21 / n.
At n = 100, its standard deviation is sqrt(0.0021), approximately 0.0458. At n = 10,000, it is sqrt(0.000021), approximately 0.00458. Increasing sample size one hundredfold reduces the standard deviation tenfold in this model.
For an error tolerance of 0.05, Chebyshev's inequality gives P(|p_hat − 0.3| ≥ 0.05) ≤ 0.21 / (n × 0.0025) = 84 / n. At n = 10,000, the bound is 0.0084. This is an upper bound, not the exact error probability.
As n grows, the bound approaches zero, establishing consistency. By contrast, the estimator p_hat + 0.1 converges to 0.4 and is inconsistent for p = 0.3, despite becoming increasingly stable.

Illustrative case study

Fictional case: Elm Support estimates average handling time from a growing collection of completed calls. The dashboard becomes less variable as the dataset grows. An analyst then discovers that unusually long calls were excluded by a logging timeout.
The team distinguishes the mean of the recorded-call distribution from the mean of all calls. A consistent estimator for the first target need not estimate the second correctly. Elm repairs the collection process and redefines the reporting population before treating additional data as stronger evidence. This case illustrates target and sampling issues, not a claimed operational result.

Common misconceptions and mistakes

Consistency is not the same as unbiasedness. An unbiased estimator may keep substantial variability as n grows; using only the first observation to estimate a mean is an example. A biased estimator can nevertheless be consistent if its error disappears in the appropriate limit.
Consistency also does not imply that the estimate moves closer to the target at every step. Nor does a stable training loss establish statistical consistency. Under model misspecification, a fitting method may converge to a best approximation within the model rather than to a parameter representing the intended real-world quantity.

Business use

For businesses, consistency helps justify using more data with a particular estimation method. It is a long-run property, not a substitute for finite-sample uncertainty, representative measurement, or validation.
Ask which target is being estimated, what assumptions support consistency, and how large the current uncertainty is. Investigate duplicate records, dependence, selection effects, and changing populations where relevant. A million observations from an unsuitable collection process can support a very precise answer to the wrong question.

Check your understanding

Let X1, X2, and so on be independent observations with mean mu and positive finite variance. Define the estimator T_n = X1 for every n. Is it consistent for mu?

Answer: Generally no. It ignores all additional observations, so its distribution does not concentrate around mu. It remains unbiased, showing that unbiasedness alone does not establish consistency.

Related terms

Law of large numbers
Bias
Variance
Mean Squared Error
Estimator
convergence in probability
strong consistency
model misspecification
identifiability

Learn next

Learn sampling distributions before studying consistency. Reproduce the sample-proportion calculation, then compare three rules: the sample mean, the mean plus 1 / n, and the first observation alone. Next, study how dependence or a changing target alters the assumptions behind convergence.

Sample mean

Sources

Cosma Shalizi, original course notes, “Inference I—Inference with Independent Data,” including modes of convergence and the sample-mean argument: https://stat.cmu.edu/~cshalizi/dst/20/lectures/13/lecture-13.html Andrew Chesher, original Course Notes 3, convergence in probability and consistency: https://www.homepages.ucl.ac.uk/~uctpjea/lnotes3.pdf Independent source review: Shalizi’s sample-mean, convergence-mode and Chebyshev passages, and Chesher’s page 2 consistency discussion, were read.

Previous: Consistency regularization Next: Consonant