Can an unbiased estimate be wrong in one sample?

Yes; unbiasedness concerns its average over repeated samples.

Is standard error the actual error?

No; it describes variability of the estimator.

Does a census remove every error?

No; measurement, coverage, and processing errors may remain.

AI Glossary / Statistics and probability / Intermediate

Sampling error

Definition

Sampling error is the random variation in an estimate that arises because information is collected from a sample rather than the entire target population. Different valid samples can produce different results. It does not necessarily indicate a mistake, and it is distinct from systematic selection problems, inaccurate responses, or processing errors.

What it means

A carefully designed survey cannot usually reproduce the population value exactly from every possible sample. Some samples contain more high values, others more low values. Their estimates therefore fluctuate even when measurement and calculation are correct.
The phrase sometimes refers to a particular estimate's departure from the population value due to sampling, and sometimes to the broader uncertainty generated by sampling. Because the true population value is usually unknown, analysts generally estimate the variability of the procedure rather than observe its realized error directly.

How it works

Specify the target population, parameter, estimator, and sampling design. Random error has meaning relative to these choices. A mean for all current customers is a different target from a mean for only recently active customers.
Imagine the possible samples selected by the same probability design. Calculate the estimate for each and consider its distribution. The variation across these estimates is sampling variability; its variance is the sampling variance.
The square root of sampling variance is the standard error. Estimated standard errors use observed data and design information to quantify how much estimates could vary across repeated samples. They are not measurements of the exact error in the current estimate.
A confidence interval or margin of error converts a standard-error estimate into an uncertainty statement under further conditions. The confidence level describes the long-run performance of the interval procedure, not the percentage of individual responses within its limits.
Larger independent samples commonly reduce sampling variability. The improvement depends on design, outcome variability, and estimator. Clustering, unequal weights, stratification, and sampling a substantial fraction of a finite population can all change precision.
Separate sampling variation from non-sampling error. Missing parts of the population, inaccurate answers, coding mistakes, and nonresponse can distort estimates in ways that a conventional sampling margin does not capture.
A probability sample permits design-based uncertainty calculations when its selection probabilities and relevant design features are available. A convenience sample needs additional modeling assumptions for population inference; attaching a familiar margin-of-error formula does not establish representativeness.
A complete, correctly measured census removes sampling variation for the enumerated finite population. It can still contain non-sampling errors. Likewise, zero estimated sampling uncertainty does not guarantee that the chosen measure answers the intended business question.

Three practical examples

Customer surveys: two random samples can report slightly different satisfaction averages without either being incorrectly calculated.
AI audits: an observed error rate changes when different eligible cases are sampled for review.
Quality inspection: estimates of average batch characteristics vary across randomly selected subsets of items.

Think of it this way

Think of tasting a well-mixed soup using different small spoonfuls. The spoonfuls can still differ by chance. Sampling error concerns this variation; it does not explain an unwashed spoon, a badly mixed pot, or recording the wrong taste score.

Worked example

Use an original finite population of four values: 10, 20, 30, and 40. Its mean is 25. Select two different units by simple random sampling without replacement, so all six unordered pairs are equally likely.
The possible sample means are 15, 20, 25, 25, 30, and 35. Their errors relative to 25 are -10, -5, 0, 0, 5, and 10. The average error is zero, showing that this sampling procedure's mean estimator is unbiased.
Its sampling variance is (100 + 25 + 0 + 0 + 25 + 100) / 6 = 125/3, approximately 41.667. The standard error is square root of 125/3, approximately 6.455.
A sample containing 10 and 20 has mean 15 and realized error -10. This does not contradict unbiasedness: unbiasedness averages over possible samples. In this toy example the entire population is known, so the realized error can be calculated. In an ordinary sample survey that population information is typically unavailable.

Illustrative case study

Fictional case: Maple Services samples support tickets to estimate average resolution time. Two monthly audits produce different averages, and a manager assumes one audit must be wrong. Analysts quantify sampling uncertainty and check whether the sample design changed. They also discover that one queue was excluded from the ticket list, a separate coverage problem. The company corrects the frame and reports uncertainty. It does not claim that a larger sample alone would have repaired the missing queue.

Common misconceptions and mistakes

Do not equate sampling error with researcher negligence. Do not describe a biased selection process as merely random fluctuation. A small standard error does not rule out systematic error. Two estimates differing numerically are not automatically meaningfully different. A margin of error for one estimate does not automatically give the uncertainty of their difference, especially when samples overlap.

Business use

Use sampling-error information to decide how much confidence to place in survey changes, evaluation scores, and quality estimates. Present uncertainty alongside estimates and explain what it covers. Plan sample sizes around useful precision, while separately maintaining measurement, coverage, and processing quality.

Check your understanding

Why is the toy estimator unbiased?

Answer: The six possible means average to 25.

Why does the sample mean of 15 still have an error?

Answer: Unbiasedness does not guarantee exactness in each sample.

Would adding more records fix an excluded customer group automatically?

Answer: No.

Related terms

Sampling distribution
Standard error
Non-sampling error
Selection bias
Census
sampling variance
margin of error
probability sampling

Learn next

Study how survey designs affect variance and how confidence intervals are constructed. Then examine total survey error to understand why statistical precision and overall data quality need separate attention.

Data Quality

Sources

Australian Bureau of Statistics, Types of error: https://www.abs.gov.au/statistics/understanding-statistics/statistical-terms-and-concepts/types-error Australian Bureau of Statistics, Errors in Statistical Data: https://www.abs.gov.au/websitedbs/d3310114.nsf/home/Basic%20Survey%20Design%20-%20Errors%20in%20Statistical%20Data University of Vermont glossary: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html The finite-population example is original.

Previous: Sampling distribution of the mean Next: Sampling fraction