AI Glossary / Statistics and probability / Foundation
Standard error
Definition
The standard error is the standard deviation of an estimator's sampling distribution. It describes how much an estimate would vary across repeated samples or repetitions of the specified data-generating procedure. In practice, it is often estimated from the available data. Different statistics have different standard errors. The standard error of a mean is a common example, but the term also applies to proportions, regression coefficients and many other estimates.
What it means
A sample average is not the same on every possible sample. Standard error quantifies that repeated-sampling variation under a specified design or model. It does not describe the spread of individual observations, which is typically summarized by their standard deviation. A small standard error indicates precision under the assumptions used; it does not guarantee freedom from selection bias, measurement error or model misspecification. An estimate can be very precise and still systematically miss the intended target.
How it works
Identify the estimator and the sampling or modeling assumptions. For the mean of n independent identically distributed observations with population standard deviation sigma, the standard error is sigma divided by the square root of n. A common estimate replaces sigma with the sample standard deviation s. Other settings require different calculations, including finite-population corrections, survey-design methods, cluster-aware variance estimation or suitable resampling. A confidence interval additionally needs a justified reference distribution or interval procedure; a standard error alone is not a confidence level.
Three practical examples
Example 1: A survey reports the sampling precision of an estimated population average. Example 2: An AI evaluation reports uncertainty in mean performance across independently sampled evaluation units. Example 3: A regression analysis reports uncertainty for an estimated coefficient under a stated covariance estimator. In each example, the analysis must identify what can vary across repetitions. Thousands of measurements from a small number of dependent units do not necessarily provide the information implied by thousands of independent observations.
Think of it this way
Think of repeatedly taking a scoop from a well-mixed container and calculating each scoop's average. The individual pieces vary within each scoop; the scoop averages also vary across scoops. Standard deviation can describe the first kind of variation, while the standard error of the mean describes the second. The analogy works only when the scooping procedure reflects the assumed sampling design.
Worked example
Suppose a population takes values zero and two with equal probability. Its mean is one and its standard deviation is one. Draw four independent observations with replacement and calculate their mean. There are sixteen equally likely ordered samples. Their means can be 0, 0.5, 1, 1.5 or 2, with counts 1, 4, 6, 4 and 1. The mean of these sixteen sample means is one, and their population variance is 0.25. Their standard deviation, the standard error, is therefore 0.5, matching sigma divided by the square root of n = 1/2. This exact example shows that the sampling distribution need not be normal for its standard error to exist. In a separate planning illustration with sigma fixed at twelve, increasing independent sample size from 36 to 144 reduces the mean's standard error from two to one.
Illustrative case study
In this fictional case study, Linden Research evaluates an AI assistant using many tasks completed by a small number of employees. An initial report treats every task as independent and gives an overly narrow uncertainty interval. The analyst revises the method to reflect employee-level dependence and clearly states the target population. Managers receive the estimated average with an appropriately qualified standard error, alongside information about task coverage and possible selection bias. The larger uncertainty is treated as useful evidence about the evaluation's limits.
Common misconceptions and mistakes
Common mistakes include using a standard error to describe individual variability, assuming it measures every source of error, and interpreting a small value as proof of practical importance. The familiar inverse-square-root sample-size rule relies on the relevant independence and variance assumptions. Repeating identical records does not create new information. Bootstrap estimates also require a resampling scheme consistent with the data structure; resampling individual rows can be inappropriate for clustered or dependent observations.
Business use
Businesses use standard errors to distinguish a precise estimate from one that remains uncertain and to plan data collection. Report the estimate, its units, the standard-error method and the effective sampling structure. Consider intervals and decision-relevant effect sizes rather than ranking results only by precision. More observations can reduce sampling variation while leaving systematic error untouched. When comparing two estimates, calculate uncertainty for their difference rather than assuming that separate error bars fully determine whether the difference is meaningful.
People also ask
Is standard error the same as standard deviation?
It is a standard deviation of an estimator's sampling distribution, not usually the standard deviation of the raw data.
Does it apply only to means?
No.
Does quadrupling sample size always halve it?
Only under a suitable setting such as independent identically distributed sampling with fixed variance for a mean.
Can a small standard error coexist with bias?
Yes.
Check your understanding
What is the standard error in the four-draw example?
Answer: 0.5.
What is the raw population standard deviation?
Answer: One.
Is the distribution of the sixteen means exactly normal?
Answer: No; it has five possible values.
Why does copying observations not improve precision as new independent sampling would?
Answer: Copies do not provide independent information about the target.
Related terms
- Standard deviation
- Confidence interval
- Sampling error
- Related terms include sampling distribution
- estimator
- bootstrap and design effect
Learn next
Learn next how variance of an average changes with covariance between observations. Compare independent, paired, clustered and survey samples, and examine how each design changes uncertainty calculations. Then study confidence intervals and why the reference distribution, degrees of freedom and estimator assumptions matter alongside the standard error.
More in Statistics and probability
- Squared ranks test - The squared ranks test, often called the Conover squared ranks test, is a rank-based procedure for comparing dispersion across independent groups.
- Stable model - A stable model is a formally defined interpretation of a logic program with default negation.
- Staged tree (mathematics) - A staged tree is a probabilistic event tree whose nonterminal situations are grouped into stages when they share the same conditional distribution over...
- Standard error of differences between means - The standard error of differences between means is the standard deviation of the sampling distribution of a difference between estimated means.
- Standard lottery - A standard lottery is a reference gamble used in expected-utility assessment, typically offering a defined best outcome with probability p and a defined...
- Standard normal distribution - The standard normal distribution is the normal, or Gaussian, probability distribution with mean zero and variance one.
Sources
Sources: Australian Bureau of Statistics, Methods: Four pillars of labour statistics, https://www.abs.gov.au/statistics/detailed-methodology-information/concepts-sources-methods/labour-statistics-concepts-sources-and-methods/2021/methods-four-pillars-labour-statistics; Howell, Statistical Glossary, https://www.uvm.edu/~statdhtx/fundamentals9/Glossary/Glossary.html. These primary institutional and author references describe sampling precision. The sixteen-sample example is original and independently enumerated; the research case is fictional.
Previous: Standard deviation Next: Standard error of differences between means