Is variance estimation the same as calculating sample variance of outcomes?

Not generally; the target is often an estimator's sampling variance.

Is standard error its square root?

Yes, when based on the same variance estimate.

Does a bootstrap always work?

No; the resampling scheme and regularity conditions matter.

AI Glossary / Statistics and probability / Intermediate

Variance estimation

Definition

Variance estimation is the task of estimating the variability of a statistical estimator across repeated samples or repeated realizations of its data-generating process. In survey statistics, it commonly means estimating the sampling variance of a mean, total, proportion or other estimate under the actual sampling design. It is distinct from merely describing the spread of individual observations. The square root of an estimated variance is an estimated standard error.

What it means

A population estimate would generally change if a different valid sample were drawn. Variance estimation quantifies that repeat-sampling variation using the observed data and assumptions. It helps support confidence intervals, tests and comparisons of precision. The result is not the probability that the point estimate is correct, nor does it automatically capture systematic errors such as undercoverage or measurement bias. A precise estimate can still target the wrong population or be consistently distorted.

How it works

Choose a method appropriate to the estimator and design. Simple settings have analytical formulas, such as sample variance divided by sample size for an independent-sample mean. Sampling without replacement from a finite population introduces a finite population correction. Complex surveys may use Taylor linearization or replication methods, including jackknife, bootstrap or balanced repeated replication, with design-appropriate weights and units. Resampling individual rows indiscriminately can fail when the design includes clusters, strata or unequal probabilities. The variance method should reproduce the important features of how the estimate was obtained.

Three practical examples

Example 1: A survey estimates uncertainty around the percentage of employees using AI tools. Example 2: An evaluation estimates the variance of a mean task score while accounting for repeated runs within tasks. Example 3: A business estimates uncertainty in a ratio using linearization or a suitable replication procedure. Each concerns the variability of an estimator, not only the raw outcome distribution. The correct approach can differ even when the final reported statistic looks similar.

Think of it this way

Think of repeatedly drawing different valid samples and watching the estimate move around. Its variance describes the spread of those estimates. In practice, you usually see only one sample, so a variance estimator tries to learn that repeat-sampling spread from the information available. The quality of that attempt depends on respecting the actual sampling process.

Worked example

Consider a finite population with values zero, two, four and six, whose mean is three. Draw two values by simple random sampling without replacement. The six possible sample means are one, two, three, three, four and five. Their sampling variance around three is (4 + 1 + 0 + 0 + 1 + 4)/6 = 5/3. For any observed sample, estimate the mean's variance as (1 minus n/N) times s squared divided by n, with n = 2, N = 4 and s squared using divisor n minus one. Across the six samples, these variance estimates average to 5/3, matching the true sampling variance. Individual estimates range from 0.5 to 4.5, showing that an unbiased variance estimator can itself vary substantially in a small sample.

Illustrative case study

In this fictional case study, Poplar Insights surveys employees through selected branches. The initial report divides the outcome sample variance by the number of respondents as though all employees were independently sampled. A design review identifies branch clustering and unequal selection probabilities. Analysts use the supplied design information to revise the uncertainty calculation and document the estimator. They also discuss nonresponse separately because a corrected sampling variance does not automatically remove possible response bias.

Common misconceptions and mistakes

Common mistakes include confusing observation variance with estimator variance, ignoring finite population sampling when it matters, and bootstrapping rows from clustered data without preserving the relevant structure. Another mistake is assuming a standard error captures all uncertainty. Model choices, measurement error, coverage and nonresponse may require additional analysis. Variance estimates can be unstable with few independent groups, and more rows within existing groups do not necessarily provide the same information as more independently sampled groups.

Business use

Businesses should request both point estimates and a documented uncertainty method. Preserve design variables and replicate weights when they are supplied, and verify that the analysis software uses them correctly. Compare precision in relation to the decision, not only whether an interval is narrow. If an estimate combines several data sources or fitted stages, consider whether the variance calculation reflects that process. Explain limitations clearly so that a small standard error is not mistaken for comprehensive evidence of accuracy.

Check your understanding

What is the true sampling variance of the mean in the example?

Answer: 5/3.

Why use a finite population correction?

Answer: Sampling is without replacement from a population of four.

Do all six samples give the same estimated variance?

Answer: No.

What does their average equaling 5/3 demonstrate?

Answer: Unbiasedness of this variance estimator under the specified sampling design, not certainty for any single sample.

Related terms

Standard error
Bootstrap
Related terms include sampling variance
finite population correction
Taylor linearization
jackknife and replicate weights

Learn next

Learn next how stratification and clustering change the variance of survey estimates. Compare analytical and replication approaches on a small design where every possible sample can be enumerated. Then examine why uncertainty from a fitted model or ratio may require more than a simple outcome variance divided by sample size.

Sample size

Sources

Source: OECD, Glossary of Statistical Terms, Variance estimation, https://www.oecd.org/content/dam/oecd/en/publications/reports/2008/09/oecd-glossary-of-statistical-terms_g1gh9ad7/9789264055087-en.pdf. This primary institutional source describes the target and broad methodological approaches. The six-sample finite-population example is original and exhaustively checked. The branch-survey case is fictional, and bias is explicitly distinguished from sampling variance.

Previous: Variance components Next: Variance inflation factor ( VIF )