Must the original data be normal?

No. The classical theorem allows many nonnormal distributions, provided its assumptions hold.

Is a sample size of thirty always sufficient?

No. Approximation quality depends on the distribution and the probability or precision being assessed.

Does the theorem correct biased sampling?

No. A well-behaved sampling distribution around the wrong population quantity does not remove selection bias.

AI Glossary / Mathematics and optimization / Intermediate

Central limit theorem

Definition

The central limit theorem explains why suitably standardized sums or averages often have distributions close to a normal distribution. In its classical form, it applies to independent, identically distributed observations with a finite mean and a finite, positive variance. The approximation improves in a limiting sense as the number of observations increases.

What it means

The theorem concerns the distribution of a sum or average across repeated samples. It does not say that the individual observations become normally distributed as more data are collected.

For example, a yes/no response remains binary however many people respond. Nevertheless, the proportion of yes responses across repeated sufficiently large independent samples can have an approximately bell-shaped distribution. That distinction is central to interpreting statistical estimates and their uncertainty. This entry emphasizes the classical independent, identically distributed version; extensions require their own conditions.

How it works

Suppose each observation has mean mu and variance sigma squared. For n independent observations with the same distribution, their sum has mean n×mu and variance n×sigma squared. Their average has mean mu and standard deviation sigma/√n, called its standard error.

Subtract the mean from the sum and divide by sigma×√n. The central limit theorem says that the resulting distribution approaches the standard normal distribution. Equivalently, standardize the average by subtracting mu and dividing by sigma/√n.

This gives a practical approximation, not an automatic finite-sample guarantee. Skewness, rare large observations, and the probability region being examined affect its usefulness. A fixed rule such as “thirty observations is enough” cannot cover every distribution. Dependence, selection bias, and changing data-generating processes need separate attention.

Three practical examples

1. Model evaluation: approximate the sampling uncertainty of average per-example losses when the observations and losses meet suitable assumptions.

2. Quality monitoring: model the sampling distribution of an average measured deviation across independently sampled items, even when individual deviations are not normally distributed.

3. Simulation: estimate uncertainty in a Monte Carlo average from independent runs with finite variance. Correlated runs require an analysis that accounts for that correlation.

Think of it this way

Think of many independent small contributions combining into an aggregate whose fluctuations have a regular overall shape. The pattern concerns the combined result across repetitions, rather than transforming each contribution into a bell curve.

The analogy fails if a shared influence makes the contributions move together or if unusually large contributions dominate in a way that violates the needed assumptions. “Many” alone is not the whole explanation.

Worked example

Consider independent yes/no observations with probability 0.4 of yes. Encode yes as 1 and no as 0. Each observation has mean 0.4 and variance 0.4×0.6=0.24.

For n=600, the sample proportion has standard error √(0.24/600)=√0.0004=0.02. Under the normal approximation, about 95% of repeated sample proportions lie within 1.96 standard errors of 0.4: approximately 0.3608 to 0.4392.

For n=2,400, the standard error is √(0.24/2400)=0.01. The corresponding approximate range is 0.3804 to 0.4196. Quadrupling the sample size halves the standard error; it does not reduce it by a factor of four.

These are illustrative repeated-sampling ranges around a specified true probability, not confidence intervals calculated from an observed sample. The proportions remain discrete, so the normal approximation is not exact. If the observations come from correlated groups, the independence-based standard errors can be misleading even though the total row count is large.

Illustrative case study

Fictional case: an AI testing team collects thousands of question variants generated from a small number of underlying templates. Its first report treats every variant as an independent test when estimating uncertainty in average accuracy.

A reviewer identifies that errors within a template often occur together. The team reorganizes the evaluation around independent sampling units and seeks an uncertainty method appropriate to that structure. The lesson is not that a large evaluation is unhelpful: the effective information depends on how examples were produced. The classical theorem cannot justify ignoring their dependence.

Common misconceptions and mistakes

Do not confuse the central limit theorem with the law of large numbers. The latter concerns an average approaching its expected value; the former describes the limiting shape of appropriately scaled fluctuations.

Do not claim that more observations remove systematic sampling bias. Do not assume that an approximately normal mean implies normally distributed raw data. Also distinguish a standard deviation of observations from the standard error of their average. Finally, extremely small tail probabilities can require more careful methods than an ordinary normal approximation.

Business use

The theorem supports common ways of discussing uncertainty in experiments, evaluations, and operational averages. Its practical contribution is connecting variability, sample size, and the precision of an estimate.

Decision makers should ask what was independently sampled, whether the process remained stable, and whether rare large outcomes affect the estimate. A narrow uncertainty interval conditional on a flawed sample can still misrepresent the intended population. Reporting these assumptions is part of explaining what a numerical result means.

Check your understanding

An average has standard error 0.06 from 100 independent observations. Under the same variance assumptions, what happens at 900 observations?

Answer: The sample size increases ninefold, so the standard error falls by √9=3, giving 0.02. This scaling alone does not verify independence or approximation quality.

Related terms

Learn next

compare the law of large numbers with the central limit theorem, then distinguish a sampling range from a confidence interval. Explore how clustered observations change uncertainty calculations before applying independent-sample formulas to repeated users or templates.

Sources

MIT, Introduction to Probability, Lecture 19, central limit theorem and normal approximation: https://ocw.mit.edu/courses/res-6-012-introduction-to-probability-spring-2018/0f6325b76d575e9ed438b871203ab4c0_MITRES_6_012S18_L19.pdf . Aalto Dictionary of Machine Learning, central limit theorem entry: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Relevant theorem statements were read. The sample-size calculation and fictional testing case are original.

Previous: Central composite design Next: Central moment