Is this only a normality test?

No. It can assess other distributional hypotheses, provided a suitable reference procedure is available.

Can the same critical value be used for every application?

No. Parameter fitting, corrections and the null specification must match the reference calculation.

Does a small p-value identify the exact problem?

No. It indicates disagreement with the null under the test assumptions. Plots and further analysis are needed to understand the nature and relevance of the departure.

AI Glossary / Statistics and probability / Intermediate

Anderson-Darling test

Definition

The Anderson-Darling test is a goodness-of-fit test that compares an empirical distribution with a specified distribution or fitted distribution family. Its statistic emphasizes discrepancies in the tails. Large values indicate greater disagreement, and the appropriate reference calculation depends on the null model and how any parameters were estimated.

What it means

A distributional assumption concerns the whole pattern of observations, including extreme values, rather than only the mean and variance. The Anderson-Darling statistic measures how far the empirical cumulative distribution departs from the hypothesized cumulative distribution, giving relatively strong weight to tail differences.

A small statistic or nonsignificant result does not prove the model is true. It indicates that the test has not found sufficiently strong evidence against it under the chosen procedure. Sample size matters: a small sample may miss important departures, while a large sample may detect discrepancies too small to matter for the intended application.

How it works

Sort the observations and evaluate the null cumulative distribution function, F, at each ordered value. A standard unadjusted statistic is A-squared = -n - (1/n) times the sum, for i from 1 to n, of (2i - 1)[log F(x-i) + log(1 - F(x-(n+1-i)))]. Here x-i denotes the ith ordered observation, not subtraction from x.

Use a matching critical value or p-value method. A fully specified continuous distribution is different from a family whose parameters were estimated from the same sample. Fitting changes the null behavior, and some procedures also apply sample-size corrections to the statistic. Mixing a corrected statistic with an incompatible reference table can invalidate the conclusion.

Check sampling assumptions and data features such as dependence, ties, censoring or truncation. A continuous-data reference procedure should not be transferred automatically to rounded or discrete observations. Simulation-based calibration can be useful when it reproduces the complete null fitting and data-generation procedure. Inspect probability plots and substantive consequences alongside the test result.

Three practical examples

1. A process analyst evaluates whether a proposed distribution adequately describes independent measurements before using it for tail-probability calculations.

2. A simulation developer checks generated values against a fully specified target distribution, documenting the test and any estimated parameters.

3. A researcher compares distributional diagnostics for model errors, recognizing that fitted residuals require attention to their dependence and estimation rather than blindly treating them as an independent raw sample.

Think of it this way

Think of comparing two cumulative profiles while paying special attention to their ends. A small mismatch near the middle and a mismatch near an extreme need not receive the same weight. That emphasis makes the test useful for some tail-sensitive questions, but it does not replace direct examination of the extremes.

Worked example

Consider an original arithmetic example with three observations, 0.2, 0.5 and 0.8, and a fully specified uniform distribution on zero to one. Because F(x) = x within that interval, the ordered cumulative probabilities are the same three numbers.

The weighted log sum is 1[log(0.2) + log(0.2)] + 3[log(0.5) + log(0.5)] + 5[log(0.8) + log(0.8)]. Using natural logarithms, the terms are approximately -3.2189, -4.1589 and -2.2314, totaling -9.6092.

The unadjusted statistic is therefore -3 - (-9.6092/3), approximately 0.2031. This calculation supplies the statistic only. It does not supply a p-value, prove uniformity or justify using critical values for a fitted normal distribution. A valid significance assessment must match the specified uniform null and the relevant finite-sample or other justified reference procedure.

Illustrative case study

Fictional case study: Tailcheck evaluates a proposed distribution for manufacturing measurements. The analysts calculate an Anderson-Darling result and inspect a probability plot. They notice that a few extreme observations are poorly represented by the model, which matters for a planned tolerance assessment.

They investigate measurement errors and alternative models without deleting inconvenient values automatically. The report explains whether parameters were fitted and which reference procedure was used. It also distinguishes statistical evidence of mismatch from the operational impact of that mismatch.

This fictional case illustrates combining a formal test with model diagnostics. It does not establish that the distribution with the largest p-value is the best predictive or engineering model.

Common misconceptions and mistakes

The Anderson-Darling statistic is not a p-value and is not constrained to the interval zero to one. A nonsignificant result does not establish normality or any other distributional form. Testing several distributions and selecting the most favorable result also requires care in interpretation.

The one-sample test described here is distinct from Anderson-Darling variants comparing multiple samples. Software implementations can differ in supported families, fitting methods, corrections and p-value calculations. Read the documentation for the procedure actually used rather than transferring a cutoff from another implementation or distribution.

Business use

The test can help assess assumptions used in forecasting, reliability analysis, simulation and other statistical workflows. Its tail emphasis is relevant when rare large or small values influence decisions, but no single goodness-of-fit statistic determines whether a model is fit for purpose.

Document the data unit, sample size, null distribution, parameter-estimation method and calibration procedure. Evaluate the practical effect of departures on the intended prediction or decision. If observations are dependent or selected, choose an analysis that represents that process. For AI-generated or transformed data, check whether preprocessing has altered the distribution or introduced ties before applying an off-the-shelf test.

Check your understanding

A program reports an Anderson-Darling statistic of 1.2. Can that number be interpreted as a 120% probability?

Answer: No. It is a discrepancy statistic. A matching reference procedure is needed to obtain a p-value or decision threshold.

Related terms

Goodness-of-fit
Cumulative distribution function
Kolmogorov-Smirnov test
empirical distribution function
tail behavior
probability plot and parametric bootstrap

Learn next

compute the statistic for a small fully specified distribution, then compare it with a case in which parameters are fitted. Explain why the same observations can require a different null reference after estimation.

Sources

Primary sources: NIST/SEMATECH, Anderson-Darling Test: https://www.itl.nist.gov/div898/handbook/eda/section3/eda35e.htm

SciPy documentation, scipy.stats.anderson: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.anderson.html

Previous: AND-parallelism Next: Angelic nondeterminism