Why is it called analysis of variance if it compares means?

It uses a decomposition of variation to assess evidence about differences among means.

Can ANOVA compare two groups?

Yes. Under matching equal-variance independent-group assumptions, the one-way F test with two groups corresponds to the squared two-sided t statistic.

What if group variances differ?

Methods such as Welch's ANOVA may be appropriate for some independent-group settings. The choice should reflect the design and inferential question.

AI Glossary / Statistics and probability / Intermediate

Analysis of variance

Definition

Analysis of variance, or ANOVA, is a family of statistical methods that partitions variation in an outcome into components associated with modeled factors and residual variation. A common application tests whether population means are equal across groups by comparing between-group variation with variation within groups.

What it means

Despite its name, a standard one-way ANOVA is usually used to compare means rather than to test whether variances are equal. If group means differ substantially relative to the variation among observations within groups, the evidence may be inconsistent with an equal-means null hypothesis.

ANOVA also provides a framework for factorial experiments, interactions and other models. The appropriate analysis depends on how observations were collected and which effects are being estimated. A table of sums of squares does not by itself determine whether a group difference is causal or important in practice.

How it works

For a simple one-way model, identify the grouping factor and calculate each group mean and the overall mean. Decompose total squared deviations from the overall mean into a between-group sum of squares and a within-group, or residual, sum of squares.

Divide each component by its degrees of freedom to obtain mean squares. With k groups and N independent observations, the usual numerator degrees of freedom are k - 1 and the residual degrees of freedom are N - k. The F statistic is the between-group mean square divided by the residual mean square.

Under the classical equal-means null and appropriate independent, normal, equal-variance errors, this ratio has an F reference distribution. Assess assumptions using design knowledge and diagnostics, not just automatic tests. Unequal variances, repeated observations or clustering can require a different method. If the overall test rejects, planned contrasts or suitable multiple-comparison procedures can address which group comparisons matter; the omnibus result alone does not identify them.

Three practical examples

1. A randomized manufacturing experiment compares mean output across several process settings while monitoring within-setting variability.

2. A training evaluation compares final scores across three programs, considering whether baseline adjustment or a more complex design is needed.

3. A factorial experiment studies two settings and their interaction, asking whether the effect of one setting changes across levels of the other.

Think of it this way

Think of comparing several baskets of measurements. If the basket averages are far apart while values within each basket are tightly clustered, the baskets appear more distinct than if every basket contains a wide mix. ANOVA formalizes that relative comparison under a statistical model.

Worked example

Consider an original dataset with group A values 1, 2, 3; group B values 4, 5, 6; and group C values 7, 8, 9. Their means are 2, 5 and 8, and the overall mean is 5.

The between-group sum of squares is 3(2 - 5) squared + 3(5 - 5) squared + 3(8 - 5) squared = 54. Each group has within-group squared deviations 1, 0 and 1, totaling 2. Across the three groups, the residual sum of squares is 6. Total sum of squares is therefore 54 + 6 = 60.

The between-group degrees of freedom are 2 and the residual degrees of freedom are 6. Mean squares are 54/2 = 27 and 6/6 = 1, producing F = 27. Under the classical F(2, 6) null reference, the upper-tail p-value is 0.001. This small example demonstrates the arithmetic. Whether its reference model and any causal interpretation are appropriate depends on how the measurements were generated and sampled.

Illustrative case study

Fictional case study: Meanworks compares three operating procedures in a randomized experiment. Analysts calculate an overall ANOVA, inspect residuals and report group means with uncertainty. They also evaluate prespecified contrasts relevant to the decision rather than treating every possible comparison as equally important.

One procedure has a higher mean, but the team examines implementation cost and the size of the improvement before deciding whether to adopt it. The report describes the randomization and experimental units so readers can assess the scope of the inference.

This fictional case illustrates ANOVA as one component of evaluation. It does not establish that a significant overall test guarantees a worthwhile operational change.

Common misconceptions and mistakes

A significant ANOVA does not mean every pair of groups differs. It indicates evidence against the overall equal-means hypothesis under the model. Repeated unadjusted pairwise tests can inflate the chance of false positive claims.

A nonsignificant result does not prove that means are equal or practically equivalent. The normality assumption concerns errors within the model, not necessarily the pooled outcome distribution. Also, observations from the same person, machine or cluster should not be treated as independent merely because they occupy separate spreadsheet rows. Independence is primarily a design issue.

Business use

ANOVA helps organizations evaluate multiple alternatives within a coherent model instead of relying on separate informal comparisons. It can show whether observed differences are large relative to residual variation and support focused follow-up questions.

Report group sizes, means, uncertainty, the model and relevant effect sizes alongside the F statistic and p-value. Preserve the distinction between the unit randomized and the unit measured. In unbalanced or multifactor analyses, specify the hypotheses being tested because different sums-of-squares conventions can answer different questions. For AI comparisons, account for shared test cases, repeated runs and other dependence; a standard independent-groups ANOVA may not match the evaluation design.

Check your understanding

An ANOVA has a between-group mean square of 12 and a residual mean square of 3. What is its F statistic?

Answer: F = 12/3 = 4. A p-value also requires the relevant numerator and denominator degrees of freedom and a justified reference distribution.

Related terms

Factor
Sums of squares
Contrast
Multiple comparisons
mean square
sum of squares
F test
interaction
Welch's ANOVA and analysis of covariance
specific hypotheses and extensions of the basic method

Learn next

reconstruct a one-way ANOVA table from a small dataset and check that its sums of squares and degrees of freedom reconcile. Then examine a planned contrast and explain how its question differs from the overall equal-means test.

Sources

Primary sources: NIST/SEMATECH, One-way ANOVA overview: https://www.itl.nist.gov/div898/handbook/prc/section4/prc431.htm

NIST/SEMATECH, One-Way ANOVA: https://itl.nist.gov/div898/handbook/ppc/section2/ppc231.htm

Penn State, STAT 200, One-Way ANOVA: https://online.stat.psu.edu/stat200/Lesson10

Previous: Analysis of similarities Next: Analytic survey