How many pairs are there for k groups?

There are k(k-1)/2 distinct pairs.

Is HSD limited to comparing the largest and smallest means?

No; it addresses all pairwise differences.

Can unequal sample sizes be handled?

Tukey-Kramer extends the common-variance approach, with different properties from the balanced case.

AI Glossary / Statistics and probability / Intermediate

Tukey's HSD test

Definition

Tukey's HSD test, short for honestly significant difference, is a procedure for comparing every pair of group means while controlling the familywise probability of one or more false rejections under its model assumptions. It uses the studentized range distribution and is commonly associated with a one-way analysis of variance.

What it means

With several groups, an overall comparison does not identify which specific means differ. Testing each pair separately at an unadjusted significance level increases the chance of at least one false positive across the collection.
Tukey's HSD addresses the family of all pairwise mean comparisons. It can also produce simultaneous confidence intervals for those differences. The protection concerns that defined family, not every analysis that might later be performed on the same data.

How it works

Specify the groups and the outcome, then fit an appropriate model. In the standard balanced one-way setting, errors are independent and normally distributed with a common variance, and each group has the same sample size.
Estimate the common error variance using the within-group residual mean square, MSE. With k groups and total sample size N, the residual degrees of freedom are N-k in the simple one-way model.
Obtain the studentized-range critical value q for the chosen familywise significance level, k means, and the residual degrees of freedom. For equal group sizes n, the honestly significant difference is q times the square root of MSE/n.
Compare each absolute mean difference with that common threshold. Equivalently, form simultaneous intervals by adding and subtracting the HSD from each signed mean difference. An interval excluding zero corresponds to rejection for that pair at the chosen familywise level.
Do not confuse the scaling with an ordinary two-sample t statistic. The studentized-range formulation uses the standard error of a mean in the balanced expression, not the standard error of a difference. The distribution and scaling are designed together.
For unequal sample sizes under a common-variance model, the Tukey-Kramer adjustment uses a pair-specific scale, square root of [MSE/2 times (1/ni + 1/nj)]. Heterogeneous variances call for a method justified for that setting rather than blindly applying the pooled-variance formula.
Interpret the familywise guarantee conditionally on the assumptions and analysis specification. Independence can fail with repeated measurements or clustered data; the number of rows does not automatically equal the number of independent units.
Report effect sizes and uncertainty, not only significance labels. A statistically detectable mean difference may be operationally minor. A pair whose interval contains zero has not been demonstrated equal or equivalent. Equivalence requires a separately defined margin and analysis.

Three practical examples

A manufacturer compares average output under several independently assigned production settings.
A training evaluation compares mean assessment scores across three course formats under an appropriate randomized design.
A service experiment compares several interface variants when all pairwise differences are of interest.

Think of it this way

Think of inspecting every pair of contestants while allowing a fixed overall chance of sounding at least one false alarm. The threshold accounts for the entire set of pairwise checks, rather than pretending that each check is the only one conducted.

Worked example

Consider an original balanced example with three groups of five measurements. Group A is 7, 9, 10, 11, 13; B is 9, 11, 12, 13, 15; C is 13, 15, 16, 17, 19. Their means are 10, 12, and 16.
Each group has within-group squared deviations summing to 20, so total residual sum of squares is 60. With N = 15 and k = 3, residual degrees of freedom are 12 and MSE = 60/12 = 5.
For a 5% familywise level, the studentized-range table gives q approximately 3.773 for three groups and 12 residual degrees of freedom. Since square root of MSE/n equals square root of 5/5 = 1, the HSD is approximately 3.773.
The absolute differences are 2, 6, and 4. A versus B does not exceed the threshold; A versus C and B versus C do. The simultaneous interval for C minus B is 4 plus or minus 3.773, approximately [0.227, 7.773]. These conclusions assume the independent normal equal-variance model; the invented values do not themselves verify those assumptions.

Illustrative case study

Fictional case: Northfield Packaging compares three machine settings using independently randomized production runs. The team reports all pairwise mean differences with Tukey intervals and distinguishes a statistically detectable improvement from the minimum gain needed to cover switching costs.
When a later dataset contains repeated measurements from the same run, the analysts do not reuse the original calculation as though every reading were independent. They revise the model and comparison procedure to reflect the experimental structure.

Common misconceptions and mistakes

HSD is not Tukey's B method or Tukey's test of additivity. Its familywise level is not the probability that a particular reported difference is false. Nonsignificance does not prove equality. A significant omnibus ANOVA does not remove the need to account for multiple pairwise comparisons. The classical pooled-error method should not be assumed valid under arbitrary unequal variances.

Business use

Tukey's HSD helps businesses compare multiple options without treating every pairwise test as an isolated decision. Define the comparison family in advance, retain the model assumptions in the report, and present differences in practical units. If only comparisons with a control are needed, consider a procedure designed for that smaller family rather than automatically requesting every pair.

Check your understanding

Why does the worked example use 12 residual degrees of freedom?

Answer: There are 15 observations and three fitted group means.

Which comparison is not significant under the stated assumptions?

Answer: A versus B, whose difference is 2.

Does that establish equivalence?

Answer: No.

Related terms

Tukey's range test
Familywise error rate
Tukey-Kramer procedure
studentized range
ANOVA
simultaneous confidence interval

Learn next

Derive the pooled within-group mean square, then compare an ordinary pairwise interval with a Tukey simultaneous interval. Check how the comparison family changes the question.

Sources

NIST/SEMATECH, Tukey's method: https://www.itl.nist.gov/div898/handbook/prc/section4/prc471.htm SciPy, tukey_hsd documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.tukey_hsd.html MedCalc, studentized-range critical-value table: https://www.medcalc.org/en/manual/tukey-q-table.php University of Vermont glossary, Tukey's HSD test: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html

Previous: Tukey's B method Next: Tukey's range test