Does Dunnett compare every pair of groups?

No; its usual family compares each treatment with one control.

Yes, when the direction is justified and specified appropriately.

Are its p-values already adjusted?

Standard implementations return values adjusted for the specified treatment-control family; verify the software output.

AI Glossary / Statistics and probability / Intermediate

Dunnett's test

Definition

Dunnett's test is a multiple-comparison procedure for comparing several treatment means with one designated control mean. The original single-step procedure accounts for the dependence created by sharing that control and controls the family-wise probability of falsely declaring a treatment-control difference under its model assumptions. It is intended for a many-to-one comparison family. It does not automatically cover every comparison between treatments, and it is different from Duncan's multiple range test despite the similar names.

What it means

Suppose an organization tests three alternatives against its existing process. Its main questions may concern each alternative versus the existing process, rather than every possible pairing among the four groups. Dunnett's procedure matches that narrower objective. A common control supplies information to each comparison, so the resulting statistics are correlated even when observations in different groups are independent. Accounting for this structure avoids treating the comparisons as unrelated and can provide greater power than a procedure protecting a larger family of unnecessary comparisons.

How it works

Specify the control, treatments and direction of each inferential question before examining results. In the classical independent-groups model, pool within-group variation to estimate a common error variance. Divide each treatment-control mean difference by its standard error, using the sizes of both groups. Evaluate the collection through the relevant multivariate t distribution, whose correlation reflects the shared control. Obtain adjusted p-values or simultaneous confidence intervals. The original procedure assumes independent observations, normally distributed errors within groups and a common variance. Extensions exist, but software-specific alternatives should be identified rather than silently treated as the original method.

Three practical examples

Process trials: compare several proposed production settings with the established setting on a continuous response.
Interface experiments: evaluate prespecified variants against an existing design when the outcome model is suitable.
Research review: check whether a reported simultaneous confidence interval covers a treatment-control comparison or an unrelated treatment pair.

Think of it this way

Think of measuring several challengers against the same reference ruler. Uncertainty in that reference affects every comparison together. Dunnett's method recognizes that shared contribution while protecting the chosen set of claims, instead of pretending that each comparison used a completely separate reference.

Worked example

Consider a hypothetical control and two treatments, each with ten independent observations. Their means are 20, 22 and 23, and their pooled residual mean square is four, with 27 error degrees of freedom. Each treatment-control standard error is sqrt[4(1/10 + 1/10)] = sqrt(0.8), approximately 0.8944. The two t statistics are 2/0.8944 = 2.2361 and 3/0.8944 = 3.3541. The covariance of the two mean differences is the control mean's variance, 4/10 = 0.4, giving correlation 0.4/0.8 = 0.5. These calculations establish the inputs to Dunnett's joint reference distribution. They do not by themselves supply adjusted p-values or significance decisions; those require the multivariate t calculation, selected sidedness and family level. The treatment-to-treatment difference of one is outside this specified family.

Illustrative case study

Fictional case: Harbour Assembly compares two fixture designs with its current fixture. Before data collection, the team identifies the current design as the control and chooses a continuous measurement and comparison direction. Analysts inspect the model assumptions and report treatment-control estimates with simultaneous intervals. When a manager asks whether the two new fixtures differ from each other, the report explains that this is an additional question requiring its own justified analysis. This invented case illustrates matching comparisons to decisions, without claiming that a statistical difference guarantees a profitable change.

Common misconceptions and mistakes

Do not confuse independent observations with independent comparison statistics: the shared control induces dependence. Do not apply the original equal-variance procedure automatically to severely unequal group variances. Non-rejection does not prove equivalence or acceptable performance. A significant overall ANOVA is not an intrinsic prerequisite for a correctly planned Dunnett procedure, although a study protocol may impose a gatekeeping rule. Choosing the best-looking group as the control after observing outcomes changes the selection problem. Repeated observations from the same people require a model that accounts for that dependence.

Business use

Businesses can use the procedure when the existing service or process is the meaningful reference and several alternatives are being evaluated. Define which differences would justify implementation, then interpret simultaneous intervals alongside costs and operational constraints. Preserve the control definition, comparison family and error model in the analysis record. If the decision requires ranking every alternative against every other, a many-to-one procedure alone is insufficient. Statistical protection applies to the modeled comparisons, not to unreported endpoints, repeated peeking or additional exploratory claims.

Check your understanding

Why are two treatment-control differences correlated when all three groups are independently sampled?

Answer: Both differences subtract the same estimated control mean, so uncertainty in that mean is shared.

Related terms

Multiple comparisons
Control group
Pooled variance
simultaneous confidence interval
family-wise error rate
Tukey's test
Duncan's test

Learn next

compare the hypothesis families protected by Dunnett and Tukey procedures, then examine how unequal variances or repeated measurements change the required model and covariance calculation.

Sources

Sources: SciPy, dunnett documentation, original single-step method and assumptions: https://cache-docs.scipy.org/doc/scipy/reference/generated/scipy.stats.dunnett.html ; SAS, Multiple Comparisons, comparisons with a control and one-sided options: https://support.sas.com/documentation/cdl/en/statug/67523/HTML/default/statug_glm_details29.htm ; Dunnett, original 1955 paper: https://doi.org/10.1080/01621459.1955.10501294 .

Previous: Dunn-Sidák test Next: Dunning-Kruger Effect Bias