Can a pairwise t test pass while the overall gate fails?

Yes, as the worked example shows.

Does protected LSD always control every family's error rate strongly?

No. Its general protection is limited, with special cases requiring their stated assumptions.

Should failed-gate estimates be hidden?

No. Estimates can be reported descriptively while keeping the inferential decision consistent with the planned procedure.

AI Glossary / Statistics and probability / Intermediate

Fisher's Least Significant Difference Test (LSD)

Definition

Fisher's Least Significant Difference Test, abbreviated LSD, is a pairwise mean-comparison procedure using a common error estimate from an analysis-of-variance model. In the source glossary's protected formulation, a significant overall F test is required before individual t comparisons are declared significant. The term overlaps with Fisher's least significant difference, which also names the numerical comparison threshold. This entry emphasizes the protected testing workflow and its interpretation. The preliminary gate provides limited protection and is not a general solution to multiple-testing error across arbitrary comparison families.

What it means

The procedure separates two questions: whether the group means collectively show evidence of differences, and which specific pairwise differences exceed an individual threshold. The second question uses the pooled ANOVA error variance. In a protected workflow, failure to pass the first question stops the planned pairwise significance decisions, even if a software table displays small unadjusted pairwise p-values. Reports should identify the gate and the comparisons it governs. Merely running ANOVA and then ignoring its result is an unprotected analysis, not the same procedure.

How it works

Choose a significance level and fit the appropriate ANOVA model. Test the overall null that all relevant means are equal. If the protected gate is passed, calculate each pair's t statistic using the mean difference divided by its model-based standard error. For independent groups with common variance, that error is sqrt[MSE(1/n_i + 1/n_j)] with the ANOVA residual degrees of freedom. Compare against the relevant individual t threshold. Under a complete null, the gate limits the chance of any pairwise rejection to no more than the overall-test level. Under partial null configurations, that reasoning no longer supplies a general strong family-wise guarantee.

Three practical examples

Legacy report interpretation: establish whether a claimed LSD analysis actually applied its stated overall-test gate.
Teaching: demonstrate how a protected decision can differ from an isolated pairwise t comparison.
Analysis planning: distinguish an overall question from a prespecified family of pairwise claims.

Think of it this way

Think of an overall checkpoint followed by individual inspections. Failing the checkpoint means the protected procedure does not proceed to individual declarations. Passing it does not make every later inspection immune to false alarms, especially when many comparisons are still being considered.

Worked example

Consider a hypothetical one-way study with three groups of six observations, means 10, 11.5 and 13, and residual mean square four. The grand mean is 11.5 and treatment sum of squares is 6(2.25 + 0 + 2.25) = 27. With two treatment and 15 error degrees of freedom, F = (27/2)/4 = 3.375. Its upper-tail probability is approximately 0.06162, so a protected LSD procedure at 0.05 does not pass the overall gate. Yet the extreme pair differs by three with standard error sqrt[4(1/6 + 1/6)] = 1.1547, giving t = 2.5981. This exceeds the individual two-sided 5% threshold of about 2.13145 with 15 degrees of freedom. The isolated pairwise result would pass, but the specified protected workflow does not declare it significant. The example illustrates procedural consistency, not proof that the pair's population means are equal.

Illustrative case study

Fictional case: Sycamore Study Review compares a written analysis plan with its software output. The plan specifies protected LSD, while the program prints pairwise tests regardless of the overall result. Analysts retain the displayed estimates but apply and explain the planned gate when reporting significance decisions. They also describe the method's multiplicity limits rather than implying all intervals are simultaneous. This invented case illustrates faithful reporting of a prespecified procedure and does not recommend choosing a different gate after seeing which comparisons are attractive.

Common misconceptions and mistakes

Do not confuse a program's printed pairwise table with proof that a protected procedure was followed. Do not claim non-rejection means equivalence. The complete-null protection argument does not establish strong control when some groups genuinely differ. A special three-group all-pairs result exists under the classical assumptions, but it should not be extended casually to larger families or different designs. The simple pooled-error calculation also needs modification for dependence, unequal variances or model-specific contrasts. Statistical significance says nothing by itself about the size of an economically meaningful benefit.

Business use

Businesses should preserve the analysis protocol and distinguish descriptive estimates from gated significance declarations. If simultaneous conclusions are required across many options, select a method designed for that objective. State whether the analysis is confirmatory or exploratory and report effect sizes alongside the testing result. When reproducing older work, identify the exact protected or unprotected variant and its assumptions. Transparent interpretation is more useful than treating a familiar test name as a guarantee of rigorous multiplicity control.

Check your understanding

In the worked example, what is the protected decision at alpha 0.05?

Answer: The omnibus p-value exceeds 0.05, so the procedure does not proceed to pairwise significance declarations, despite the isolated extreme-pair t statistic.

Related terms

Fisher's least significant difference
Pooled variance
protected t test
omnibus F test
pairwise comparison
weak family-wise control
strong family-wise control

Learn next

distinguish complete-null from partial-null error control, then compare the protected LSD workflow with a simultaneous all-pairs procedure using the same group means and residual model.

Sources

Sources: University of Vermont, Fisher's Least Significant Difference Test (LSD), protected formulation: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html ; GraphPad, protected and unprotected LSD distinction: https://www.graphpad.com/guides/prism/latest/statistics/stat_fishers_lsd.htm ; Penn State, ANOVA Foundations: https://online.stat.psu.edu/stat502/Lesson02 ; Curtin, categorical-variable models and protected-testing limitations: https://jjcurtin.quarto.pub/13_categorical_variables/ .

Previous: Fisher's least significant difference Next: Fisher's method