Is this the same as Bonferroni?

No; their threshold formulas differ, although the numerical results are often close.

Does it always require independence?

Independence is sufficient, while some other dependence settings also permit validity; arbitrary dependence needs care.

Is Holm-Sidak the same calculation?

No. It is a step-down procedure with different sequential thresholds.

AI Glossary / Statistics and probability / Intermediate

Dunn-Sidák test

Definition

The Dunn-Sidák test is a multiple-testing adjustment that assigns a smaller significance threshold to each test in a specified family. In its single-step form, the threshold is 1 - (1 - alpha)^(1/m), where alpha is the intended family-wise significance level and m is the number of tests. It is closely related to the Bonferroni adjustment, but uses a multiplicative probability calculation. Its validity depends on the joint behavior of the tests; independence provides a straightforward sufficient setting.

What it means

Running several tests creates several opportunities for a false positive. A family-wise procedure addresses the probability of rejecting at least one true null hypothesis within the defined family. Dunn-Sidák is commonly used as another name for the Sidak adjustment in this context. It does not calculate whether a scientific hypothesis is probably true, and it does not remove bias from the underlying study. Analysts must define the comparison family and justify the adjustment's assumptions before interpreting its output. The family might contain several endpoints, contrasts or planned comparisons.

How it works

First obtain valid raw p-values for the specified hypotheses. For m tests, compare each raw p-value with the common adjusted threshold, or calculate its adjusted value as 1 - (1 - p)^m and compare that with alpha. With independent uniform null p-values, the probability of no false rejection across m true nulls equals the product of their individual non-rejection probabilities. This gives the threshold formula. Certain dependence conditions also support validity, but arbitrary dependence should not be assumed harmless. Single-step Sidak and step-down Holm-Sidak are different procedures; record the exact method selected in the software.

Three practical examples

Experimental reporting: adjust a prespecified family of endpoint tests where the necessary probability assumptions are justified.
Method comparison: examine the small numerical difference between Sidak and Bonferroni thresholds for the same family.
Analysis review: identify whether software reports raw p-values, single-step adjusted values or step-down results.

Think of it this way

Think of allocating a shared false-alarm allowance across several alarms. The allocation accounts for how opportunities accumulate. However, the multiplication rule relies on how the alarms relate to one another; merely counting alarms does not establish their independence or validate the chosen allocation.

Worked example

Consider four hypothetical independent tests with valid continuous null p-values and a family-wise level of 0.05. The single-step threshold is 1 - 0.95^(1/4), approximately 0.01274146. A raw p-value of 0.012 passes this threshold, while 0.013 does not. The adjusted value for 0.012 is 1 - 0.988^4 = 0.04714289, giving the same decision. Bonferroni would use 0.05/4 = 0.0125, which is slightly smaller. If all four nulls hold under the stated independence assumptions, the probability of at least one rejection is 1 - (1 - 0.01274146)^4, approximately 0.05. This is a property of the testing procedure under those assumptions, not a 5% posterior probability that a reported finding is wrong.

Illustrative case study

Fictional case: Elm Interface Research prepares four planned endpoint tests for a prototype evaluation. Its statistician reviews their dependence before selecting a multiplicity procedure and labels the method precisely in the report. A reviewer notices that another analysis used Holm-Sidak and asks for the results to be distinguished. The team retains the original family definition and explains any sensitivity analysis separately. This invented case illustrates method transparency, without implying that interface endpoints are normally independent or that Sidak is automatically the appropriate choice.

Common misconceptions and mistakes

Do not treat the slightly larger Sidak threshold as a free improvement over Bonferroni under every dependence pattern. Do not reduce m after seeing which comparisons look promising, or apply an adjustment only to selected significant results. Changing the family changes the claim being protected. Do not interpret non-rejection as equivalence or absence of an effect. Multiplicity correction cannot repair invalid p-values, outcome selection or model misspecification. The adjustment is also distinct from controlling the false discovery rate, which concerns a different error criterion.

Business use

A business comparing several options should define its decision questions before calculating adjusted results. Record the comparison family, sidedness, adjustment method and supporting assumptions alongside effect estimates and uncertainty. The modest difference between two thresholds may matter for a borderline result, but practical importance should not be reduced to that boundary. Where dependence is uncertain, evaluate an approach whose guarantees fit the setting. Keeping analysis choices explicit helps later reviewers distinguish a planned inference from an exploratory lead that needs further evidence.

Check your understanding

For four tests at family level 0.05, is a raw p-value of 0.013 rejected by single-step Sidak?

Answer: No. It exceeds approximately 0.01274146; comparing it with an unadjusted 0.05 would answer a different question.

Related terms

Multiple comparisons
Bonferroni correction
family-wise error rate
adjusted p-value
Holm-Sidak procedure
statistical power

Learn next

derive the probability of at least one false rejection for independent tests, then compare family-wise and false-discovery-rate objectives using the same predefined hypothesis family.

False discovery rate

Sources

Sources: SAS, p-Value Adjustments, Sidak formula and dependence conditions: https://support.sas.com/documentation/cdl/en/statug/66859/HTML/default/statug_multtest_details11.htm ; SAS, PROC MULTTEST Statement, single-step and step-down options: https://support.sas.com/documentation/cdl/en/statug/66103/HTML/default/statug_multtest_syntax01.htm ; University of Vermont, statistical glossary terminology: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html .

Previous: Duncan's new multiple range test Next: Dunnett's test