Is the counternull always twice the estimate?

No; that rule applies to particular symmetric setups with a zero null on the analysis scale.

Can there be more than one counternull value?

Yes; general and randomization-based settings can produce sets of values rather than a unique reflected point.

AI Glossary / Statistics and probability / Advanced

Counternull

Definition

A counternull is a nonnull effect value that has the same evidential standing as a specified null value under a stated statistical procedure, commonly meaning that testing it produces the same p-value. It helps show why failing to reject a null hypothesis does not establish that the effect is zero. Its calculation depends on the model and effect scale.

What it means

A study can produce an uncertain effect estimate. If zero is not rejected, another effect value on the other side of the estimate may be equally compatible by the selected test. Reporting that value encourages attention to effect magnitude instead of treating a nonsignificant result as a finding of no effect.
The counternull is not a second point estimate, a confidence limit, or a probability that an alternative is true. Equal p-values are not posterior probabilities. It supplements the estimate and uncertainty; it does not replace study design.

How it works

Specify the estimand, null value, effect scale, and inferential procedure. Identify what equivalent evidence means in that procedure. A common introductory setting is a symmetric normal location model with a standard error held fixed while different hypothesized effect values are tested.
In that setting, a two-sided test depends on the absolute distance between the estimate and the hypothesized value. Reflecting the null across the estimate gives the same distance. If the estimate is theta-hat and the null is theta_0, the reflected value is theta_c = 2 theta-hat - theta_0. When theta_0 is zero, this becomes twice the estimate.
This doubling rule is not universal. Asymmetric reference distributions, parameter-dependent standard errors, constrained parameters, or nonlinear effect scales can require a different calculation. If the analysis is symmetric on a logarithmic scale, reflection must be performed on that scale and then transformed back. For example, an estimated ratio of 1.2 with null ratio one has reflected ratio 1.44 under the relevant fixed-standard-error log-scale setup, not 2.4.
Randomization-based procedures can yield counternull sets rather than one unique scalar value. Discrete distributions and the specification of alternative hypotheses affect the calculation. State the actual method rather than presenting a generic formula as exact in every design.
Report the counternull alongside the original estimate, uncertainty interval, null test, and a meaningful effect threshold. If the estimate coincides with the null in the simple symmetric case, reflection returns the same value and does not create a distinct nonnull alternative.

Three practical examples

An analyst explains why a nonsignificant pilot study does not demonstrate that a new workflow has no effect.
A research reviewer examines effect magnitudes alongside p-values when assessing whether a statistically detectable result matters operationally.
An AI evaluation team uses a counternull illustration to communicate uncertainty about a measured performance difference without claiming either candidate effect is proven.

Think of it this way

Think of two points placed equally far on opposite sides of a balance point. In a symmetric test, the observed estimate is the balance point, and the null and counternull receive the same test-based assessment. Equal distance does not tell you that either point is the true destination.

Worked example

Suppose an illustrative effect estimate is three score points and its known standard error is two points. Assume the estimator follows a normal distribution centered on the true effect, with this fixed standard error. The null hypothesis is an effect of zero.
The standardized statistic is (3 - 0)/2 = 1.5. Its two-sided normal p-value is approximately 0.133614. The reflected counternull is 2 x 3 - 0 = 6 points. Testing six instead gives (3 - 6)/2 = -1.5, with the same two-sided p-value.
The normal likelihood at zero and six is also equal in this particular fixed-variance model, because both are three points from the observed estimate. Each likelihood is exp(-1.125), approximately 0.32465, times the likelihood at the estimate. This likelihood ratio is not a probability assigned to either hypothesis.
An illustrative 95% interval is 3 plus or minus 1.96 x 2, or -0.92 to 6.92 points. Both zero and six lie inside it, while the point estimate remains three. The example demonstrates uncertainty, not evidence that the true effect is equally likely to be exactly zero or exactly six, and not a claim that six is a confidence bound.

Illustrative case study

Fictional case: Acorn Learning runs a small pilot of an AI-assisted lesson format. Its report says the result is not statistically significant and concludes that the formats are equivalent. A reviewer asks for the estimated difference and uncertainty rather than only the significance label.
Using an appropriate symmetric approximation, the analyst adds a counternull illustration and compares it with the organization's minimum meaningful improvement. Managers see that a potentially useful effect has not been ruled out. They consider further evidence while keeping the original estimate and interval central. The team does not present the counternull as a promised improvement or a replacement for an equivalence analysis.

Common misconceptions and mistakes

Do not double every effect estimate without checking the scale and inferential model. Do not interpret the counternull as the most likely effect, an upper bound, or evidence of practical equivalence. Equal p-values do not imply equal posterior probabilities. The same two-sided assessment can involve test statistics with opposite signs, so the reasoning should not be transferred blindly to a fixed-direction one-sided test.

Business use

The concept can improve discussion of uncertain experiments and pilot projects by exposing the gap between insufficient evidence against zero and evidence of negligible effect. Explain it in operational units. Decisions still need costs, meaningful-effect thresholds, uncertainty intervals, and a design that supports the intended interpretation. For equivalence or noninferiority questions, use methods explicitly designed for those questions.

Check your understanding

What is the counternull in the worked normal example?

Answer: Six points.

Why is its two-sided p-value the same as zero's?

Answer: Both hypotheses are 1.5 standard errors from the estimate.

Does this establish equivalence between the treatments?

Answer: No; a nonsignificant difference and a counternull calculation do not constitute an equivalence test.

Related terms

Learn next

Plot the two-sided p-value as a function of the hypothesized effect in a simple normal model. Then compare reflection on raw and logarithmic scales and study how randomization-based counternull sets differ from the elementary formula.

P-value

Sources

Robert Rosenthal and Donald B. Rubin, The Counternull Value of an Effect Size: A New Statistic (1994), original abstract: https://journals.sagepub.com/doi/10.1111/j.1467-9280.1994.tb00281.x Counternull sets in randomized experiments, original research full text: https://pmc.ncbi.nlm.nih.gov/articles/PMC12326273/

Previous: Countermeasure Next: Counters