Is this a logical contradiction?

No. Tail probabilities and model-averaged predictive comparisons are different quantities.

Does every Bayesian analysis favor the null?

No. Results depend on the models, priors, data, and inferential question.

Must the null be literally exact?

The classic example uses a point null; related disagreements can occur with suitably specified narrow alternatives around it.

AI Glossary / Statistics and probability / Advanced

Lindley's paradox

Definition

Lindley's paradox is a disagreement that can arise between a conventional significance test and a Bayesian comparison of a point-null hypothesis with an alternative. A result can have a small p-value while a Bayes factor favors the null. In a classic construction, the p-value stays fixed as sample size increases, while the alternative retains a fixed prior scale. The disagreement reflects different questions and assumptions; it is not an arithmetic contradiction or proof that one framework always fails.

What it means

A p-value assesses how extreme the observed statistic, or something more extreme, would be under a specified null model. A Bayes factor compares how well two models predict the observed data after integrating over their parameter priors.

Holding statistical significance fixed while increasing sample size makes the observed departure from the null progressively smaller in the standard normal example. A broad alternative can predict such a narrowly concentrated result less effectively than the point null. The sample size and the prior's scale therefore matter alongside the p-value.

How it works

For independent normal observations with known variance sigma squared, the sample mean has variance sigma squared/n. Compare H0: theta = 0 with H1 assigning theta a proper normal prior centered at zero with variance tau squared.

Set z = sample mean divided by its standard error and r = n tau squared/sigma squared. The Bayes factor favoring H0 is BF01 = sqrt(1 + r) exp[-z squared r/(2(1 + r))]. This follows by taking the ratio of the two normal predictive densities for the mean.

At fixed z and fixed positive tau, the exponential factor approaches a constant while sqrt(1 + r) grows. Posterior odds equal this factor times prior model odds. Prior parameter scale and prior model probability are distinct choices.

Three practical examples

A large online experiment reports a barely significant change in a continuous score. Analysts compare its practical magnitude with evidence under a pre-specified alternative model.
A measurement team tests exact calibration against a range of possible offsets and explains why its Bayes factor differs from a tail-area test.
An AI evaluation report includes p-values and Bayesian model comparisons, making the null hypothesis, alternative prior, and model probabilities visible rather than presenting incompatible labels without context.

Think of it this way

Think of two forecasters: one predicts a value exactly at a reference point, while another spreads predictions across a broad range. A very precise observation just beside the reference can be unusual by a null-based tail rule yet fit the concentrated forecast better than the broad one. The scoring rules ask different questions about that observation.

Worked example

Assume observations are independent normal values with known sigma = 1. Under H1, give theta a normal prior with mean zero and standard deviation tau = 1. Give H0 and H1 equal prior probabilities.

First take n = 100 and an observed mean of 0.20. Its standard error is 0.10, so z = 2 and the two-sided p-value is approximately 0.04550. Here r = 100 and BF01 = sqrt(101) exp[-200/101] = 1.3873. The posterior probability of H0 is 1.3873/(1 + 1.3873) = 0.5811.

Now take a separate constructed dataset with n = 10,000 and mean 0.020. Its standard error is 0.010, leaving z = 2 and the same p-value. But BF01 = sqrt(10001) exp[-20000/10001] = 13.5369. The posterior probability of H0 becomes 0.9312.

Both calculations use the same fixed alternative prior. They illustrate a sequence with shrinking observed effects, not repeated sampling under a fixed nonzero effect. The probabilities are conditional on these explicit models and prior odds.

Illustrative case study

Fictional case: Northstar Search evaluates a ranking adjustment with a very large sample. A significance-test summary labels a small score improvement significant, while a Bayesian comparison favors an exact-zero model over a broadly spread alternative.

The team checks the prior scale, computes effect estimates with uncertainty, and defines the smallest gain worth the deployment cost. It separates evidence about exact zero from the decision about a useful improvement.

Its final review reports the assumptions behind both analyses. It does not switch priors merely to obtain a preferred conclusion, and it does not treat a model probability as a guaranteed statement about every possible explanation of the data.

Common misconceptions and mistakes

Do not read the p-value as the posterior probability of H0. Also avoid confusing a Bayes factor with a posterior probability: prior model odds are required for that conversion.

The classic fixed-p-value construction does not imply that increasing sample size must favor the null when a fixed nonzero effect is present. An excessively diffuse alternative prior creates a related sensitivity issue, often associated with Bartlett's paradox. Improper priors generally cannot be inserted naively into marginal-likelihood comparisons because arbitrary normalizing constants may not cancel.

Business use

For business decisions, the lesson is to specify the question before selecting an evidence threshold. A tiny detectable change, support for a point-null model, and an economically useful benefit are distinct judgments.

Report effect size, uncertainty, baseline scale, and implementation cost. When Bayesian model comparison is used, document sensible prior scales and sensitivity analyses. An AI summarizer should preserve these details instead of compressing every result into significant or insignificant.

Check your understanding

Two studies have z = 2 but different sample sizes. Must their Bayes factors match?

Answer: No. With a fixed prior scale for the effect, their predictive comparisons can differ substantially. In the worked example, the Bayes factors are about 1.39 and 13.54 despite identical two-sided p-values.

Related terms

Bayes factor
P-value
Practical significance
point-null hypothesis
prior predictive distribution
posterior odds
Bartlett's paradox

Learn next

Next, derive the normal predictive-density ratio and vary the alternative prior scale. Then compare point-null testing with estimation and decisions based on a region of practically negligible effects.

Sources

Sources: Lindley, A Statistical Paradox (1957), original paper: https://gwern.net/doc/statistics/bayes/1957-lindley.pdf . Wagenmakers and Ly, History and Nature of the Jeffreys–Lindley Paradox, author-institution full text: https://ir.cwi.nl/pub/32113/32113.pdf . The normal-prior worked example is an original construction, independently evaluated from predictive-density formulas; it is not a reproduction of Lindley's numerical table.

Previous: Linde - Buzo - Gray algorithm Next: Line chart