Can a small positive prior become a large posterior?

Yes, with sufficiently strong appropriate evidence, as the worked example shows.

Does the rule ban continuous distributions?

No. Its simple finite-hypothesis advice should not be misapplied to probability-zero singleton events in continuous spaces.

Can the background model change?

Yes. Reconsidering its assumptions is different from conditioning within the original probability model.

AI Glossary / Statistics and probability / Intermediate

Cromwell's rule

Definition

Cromwell's rule is advice to avoid assigning exact prior probabilities of zero or one to uncertain empirical propositions. Named by statistician Dennis Lindley, it reflects a consequence of ordinary Bayesian updating: a proposition given zero prior probability remains at zero after conditioning on evidence with positive probability within the same model.

What it means

Very unlikely and impossible are different commitments. A tiny positive probability leaves room for sufficiently strong evidence to change an assessment. An exact zero excludes the proposition from ordinary updating in the stated probability model.

The corresponding problem occurs with certainty. Giving a proposition probability one gives its negation zero, so ordinary conditionalization cannot move probability to the excluded alternative under the same conditions.

The rule is a principle of cautious modeling, not an instruction to believe every imaginable claim equally. Evidence can justify extremely unequal probabilities. It also requires care in continuous models, where an individual point commonly has probability zero without being logically impossible.

How it works

For a discrete hypothesis H and evidence E with P(E) greater than zero, Bayes' theorem gives P(H|E) = P(E|H)P(H)/P(E). If P(H) = 0, the posterior is zero. More generally, conditioning on positive-probability evidence cannot give positive probability to a prior probability-zero event.

For two hypotheses with positive prior probabilities and a finite likelihood ratio, posterior odds equal prior odds multiplied by that ratio. Small positive odds can therefore grow substantially; zero odds cannot through this calculation.

If the model assigns zero probability to the observed evidence, the elementary conditional formula is undefined. That is a reason to examine the model or conditioning framework, not to claim that its ordinary update has produced a valid contradiction. Revising the hypothesis space or background assumptions is distinct from updating within an unchanged model.

Three practical examples

A fault-diagnosis model leaves supported but rare failure modes with small prior probabilities rather than deleting them because they were absent from a short training sample.
A forecasting team distinguishes a probability rounded to 0% for display from a mathematically exact zero stored in the model.
An analyst checks whether a prior rules out plausible parameter regions before interpreting a posterior as evidence that those regions are impossible.

Think of it this way

Think of an investigation with a list of possible explanations. A low prior places an explanation near the bottom of the list. An exact zero removes it from the ordinary scoring process altogether. New evidence can reorder listed explanations, but reopening an excluded one requires reconsidering the list or its assumptions.

Worked example

Consider an original two-hypothesis model. H is a rare equipment fault and not H is its complement. Assign P(H) = 0.01, giving prior odds 1:99.

Suppose an observed diagnostic signal has likelihood ratio 100 in favor of H. For example, the stipulated probabilities could be P(E|H) = 0.5 and P(E|not H) = 0.005. Posterior odds become 100:99, so P(H|E) = 100/199, approximately 50.25%.

If a second signal is conditionally independent of the first given either hypothesis and has the same likelihood ratio, the joint likelihood ratio is 10,000. Posterior probability becomes 10,000/10,099, approximately 99.02%.

Starting instead with P(H) = 0 keeps the posterior at zero after either signal in this model, because the signals still have positive probability under not H. Multiplying zero prior weight by either likelihood cannot restore it.

Conditional independence is essential to multiplying signal likelihood ratios. Repeated copies of the same alert do not automatically provide independent evidence. These invented values illustrate the rule; they are not measured diagnostic performance.

Illustrative case study

Fictional case: Pine Systems develops an equipment-monitoring tool. One failure category never appears in its small initial sample, and a prototype assigns that category probability zero. Engineers later observe evidence consistent with that failure, but the probability calculation cannot raise its weight.

The team reviews whether the category is genuinely impossible or merely unobserved. It adopts a justified smoothing or prior scheme and tests sensitivity to rare-event assumptions. It also adds a route for investigating cases poorly explained by any modeled category.

Pine records why the revised model includes each alternative and evaluates whether its probability estimates are useful in subsequent observations.

Common misconceptions and mistakes

Cromwell's rule does not require assigning equal probabilities to competing explanations.
An unobserved event in a finite sample is not necessarily an impossible event.
Probability zero in a continuous distribution is not synonymous with logical impossibility. Every exact value under a continuous density can have zero point probability.
Zero probability mass, zero density at a point, and a whole excluded region are different notions.
Mathematical constraints can legitimately exclude impossible outcomes within a model, while the model's applicability remains open to examination.
Changing a zero prior after discovering faulty assumptions is model revision, not a counterexample to Bayes' theorem.

Business use

The rule encourages careful handling of rare events in AI classification, anomaly detection, and scenario planning. Inspect whether software converts absent counts into permanent exclusions, and distinguish structural zeros from sampling zeros.

Document prior support: which possibilities the model can represent at all. A posterior can look decisive because alternatives were excluded before the data arrived. Sensitivity analysis helps reveal when conclusions depend heavily on such restrictions.

Also retain enough numerical precision to distinguish tiny values from exact zero. Computational underflow is a numerical issue, while assigning zero probability is a modeling choice. Appropriate log-scale calculations can address the former, but cannot repair an unjustified exclusion in the latter.

Check your understanding

Answer: 1.

Why does zero prior weight persist in the example?

Answer: The Bayesian numerator remains zero while the evidence probability is positive. 2.

Why could the two signal ratios be multiplied?

Answer: Conditional independence was explicitly assumed under both hypotheses. 3.

What should a displayed 0% prompt you to check?

Answer: Whether it is rounded, numerically underflowed, or an intentional exact exclusion.

Related terms

Prior probability
Smoothing
Subjective probability
Bayesian updating
likelihood ratio
model support
structural zero

Learn next

Learn Bayes' theorem in probability and odds forms. Then study prior support, continuous probability densities, and model checking. Compare sensitivity to several defensible rare-event priors rather than treating one convenient small number as automatically correct.

Sources

Dennis Lindley, Understanding Uncertainty, section 6.8, primary author account in a hosted book copy: https://www.al-edu.com/wp-content/uploads/2014/05/Lindley-D.V.-Understanding-uncertainty-2006.pdf

Korbinian Strimmer, Essentials of Bayesian statistics, zero-forcing discussion: https://strimmerlab.github.io/publications/lecture-notes/MATH27720-stats2/15-bayes3.html

Relevant primary author passages were read. The diagnostic example and fictional case are original; continuous-point and model-revision qualifications are explicit.

Previous: Critical value Next: Cross-attention