Does noninformative mean no influence?

No; small-sample results can change materially.

AI Glossary / Statistics and probability / Advanced

Jeffreys prior

Definition

A Jeffreys prior is a Bayesian prior constructed from the expected Fisher information of a statistical model. For one continuous parameter, its density is proportional to the square root of that information. For a regular multiparameter model, the general rule uses the square root of the determinant of the Fisher information matrix.

What it means

The rule provides a model-based way to choose a prior whose probability measure transforms consistently under smooth, one-to-one changes of parameter. It is often described as an objective or noninformative prior, but those labels do not mean that it has no effect on inference.
The likelihood model matters. A prior is not determined by a parameter's name alone. Its construction depends on how observations are assumed to arise, and its consequences still require examination alongside substantive knowledge and alternative reasonable priors.

How it works

Specify the sampling model and calculate its expected Fisher information. Take the appropriate square root, then determine whether the resulting density can be normalized over the parameter space.
Multiply the prior by the likelihood and check that the posterior has a finite normalizing constant. Some Jeffreys priors are proper probability distributions; others are improper. An improper prior can sometimes yield a proper posterior, but this must be established rather than assumed.
Invariance means that applying the rule in new coordinates agrees with transforming the original prior using the Jacobian. It does not mean that the density has the same numerical shape in every coordinate system.

Three practical examples

An AI evaluator compares binomial event-rate estimates under a Jeffreys prior and other defensible priors, especially when the observed event count is small.
A statistical analyst changes from a probability to another one-to-one parameterization and verifies that the prior transformation is consistent.
A modeling reviewer checks posterior propriety before accepting results produced from an automatically selected prior.

Think of it this way

A map changes appearance when its coordinate system changes, but the represented places should remain the same. Jeffreys' rule aims for that kind of consistency in prior probability. It does not claim that every coordinate system should display a flat density.

Worked example

Consider an original binomial evaluation: ten independent trials share an unknown event probability p, and two events are observed. With the trial count fixed, the likelihood is proportional to p squared times (1 minus p) to the eighth power.
For n binomial trials, the log-likelihood contains k log p plus (n minus k) log(1 minus p). Its negative second derivative is k divided by p squared plus (n minus k) divided by (1 minus p) squared.
Taking expectation under the model replaces k by np. Simplifying yields Fisher information n divided by p times (1 minus p). The factor square root of n does not depend on p, so it disappears when the prior is normalized.
The resulting prior density is 1 divided by pi times the square root of p times (1 minus p), for p between zero and one. This is Beta(1/2, 1/2). It is integrable despite increasing without bound near the two endpoints; an unbounded density is not necessarily an improper distribution.
Multiplication by the observed likelihood gives a Beta(2.5, 8.5) posterior. Its mean is 2.5/11, approximately 0.2273. Its variance is (2.5 times 8.5) divided by (11 squared times 12), approximately 0.01463.
For comparison, the maximum-likelihood estimate is 2/10 = 0.20. A uniform Beta(1, 1) prior would produce Beta(3, 9), whose posterior mean is 3/12 = 0.25. These differences illustrate prior influence; they do not establish which prior is substantively most suitable.
If the same ten-trial design instead recorded no events, the Jeffreys posterior would be Beta(0.5, 10.5), with mean 0.5/11, approximately 0.04545. Observing zero events therefore does not force the posterior mean probability to zero. This estimate is not an upper uncertainty bound.
Now change coordinates using p = sin squared theta, with theta between zero and pi/2. The derivative dp/dtheta is 2 sin theta cos theta. Multiplying the prior density by this positive derivative gives the constant 2/pi.
Thus this prior is uniform in theta even though it is not uniform in p. The prior probability that p is at most one quarter is the probability that theta is at most pi/6: (2/pi) times pi/6 = one third. A uniform prior on p would assign one quarter instead.
As a separate illustration of impropriety, consider a normal model with unknown mean mu and known standard deviation two. Four independent observations give Fisher information four divided by four = one for mu. Jeffreys' rule therefore yields a constant density over the whole real line, which cannot integrate to one.
If those four observations have mean ten, combining this improper prior with their normal likelihood gives a proper normal posterior with mean ten and variance one. This example shows why prior propriety and posterior propriety are distinct checks. It does not justify assuming propriety for a different model.

Illustrative case study

Fictional company Cedar Evaluation observes few instances of a defined failure in a small test. Its analyst reports the chosen binomial model, the Jeffreys posterior, and how an alternative prior changes the estimate.
The team also reports uncertainty rather than presenting the posterior mean as a guarantee about future performance. It checks whether the independence and constant-probability assumptions fit the evaluation. This fictional scenario demonstrates transparent analysis, not validated product reliability.

Common misconceptions and mistakes

Jeffreys prior is not always uniform or always improper.
An infinite density at an endpoint does not imply infinite total probability.
Invariance concerns consistent probability transformation, not identical density plots.
A posterior mean is not a confidence limit.
A prior selected by a formal rule does not remove the need to examine model assumptions or sensitivity.

Business use

Jeffreys priors can provide a transparent comparison point in Bayesian analyses. Document the sampling model, parameterization, posterior-propriety check, and sensitivity to alternatives. Where credible prior knowledge exists, evaluate whether an explicitly informative prior better represents the actual decision context.

Check your understanding

What is the posterior after two events in ten trials?

Answer: Beta(2.5, 8.5).

What is its mean?

Answer: Approximately 0.2273.

Is the prior uniform on the probability scale?

Answer: No.

Related terms

Jacobian
Bayesian prior
Fisher information
posterior distribution
reparameterization
beta distribution
improper prior
prior sensitivity

Learn next

Study change-of-variable formulas, posterior propriety, and the limitations of automatic prior rules in models with several parameters.

Sources

Merlise Clyde, Duke University, Basics of Bayesian Statistics: https://www2.stat.duke.edu/courses/Fall21/sta601.001/slides/01-basics-of-bayes.html . Lancaster University, Objective priors: https://www.lancaster.ac.uk/~prendivs/accessible/math331/lectures.tex/Ch2.S4.html . Numerical examples and company scenario are original illustrations.

Previous: JDBC-ODBC bridge Next: Jenkins - Traub algorithm