AI Glossary / Probabilistic machine learning / Advanced

Translation-invariant prior

Definition

A translation-invariant prior, in the location-parameter sense, assigns the same prior measure to equal-width parameter intervals regardless of where they lie. For an unrestricted real-valued location parameter, this corresponds to a constant density relative to ordinary length. Because that constant cannot integrate to one over the entire real line, the prior is improper.

What it means

A location parameter shifts a distribution without changing its shape. If a measurement and its unknown location are both increased by the same constant, a location-family likelihood retains the same form. A translation-invariant prior expresses the idea that choosing a different numerical origin should not privilege particular locations.

This entry concerns invariance under shifts of a statistical location parameter. It is distinct from spatial translation invariance in image models or neural-network architectures. An improper prior is a mathematical device for inference, not an ordinary probability distribution from which one can draw a uniformly random real number. Its use requires checking that the resulting posterior can be normalized.

How it works

For a location parameter mu, require that shifting an interval by any constant leaves its prior measure unchanged. With a density relative to length, this gives p(mu) proportional to one across the real line.

Multiply this prior measure by the likelihood. If the integral of the resulting nonnegative function is finite and positive, divide by that integral to obtain a proper posterior. If the integral diverges, a valid posterior distribution has not been obtained; numerical output alone does not repair the problem.

For independent normal observations with unknown mean and known positive variance sigma-squared, the flat prior gives a normal posterior centered at the sample mean with variance sigma-squared divided by the sample size. This example is unusually tractable and should not be generalized automatically to unidentified or hierarchical models.

Invariance also depends on what transformation is being discussed. A prior flat in mu is generally not flat in a nonlinear function of mu. Transforming a density requires the appropriate Jacobian. Choosing an arbitrary new flat density after reparameterization changes the prior assumption.

Three practical examples

1. A measurement analyst uses the normal known-variance model to illustrate why shifting a reference origin shifts the posterior mean without changing posterior uncertainty.

2. A statistical instructor contrasts a flat location prior with a scale-invariant prior, showing that additive and multiplicative transformations require different measures.

3. A modeling team checks whether an intercept with an improper prior remains identified after adding other unrestricted offsets to a hierarchical model.

Think of it this way

Think of a ruler with no preferred zero mark. Moving the zero should not make one equally long interval more favored than another. Extending that ruler infinitely creates a difficulty: equal positive weight across infinitely many disjoint intervals cannot sum to one. The symmetry is understandable, but it does not supply a proper probability distribution by itself.

Worked example

Suppose observations 8, 10 and 12 are independent normal measurements with common unknown mean mu and known measurement variance four. Adopt p(mu) proportional to one.

The sample mean is 10 and the sample size is three. Completing the square in the likelihood gives a posterior normal distribution with mean 10 and variance 4/3, approximately 1.3333. The data make this posterior proper even though the prior was improper.

Change the measurement origin by adding 100 to every observation. The values become 108, 110 and 112. The corresponding posterior for the shifted location has mean 110 and the same variance 4/3. Subtracting 100 returns the original inference, illustrating translation consistency.

A future noisy observation has posterior predictive mean 10 and variance 4 + 4/3 = 16/3, approximately 5.3333. Parameter uncertainty and measurement noise are different contributions.

Finally, define eta = exp(mu). Because mu = log(eta), the transformed prior measure is proportional to 1/eta for eta greater than zero. It is not flat in eta. This last calculation illustrates transformation of the same prior measure, not a new claim that every positive parameter should receive that prior.

Illustrative case study

Fictional case: Cedar Calibration estimates a sensor's additive offset using a simple normal measurement model. Analysts use the flat-prior calculation as a transparent baseline and verify that changing the reference origin leaves the substantive result unchanged.

When they extend the model to include separate unrestricted machine and site offsets, they discover that only certain combinations are identifiable from the data. They reconsider the parameterization and introduce justified constraints or proper priors. A successful calculation in the original one-mean model does not establish posterior propriety in the expanded model.

Common misconceptions and mistakes

A uniform distribution on a fixed finite interval is proper, but it is not invariant under every real translation: its boundaries matter. A very wide normal prior can approximate flat behavior over a relevant region, but it remains a different, proper prior.

Calling a flat prior noninformative does not mean it expresses no assumptions. Its interpretation depends on the parameterization and model.

An arbitrary multiplicative constant cancels in a proper posterior for a fixed model, but can obstruct ordinary marginal-likelihood comparisons between models with independently unspecified improper-prior constants. Posterior existence must be established rather than assumed from the word flat.

Business use

The practical value is conceptual clarity about reference origins and transparent baseline calculations for suitable location models. Teams should document the parameterization, prove or otherwise establish posterior propriety, and compare against defensible proper priors when results may depend on weakly identified parameters. This is a modeling choice with mathematical conditions, not a universal default for business uncertainty analysis.

People also ask

Can it be sampled directly on the real line?

No; there is no proper uniform probability distribution over all real numbers.

Can an improper prior yield a proper posterior?

Yes, as in the worked normal example, but this must be checked.

Is it the same as scale invariance?

No. Translation concerns adding a constant; scale invariance concerns multiplication.

Check your understanding

Why is the posterior variance unchanged after adding 100?

Answer: The shift changes the location but not the noise scale or information supplied by the three observations.

Does a flat prior remain flat after exponentiation?

Answer: No; the Jacobian produces a density proportional to 1/eta.

Related terms

• Scale-invariant prior
• Jeffreys prior
• location parameter
• improper prior
• posterior propriety
• change of variables and identifiability

Learn next

Next, work through a density transformation with its Jacobian. Then compare additive and multiplicative invariance and test posterior propriety in a model with redundant location parameters.

More in Probabilistic machine learning

• TD(lambda) - TD(lambda), written TD(λ), is a family of temporal-difference learning methods that use eligibility traces to spread prediction errors across recently...
• Test-time adaptation - Test-time adaptation is the adjustment of a trained model or its inference statistics using data encountered during deployment or evaluation.
• Tied covariance in Gaussian discriminant analysis - Tied covariance in Gaussian discriminant analysis means that all modeled classes share one covariance matrix, while retaining their own mean vectors and...
• Truncated singular value decomposition - Truncated singular value decomposition is a low-rank representation of a matrix obtained by retaining only its largest singular values and associated left...
• Two-filter smoothing - Two-filter smoothing is a way to infer past hidden states by combining a forward filtering calculation with information propagated backward from later...
• Underdamped Langevin dynamics - Underdamped Langevin dynamics is a stochastic process that evolves both position and velocity under a potential force, friction and random noise.

Sources

Primary source: Nicholas Zabaras, Prior Modeling and Hierarchical Bayesian Models, slides on translation invariance and improper priors: https://web.ics.purdue.edu/~ibilion/www.zabaras.com/Courses/BayesianComputing/PriorModelingAndHierarchicalBayesianModeling.pdf

The numerical calculation and fictional case are original. The stated scope is a real-valued location parameter.

Independent review consulted the translation-invariance and improper-prior slides, including the normalization condition.

Previous: Transitive Rule Next: Translation memory