Is scale invariance the same as Jeffreys invariance?

No. Scale invariance concerns multiplicative changes; Jeffreys priors arise from the model's Fisher information and transform consistently under regular reparameterizations.

Can I sample an unbounded 1/sigma prior directly?

No normalized distribution exists on the full positive line.

Does uniformity in log variance produce the same scale measure?

Yes: with v = sigma squared, dv/v = 2 d sigma/sigma; the constant factor does not change the proportional prior.

AI Glossary / Probabilistic machine learning / Advanced

Scale-invariant prior

Definition

A scale-invariant prior is a prior measure for a positive scale parameter that retains its form when measurement units change multiplicatively. Its standard density is proportional to 1/sigma for sigma > 0. It assigns equal measure to intervals with the same endpoint ratio, but is improper over the entire positive line.

What it means

A scale describes magnitude, such as the standard deviation of measurement errors. Expressing that magnitude in centimeters instead of meters changes its numerical value without changing the underlying uncertainty.
The scale-invariance principle asks the prior measure to respect this change of units. It does not mean that every numerical value has equal density. Instead, doubling ranges such as 1 to 2 and 10 to 20 receive equal unnormalized measure. Calling this prior noninformative describes a particular convention, not an absence of modeling assumptions.

How it works

Let t = log(sigma). A constant density with respect to t transforms to a density proportional to 1/sigma with respect to sigma, because dt/dsigma = 1/sigma. Likewise, changing to s = c sigma gives d sigma/sigma = ds/s: the measure has the same form.
The integral of 1/sigma from zero to infinity diverges. There is therefore no normalized probability distribution with this density across all positive scales. Bayesian updating may nevertheless produce a proper posterior, but that must be established for the particular likelihood and data.
A finite log-uniform prior on L <= sigma <= U has density 1/[sigma log(U/L)]. It is proper, but its bounds encode information. Unit conversion requires converting both bounds as well as the parameter.
For a regular normal model with known mean and unknown standard deviation, the scalar Jeffreys prior has this form. With multiple unknown parameters, joint Jeffreys and reference-prior constructions need not coincide.

Three practical examples

Measurement models: an analyst examines a baseline prior for the unknown spread of instrument errors, while checking that the resulting posterior exists.
Simulation design: positive magnitudes are sampled over specified orders of magnitude using a bounded log-uniform distribution.
Prior sensitivity: a team compares conclusions under several defensible lower and upper bounds, rather than assuming a broad interval has no effect.

Think of it this way

Think of a ruler marked by doubling: 1, 2, 4, 8, 16. Equal distances on this logarithmic ruler represent equal ratios on the ordinary ruler. A log-uniform prior spreads equal probability across those doubling bands within chosen endpoints. Extending the ruler endlessly in both directions leaves no finite total amount to divide into probabilities.

Worked example

Consider an original bounded-prior example for an error scale sigma, measured in millimeters, between 1 and 16. Set p(sigma) = 1/[sigma log(16)].
The probability of 2 <= sigma <= 4 is log(4/2)/log(16) = 1/4. The probability of 4 <= sigma <= 8 is also 1/4, even though the second interval is twice as wide in millimeters.
The median is 4, because log(4/1)/log(16) = 1/2. The mean is (16 - 1)/log(16), approximately 5.410 millimeters. A symmetric-looking distribution on the logarithmic ruler is therefore not symmetric on the original scale.
Convert to centimeters: s = sigma/10. The support becomes 0.1 to 1.6, and the median becomes 0.4 centimeters. The probability of 0.2 <= s <= 0.4 remains 1/4. Keeping the old numerical bounds would specify a different physical prior.
For contrast, a uniform prior on the original interval assigns probability 2/15 to 2 through 4 and 4/15 to 4 through 8. These are substantively different choices. No observations or posterior claims are included in this calculation.

Illustrative case study

Fictional case: Pine Calibration develops a model for the variability of repeated scanner measurements. One analyst proposes a broad log-uniform scale prior because the plausible magnitude spans several orders.
The team documents the physical units and proposed endpoints, then simulates measurements from the prior predictive model. It repeats the analysis with several lower bounds and discovers that a hierarchical extension is sensitive to the amount of prior mass placed near zero.
The team investigates a proper prior with a scientifically defensible scale and reports the sensitivity. This hypothetical case illustrates prior checking; it is not evidence that any particular prior improves scanner performance.

Common misconceptions and mistakes

A density proportional to 1/sigma is not uniform in sigma. A proper prior on a finite interval is not globally invariant under arbitrary rescaling with fixed numerical endpoints. An improper prior cannot supply ordinary prior probabilities without an appropriate normalization.
Posterior propriety is not guaranteed by collecting some data or obtaining apparently stable simulation output. In hierarchical variance models, a log-uniform prior on a group-level scale can leave infinite posterior mass near zero. Finally, a positive parameter is not automatically a scale parameter in the statistical model.

Business use

The practical value is disciplined handling of units and multiplicative uncertainty. Teams should record which quantity receives the prior, its units, any bounds, and the reason for that choice. Before using predictions operationally, examine prior predictive behavior and sensitivity to plausible alternatives. An invariant formula can simplify a baseline analysis, but it cannot replace domain knowledge or validation.

Check your understanding

Under the worked prior, what is the probability that sigma lies between 8 and 16?

Answer: One quarter, because this is another doubling band. Its larger width does not give it greater probability under the log-uniform specification.

Related terms

Jeffreys prior
Reference prior
Improper prior
Log-uniform distribution
Scale parameter
Prior predictive distribution
Posterior propriety

Learn next

Practice density transformations with Jacobians, then compare uniform, log-uniform, and weakly informative priors for a positive parameter. Next examine why a prior that works in a simple observation model may fail for a hierarchical variance component.

Sources

Jose M. Bernardo, Bayesian Reference Analysis, discussion of location-scale models and Example 21: https://www.uv.es/bernardo/Monograph.pdf Andrew Gelman, Prior Distributions for Variance Parameters in Hierarchical Models (2006), Section 4.2: https://sites.stat.columbia.edu/gelman/research/published/taumain.pdf Bernardo Example 21 and Gelman Section 4.2 independently consulted, including the hierarchical near-zero propriety warning.

Previous: Scale co-occurrence matrix Next: Scale space