Why is the posterior Gaussian?

Multiplying the Gaussian likelihood in the unknown mean by its Gaussian prior produces a quadratic log density in that mean.

Does the posterior mean equal the sample mean?

Not generally. It is a precision-weighted combination, approaching the sample mean as data information dominates the prior.

What if observation variance is unknown?

A richer prior and likelihood analysis is needed, such as an appropriate normal-inverse-gamma model; the simple known-variance formulas no longer describe all uncertainty.

AI Glossary / Probabilistic machine learning / Intermediate

Gaussian-Gaussian model

Definition

A Gaussian-Gaussian model combines a Gaussian observation model with a Gaussian prior for an unknown mean or related continuous quantity. In the basic known-variance case, the posterior is also Gaussian and can be calculated exactly. The posterior mean balances prior information and observed data according to their precision, meaning inverse variance.

What it means

The two Gaussians play different roles. The likelihood describes how measurements vary around an unknown quantity. The prior describes uncertainty about that quantity before using the current observations. They are not two interchangeable descriptions of the same uncertainty.

This entry focuses on the common normal-normal conjugate model for a scalar mean with known observation variance. Gaussian is another name for normal. More general linear Gaussian models extend the idea to vectors and regression, while unknown observation variance requires additional modeling. A Gaussian prior alone does not make every Bayesian update Gaussian.

How it works

Suppose observations y_i are conditionally independent given mu and follow Normal(mu, sigma squared), with sigma squared known. Let the prior be Normal(m_0, v_0). For n observations with average y_bar, posterior precision is 1/v_0 + n/sigma squared.

The posterior variance v_n is the inverse of that precision. The posterior mean is v_n times (m_0/v_0 + n y_bar/sigma squared). Thus the prior mean and sample mean receive weights proportional to their respective precisions. More independent observations increase the sample mean's precision.

For a future observation generated by the same model, the posterior predictive variance is sigma squared + v_n. This includes both measurement variability and remaining uncertainty about mu. Sequential updates using each new independent observation give the same result as a batch update, provided the same assumptions and prior are used without counting data twice.

Three practical examples

1. Calibration: combine a prior estimate of a stable sensor offset with repeated noisy reference measurements. Check whether the stated noise variance and independence assumptions are credible.

2. Process estimation: estimate an average deviation from a target when observations are plausibly Gaussian with a known spread. Report uncertainty in the process mean separately from variation in individual future items.

3. Teaching and software validation: use the closed-form update as a small reference problem for a Bayesian inference implementation. Agreement on this simple case does not validate every more complex model, but disagreement can expose basic errors.

Think of it this way

Think of combining two estimates while giving more weight to the one with less uncertainty. A precise historical estimate can outweigh a very noisy new measurement, while enough independent measurements can dominate a diffuse prior. The analogy concerns uncertainty-weighted combination, not trust in an authority; prior precision must have a defensible interpretation rather than merely expressing confidence.

Worked example

Suppose a calibration offset mu has prior mean 10 and prior variance 4. Each new measurement has known variance 9. Four conditionally independent measurements have sample mean 13. The prior precision is 1/4 = 0.25, and the sample mean's precision is 4/9, approximately 0.4444.

Posterior precision is 0.25 + 0.4444 = 25/36. Posterior variance is therefore 36/25 = 1.44. The posterior mean is (10/4 + 4 × 13/9)/(25/36) = 11.92. Equivalently, the prior receives weight 0.36 and the sample mean weight 0.64, giving 0.36 × 10 + 0.64 × 13 = 11.92.

The posterior standard deviation of the unknown mean is 1.2. But the predictive variance for a single future measurement is 9 + 1.44 = 10.44, with standard deviation approximately 3.2311. Using 1.2 as the spread of future measurements would seriously understate their variability. These values are stipulated; no real calibration experiment or prior elicitation is being claimed.

Illustrative case study

Fictional case: Beech Instruments uses a normal-normal update to track a stable measurement offset. A report initially shows the posterior interval for the offset beside forecasts of individual readings. A reviewer notices that staff could mistake the narrower parameter interval for the range of future measurements.

The team labels the two uncertainty quantities separately and checks whether successive measurements share environmental noise. If they do, the simple independent-observation formula may be too confident. This fictional case illustrates correct interpretation and assumption review, not an empirical accuracy result.

Common misconceptions and mistakes

The prior variance is uncertainty about the mean, whereas the observation variance describes variation around that mean. Neither should be silently replaced by the other. The sample mean's variance is sigma squared divided by n only under the stated independence and common-variance assumptions.

Known observation variance is a simplifying assumption, not a fact guaranteed by having many observations. Plugging in an estimated variance without accounting for its uncertainty changes the analysis. A tightly concentrated prior can dominate limited data, and model misspecification can yield a narrow but misleading posterior. Gaussianity also permits negative values, which may be inappropriate for some quantities.

Business use

Use the model when its simplicity matches the measurement problem and the prior can be explained. Document where prior information came from and ensure that observations used to construct it are not inadvertently reused as fresh evidence.

Test sensitivity to plausible prior variances and check whether the observation process is stable. If data are correlated, heteroscedastic or affected by outliers, revise the likelihood rather than treating the closed-form convenience as a reason to ignore them. Communicate whether uncertainty refers to an unknown parameter, a future measurement or an average of future measurements.

Check your understanding

Why is the example's predictive variance 10.44 rather than 1.44?

Answer: A future measurement has its own variance 9 in addition to the remaining posterior uncertainty 1.44 about the mean. The two contributions add under this model.

Related terms

Conjugate prior
Likelihood
Posterior predictive distribution
Gaussian distribution
posterior precision
Bayesian updating
normal-inverse-gamma model
linear Gaussian model

Learn next

Derive the posterior by completing the square, then compare batch and sequential updates on the same small dataset. Next distinguish credible intervals for a mean from predictive intervals and study what changes when the variance is unknown.

Sources

Murphy, Conjugate Bayesian Analysis of the Gaussian Distribution, normal-prior section: https://www.cs.ubc.ca/~murphyk/Papers/bayesGauss.pdf

bpp authors, NormalNormalPosterior documentation: https://search.r-project.org/CRAN/refmans/bpp/html/NormalNormalPosterior.html

Independent review consulted Murphy sections 2.3–2.4 and bpp documentation. Original posterior and predictive calculations verified; calibration case fictional.

Previous: Gaussian error model Next: Gaussian mixture model