AI Glossary / Statistics and probability / Advanced
G-prior
Definition
A g-prior, commonly called Zellner's g-prior, is a Bayesian prior for regression coefficients in a normal linear model whose covariance is proportional to the inverse design cross-product matrix. In a standard zero-centered form, beta given sigma squared and g follows a multivariate normal distribution with covariance g times sigma squared times (X'X) inverse. The positive parameter g controls the prior scale relative to the information in the design. The prior is useful for tractable regression and model-comparison calculations, but its meaning depends on the model, parameterization and treatment of intercept and variance.
What it means
The design matrix describes how predictors vary and relate to one another. A g-prior uses that geometry rather than assigning identical independent prior variances to every raw coefficient. With a fixed g and zero prior mean, the posterior mean in the standard setting shrinks the ordinary least-squares estimate toward zero by factor g/(1 + g). A larger g means a more diffuse prior and less shrinkage; a smaller positive g gives stronger shrinkage. This direction is easy to confuse because other regularization parameters increase shrinkage when they increase.
How it works
Specify a Gaussian regression likelihood and ensure the relevant design matrix has full column rank. Decide which coefficients receive the g-prior and how an intercept or common nuisance parameters are handled; centered predictors with a separately treated intercept are common. Choose a fixed g, estimate it under an empirical-Bayes procedure or place a hyperprior on it. Combine prior and likelihood to obtain posterior quantities. For fixed known variance and a zero-centered prior on the modeled coefficient block, the posterior covariance is [g/(1 + g)] sigma squared (X'X) inverse. Unknown variance, model averaging and mixtures over g require their corresponding calculations rather than silently reusing every fixed-g formula.
Three practical examples
- Bayesian regression: regularize coefficient estimates in a model where design-adapted prior scaling is appropriate.
- Variable selection: obtain tractable marginal likelihoods for specified candidate linear models under carefully defined priors.
- Sensitivity analysis: examine how coefficient estimates and model probabilities change with alternative prior scales or mixtures over g.
Think of it this way
Think of combining a preliminary expectation with a data-based estimate using weights tied to their relative precision. The g-prior sets the preliminary uncertainty in relation to the predictor design. It does not make that expectation uniquely correct or remove the need to examine sensitivity.
Worked example
Consider a hypothetical centered one-predictor normal regression with known error variance sigma squared = four, X'X = 20 and ordinary least-squares slope two. Use a zero-centered g-prior with g = three. The prior slope variance is 3 times 4/20 = 0.6, while the likelihood-based slope variance is 4/20 = 0.2. Combining precisions gives posterior variance 1/(1/0.6 + 1/0.2) = 0.15. The posterior mean is 0.15 times (zero/0.6 + two/0.2) = 1.5. Equivalently, the shrinkage factor is 3/(1 + 3) = 0.75 and 0.75 times two = 1.5. This calculation verifies the fixed-g conjugate result under the stated model. It does not include uncertainty about g or the error variance, and it is not a comparison between candidate models.
Illustrative case study
Fictional case: Birch Model Review compares several linear forecasting specifications with Bayesian methods. Analysts document which slopes receive the g-prior, how the intercept is handled and which model prior is used. They find that some model probabilities change under alternative g choices and report that sensitivity instead of presenting one ranking as inevitable. They also evaluate prediction on later observations. This invented case illustrates explicit prior specification without claiming that a g-prior automatically delivers the best model or reliable forecasts.
Common misconceptions and mistakes
Do not confuse g with a gamma distribution or assume every Bayesian regression uses a g-prior. The conventional inverse requires full rank; collinearity or more predictors than supported observations can make the formula undefined. A proper prior at finite g does not become harmless for model comparison simply by taking g arbitrarily large. Model probabilities also depend on priors over models and nuisance parameters. A fixed-g shrinkage factor should not be substituted for the posterior average arising from a mixture over g. Design dependence is part of the prior specification and should be disclosed.
Business use
Businesses can use g-priors in carefully specified Bayesian linear analyses where tractability and model uncertainty are important. Examine whether the normal linear likelihood and predictor coding are suitable before focusing on the prior. Preserve the prior choices in the analysis record and assess sensitivity of decision-relevant conclusions. Compare predictive performance with simpler approaches, and distinguish coefficient shrinkage from causal identification. If the design is rank-deficient, select a justified alternative rather than silently replacing the inverse and assuming all original properties remain unchanged. A convenient default remains a modeling choice requiring explanation.
People also ask
Does a larger g imply stronger shrinkage?
No, in the standard convention it makes the prior more diffuse and the shrinkage factor closer to one.
Can g be uncertain?
Yes. Mixtures of g-priors place a prior distribution on it.
Does the g-prior solve collinearity automatically?
Not in its conventional form, which requires an invertible design cross-product matrix.
Check your understanding
With zero prior mean and fixed g = one, what is the standard posterior-mean shrinkage factor?
Answer: One-half. This result assumes the relevant normal linear-model setup and does not describe every generalized or mixture-prior extension.
Related terms
- Bayesian Regression
- Conjugate prior
- Shrinkage
- Bayes factor
- Empirical Bayes
- Zellner prior
- design matrix
- marginal likelihood
- hyper-g prior
- model averaging
Learn next
Learn normal-normal conjugate regression, then study how prior scale affects model comparison as well as estimation. Compare fixed g, empirical-Bayes choices and mixtures, including their assumptions and sensitivity properties.
More in Statistics and probability
- Full joint probability distribution - A full joint probability distribution specifies the probabilities of all combinations of values for all random variables included in a probability model.
- Functional regression - Functional regression models relationships in which an outcome, a predictor or both are functions, such as curves observed over time, wavelength or space.
- Fuzzy c-means - Fuzzy c-means is a clustering algorithm that assigns each observation a degree of membership in each of a chosen number of clusters.
- Gamma distribution - The gamma distribution is a family of continuous probability distributions for positive quantities, commonly parameterized by a positive shape and a...
- Gamma function ( G ) - The gamma function, conventionally written with the uppercase Greek letter Gamma, extends the factorial relationship beyond nonnegative integer arguments.
- Gaussian error model - A Gaussian error model assumes that the difference between an observed response and its modeled mean follows a normal distribution under specified conditions.
Sources
Liang and colleagues, Mixtures of g-priors for Bayesian Variable Selection, author-hosted paper: https://www2.stat.duke.edu/~berger/papers/g-priors.pdf Duke University, Bayesian regression teaching material, Zellner's g-prior: https://www2.stat.duke.edu/courses/Fall21/sta601.001/slides/10-gibbs.html Baragatti and Pommeret, g-prior with ridge parameter and rank limitations: https://arxiv.org/abs/1102.0470
Previous: g-computation Next: G-set