Are excluded coefficients always exactly zero?

In point-mass spike models they can be; continuous-spike variants instead concentrate them near zero.

Does the method select only one model?

It can support selection, but Bayesian model averaging can combine predictions across many possible models.

Why do correlated predictors complicate interpretation?

Several predictors may explain similar variation, so inclusion support can be distributed across alternatives.

AI Glossary / AI and machine learning / Intermediate

Spike-and-slab regression

Definition

Spike-and-slab regression is a Bayesian approach that uses a mixture prior for regression coefficients. The spike concentrates probability at or near zero, while the slab allows a wider range of values. This supports variable selection, shrinkage, and uncertainty about which predictors belong in a model.

What it means

When many potential predictors are available, some may contribute little after the others are considered. Instead of selecting one subset and treating it as certain, a Bayesian model can express uncertainty over possible subsets and coefficient values.

The spike represents an excluded or strongly shrunk effect. The slab represents an effect allowed to vary more widely. An inclusion indicator often records which component applies to a coefficient.

Some formulations use an exact point mass at zero, producing coefficients that are exactly zero in excluded models. Others use a narrow continuous distribution as the spike. These versions have different interpretations, so the chosen prior should be named.

Posterior inclusion probability summarizes how strongly the fitted model supports a predictor's inclusion under its assumptions. It is not a probability that the predictor causes the outcome, nor a universal measure of importance independent of the other predictors and prior choices.

How it works

Specify a regression likelihood and priors for coefficients, inclusion indicators, and other parameters. Choose the expected sparsity and slab scale in a way appropriate to the problem and measurement units.

Combine the prior with the observed data through Bayesian inference. Computation may use sampling or an approximation, depending on the formulation and model size.

Summarize coefficient distributions and inclusion probabilities. Predictions can average over alternative models, retaining uncertainty about which variables matter rather than committing immediately to one subset.

Check computational diagnostics, predictive performance, and sensitivity to prior settings. Strongly correlated predictors can share or compete for inclusion support. A low individual inclusion probability does not necessarily mean the broader group of related predictors carries no useful information.

Three practical examples

1. A forecasting exercise has several related candidate predictors. Spike-and-slab regression represents uncertainty about which combinations improve predictions, rather than declaring one selected subset unquestionably correct.

2. A researcher compares point-mass and continuous-spike formulations on synthetic data. The exercise distinguishes exact exclusion from strong shrinkage near zero.

3. An analyst examines two highly correlated measurements that alternate across sampled models. Reporting their joint predictive role may be more informative than ranking their individual inclusion probabilities alone.

Think of it this way

Think of a switch for each predictor paired with a dial. The switch determines whether the predictor participates, and the dial determines the size of its contribution when included.

The analogy fits a point-mass spike formulation. A continuous spike behaves more like a dial constrained near zero than a perfectly off switch. The model also represents uncertainty over these settings, rather than observing a hidden true switch directly.

Worked example

Suppose a simplified prior gives a predictor a 20% inclusion probability. Its prior inclusion odds are 0.2/0.8 = 0.25.

Imagine the data provide a Bayes factor of 6 favoring inclusion over exclusion under the specified models. Posterior odds become 0.25 × 6 = 1.5. Converting odds to probability gives 1.5/(1 + 1.5) = 0.6, or 60%.

This invented calculation illustrates Bayesian updating. It does not show how a Bayes factor is computed for a full regression, and the result depends on the prior and likelihood. A 60% inclusion probability is not a 60% chance of a causal effect or a guarantee that selecting the predictor improves future decisions.

Illustrative case study

Fictional case: Elm Forecasting Lab builds a regression for a practice demand dataset. Two nearly interchangeable calendar features receive moderate individual inclusion probabilities, while their combined presence across sampled models is common.

The team inspects this pattern instead of announcing that neither feature matters. It compares predictive performance, checks prior sensitivity, and documents how correlated alternatives affect interpretation.

The final report presents model uncertainty alongside forecasts. This invented case illustrates variable-selection uncertainty. It does not claim a real forecasting improvement or imply that posterior inclusion probabilities alone determine which operational data sources should be retained.

Common misconceptions and mistakes

Do not confuse inclusion probability with a p-value or a causal probability. These quantities answer different questions under different assumptions.

Do not assume every spike is an exact point mass. Do not treat the slab as unrestricted or prior-free; its scale affects inference. Avoid interpreting one selected model as the full Bayesian result when model averaging was intended. Finally, check whether sampling explores alternative models adequately before trusting apparently precise inclusion summaries.

Business use

Spike-and-slab regression can help businesses manage many candidate predictors while expressing uncertainty about model structure. It is particularly useful when predictive modeling and transparent discussion of variable selection are both important.

Assess held-out predictive quality and the stability of conclusions under plausible prior choices. Explain inclusion probabilities in plain language and avoid turning them into causal claims. Consider the practical costs of collecting predictors separately from their statistical roles, since a small predictive contribution may or may not justify an operational expense.

Check your understanding

A predictor has posterior inclusion probability 0.8. Does that mean it has an 80% probability of causing the response?

Answer: No. It describes inclusion uncertainty within the specified regression model and prior. Causal interpretation requires additional assumptions and an appropriate study design.

Related terms

Bayesian Regression
Shrinkage
Bayes factor
Regularization
variable selection
posterior inclusion probability
mixture prior
Bayesian model averaging

Learn next

Learn Bayes' rule in odds form and compare a point mass with a continuous density near zero. Then inspect a small regression with correlated predictors, examining coefficient uncertainty, inclusion patterns, and predictive performance under more than one plausible prior.

Sources

George and McCulloch, Approaches for Bayesian Variable Selection: https://www3.stat.sinica.edu.tw/statistica/oldpdf/A7n26.pdf, sections 2–3 consulted independently for mixture priors and model uncertainty. Malsiner-Walli and Wagner, Comparing Spike and Slab Priors for Bayesian Variable Selection: https://arxiv.org/abs/1812.07259, primary abstract consulted. Original odds arithmetic independently checked; the forecasting case is fictional.

Previous: Spike-and-slab distribution Next: Spillover (experiment)