Does Bayesian lasso require a spike-and-slab prior?

No. The usual Laplace shrinkage prior is continuous; spike-and-slab models explicitly introduce a different structure for inclusion.

Can it provide credible intervals?

Yes, from the posterior, subject to the model and the quality of inference.

Is the posterior mode the only useful summary?

No. Means, medians, intervals and posterior predictions answer different questions.

AI Glossary / Probabilistic machine learning / Advanced

Bayesian lasso

Definition

The Bayesian lasso is a Bayesian regression approach that uses Laplace, or double-exponential, priors to shrink coefficients toward zero. It extends the connection between lasso penalization and posterior modes into inference about entire posterior distributions, including uncertainty in coefficients and potentially other model parameters.

What it means

Ordinary lasso produces a penalized coefficient estimate, often with some coefficients exactly zero. Bayesian lasso instead describes plausible coefficient values after combining a likelihood with shrinkage priors. A posterior mean, median, credible interval and mode are different summaries of that distribution.
The distinction matters: continuous shrinkage does not automatically give positive posterior probability that a coefficient equals exactly zero. A zero mode can coexist with a continuous posterior spread around it. Decisions about removing variables therefore require an explicit selection rule or a different model with discrete inclusion indicators.

How it works

For Gaussian regression, combine the response likelihood with Laplace priors on coefficients. In the Park - Casella formulation, the conditional prior has density proportional to exp(−λ|β_j|/σ), with scale σ/λ. The coefficient prior is linked to the residual standard deviation σ.
A hierarchical representation writes each coefficient as conditionally normal with its own latent variance scale, and assigns exponential distributions to those scales. This supports Gibbs sampling. The shrinkage parameter can be fixed, estimated or assigned a hyperprior. Its interpretation depends on predictor scaling and the exact likelihood and prior conventions; numerical penalty parameters cannot be compared blindly across implementations.

Three practical examples

A demand analyst could fit a regression with many correlated candidate predictors while reporting posterior uncertainty in their associations with demand. Shrinkage would not establish that any predictor causes sales.
An operations team could compare forecasts from a shrinkage model and a simpler baseline using held-out periods. It would assess prediction quality rather than selecting a method solely because many coefficients appear small.
A research group could examine how posterior coefficient summaries change under different shrinkage assumptions. This would help distinguish conclusions strongly supported by the data from conclusions sensitive to the prior.

Think of it this way

Imagine gently pulling a collection of flexible indicators toward a central reference point. Strong evidence can pull an indicator away again, while weak evidence may leave it close to the center. Reporting the single highest point of a probability curve is different from reporting its whole spread. The analogy describes shrinkage, not a physical mechanism that switches variables on or off.

Worked example

Use an original one-coefficient illustration. Observe z from a normal distribution with mean β and known variance one. Give β a Laplace prior with rate one, so its density is proportional to exp(−|β|). This is a simplified fixed-scale example, not a full unknown-variance regression.
The negative log posterior, ignoring constants, is J(β)=0.5(z−β)²+|β|. With z=3, the positive-side derivative is β−3+1=β−2. The minimum is β=2. Direct checks give J(2)=2.5, J(0)=4.5 and J(3)=3. The prior has moved the posterior mode from the unpenalized estimate 3 to 2.
Now consider z=0.5 with the same prior. To the right of zero, the derivative is β+0.5, which is positive. To the left, it is β−1.5, which is negative. The minimum is therefore at zero, where J(0)=0.125. At the unpenalized estimate β=0.5, J(0.5)=0.5.
Nevertheless, the posterior is a continuous density. Its probability at the single point β=0 is zero, even though that point is the mode. A neighborhood around zero can have positive probability. The calculation illustrates the lasso-style mode connection; it does not give a posterior mean, credible interval or variable-inclusion probability.

Illustrative case study

Fictional supply planner Elm Forecast wants to evaluate many candidate signals for weekly demand. Its analysts standardize predictors consistently, reserve later periods for evaluation and compare several shrinkage settings. They report coefficient intervals alongside predictive performance.
A manager interprets a coefficient with a mode near zero as proof that the signal has no effect. The analyst explains that the posterior summary concerns an association under the model, and that removing the signal is a separate decision. The team documents its selection rule and tests the resulting smaller model. This fictional case demonstrates interpretation and validation; it claims no achieved forecast improvement.

Common misconceptions and mistakes

“Bayesian lasso and ordinary lasso always return identical estimates.” The mode connection does not make posterior means or medians identical to penalized estimates.
“A zero mode means a positive probability of an exact zero coefficient.” Continuous posterior densities do not assign probability mass to individual points.
“An interval containing zero proves no relationship.” It indicates uncertainty under the model and prior, rather than proving absence.
“Predictor units are irrelevant.” Rescaling predictors changes coefficient magnitudes and the practical effect of a common shrinkage prior unless handled consistently.
“Shrinkage resolves every issue with correlated predictors.” Strong correlations can still complicate attribution and interpretation.

Business use

Use the approach when regression uncertainty and regularization both matter. Decide whether the objective is prediction, description or a smaller feature set, because each may require different evaluation and reporting. Document prior scales, predictor transformations and the treatment of the intercept. Compare predictive performance on relevant held-out data and examine sensitivity to shrinkage assumptions. If a variable is removed, record the decision rule rather than presenting the removal as an automatic posterior fact.

Check your understanding

What is the posterior mode when z=3 in the example?

Answer: 2.

Why is the mode zero when z=0.5?

Answer: the objective decreases toward zero from the left and increases away from zero on the right.

Does that give a nonzero probability that β is exactly zero?

Answer: no, because the posterior remains continuous.

Related terms

Shrinkage
Gibbs sampling
Lasso regression
Laplace prior
maximum a posteriori estimation
hierarchical model
credible interval
spike-and-slab prior

Learn next

Compare posterior modes with posterior means in a one-dimensional shrinkage example. Then study how predictor standardization and prior scale affect multivariable regression and posterior prediction.

Sources

Park, T., and Casella, G. The Bayesian Lasso, Journal of the American Statistical Association, 2008, Sections 1–3: https://people.eecs.berkeley.edu/~jordan/sail/readings/park-casella.pdf The fixed-variance calculation and fictional business scenarios are original illustrations.

Previous: Bayesian interpretation of kernel regularization Next: Bayesian learning