AI Glossary / Probabilistic machine learning / Advanced
Stochastic gradient Hamiltonian Monte Carlo
Definition
Stochastic gradient Hamiltonian Monte Carlo, or SGHMC, is a posterior sampling method that combines momentum-based dynamics with gradients estimated from subsets of data. It adds friction and appropriately scaled noise to address the disturbance caused by stochastic gradients. Its purpose is distributional exploration, rather than simply finding one parameter setting with a low loss.
What it means
Ordinary Hamiltonian Monte Carlo uses gradients of the full log posterior to guide movement through parameter space. On a large dataset, evaluating those gradients repeatedly can be expensive. A minibatch is cheaper, but its gradient introduces additional randomness.
That randomness is not automatically the right kind or amount for posterior sampling. Replacing a full gradient with a minibatch estimate while otherwise keeping the same dynamics can produce the wrong distribution. SGHMC treats noise and momentum damping as linked parts of the sampling design. In practical finite-step implementations, gradient-noise estimates and numerical approximation remain important sources of error.
How it works
Define potential energy as the negative log posterior, up to an additive constant. Introduce a momentum variable and a mass matrix. A step moves the parameters according to momentum and adjusts momentum using a minibatch estimate of the potential's gradient, a friction term, and injected Gaussian noise.
The minibatch likelihood contribution must be scaled to estimate the full-data gradient, while the prior contribution is included at its proper scale. Friction removes energy; stochastic gradients and injected noise add it. The method aims to balance these effects so that the intended posterior is preserved by the idealized dynamics.
The original construction specifies noise covariance using a friction matrix minus an estimate of gradient-induced diffusion. This difference must be positive semidefinite. Practical algorithms often use diagonal approximations. The ideal continuous-time stationary result does not establish that every fixed-step implementation samples exactly, especially with inaccurate noise estimates or poor exploration.
Three practical examples
1. Bayesian prediction: draw approximate neural-network parameter samples using minibatches, then combine their predictions. Check predictive uncertainty and sampler behavior rather than equating parameter variability with reliable uncertainty.
2. Recommendation modeling: explore posterior uncertainty in latent user and item factors when full-data gradients are costly. Evaluate whether computational savings preserve the decisions the model supports.
3. Large regression models: compare SGHMC against a more expensive reference sampler on a manageable data subset. This helps identify scaling, friction, or discretization mistakes before moving to the full dataset.
Think of it this way
Think of moving an object through a landscape while receiving noisy pushes about which way the ground slopes. Momentum carries motion forward, friction damps it, and carefully controlled agitation supports exploration. Randomly shaking the object harder is not a substitute for getting that balance right.
Worked example
Consider an original scalar illustration of one specified update, not a demonstration of convergence. Use mass 1, step size 0.1, friction C = 0.5, and gradient-diffusion estimate Bhat = 0.1. Start at parameter theta = 2 with momentum r = 0.4.
Following the position-first order of the original algorithm, the new parameter is 2 + 0.1 times 0.4 = 2.04. Suppose the realized minibatch gradient at that new position is 1.5. The injected Gaussian noise variance is 2 times (0.5 minus 0.1) times 0.1 = 0.08, with standard deviation approximately 0.2828.
For an illustrative realized noise draw of 0.2, the new momentum is 0.4 minus 0.1 times 1.5 minus 0.1 times 0.5 times 0.4 plus 0.2 = 0.43. Without that draw, the deterministic part would be 0.23.
The numerical bookkeeping distinguishes the gradient force, friction, and deliberately injected noise. It does not estimate the true gradient-noise covariance or prove the resulting sequence has the desired posterior. Choosing Bhat = 0.7 with the same C would make the stated injected variance negative, signaling an invalid choice for this update.
Illustrative case study
Fictional case: Linden Forecasting wants posterior predictions from a model trained on millions of records. Its team implements SGHMC and first verifies minibatch gradient scaling against full gradients on a smaller dataset. It tests several step sizes and friction choices and compares posterior summaries with a reference calculation.
For the larger experiment, analysts monitor separate runs, effective sample sizes for relevant summaries, and predictive uncertainty on held-out periods. A fast chain that stays near one region is not accepted solely because it generates many saved states. This fictional workflow makes no claim that SGHMC always outperforms other samplers or that a particular setting guarantees correctness.
Common misconceptions and mistakes
SGHMC is not ordinary momentum optimization with arbitrary noise added. A sequence of low-loss points is not automatically a posterior sample. Avoid claiming that the lack of a full-data acceptance test makes practical output exact. Also distinguish momentum states from model parameters and account for autocorrelation when deciding how much information stored samples provide.
Business use
The business case is reducing per-step data processing while retaining a sampling approach to uncertainty. The relevant cost measure is useful posterior information per unit of computation, including tuning and validation. Teams should document the target posterior, minibatch scaling, integration order, mass, friction, noise treatment, and step-size schedule. These details determine whether a reproduced experiment is meaningfully the same method.
People also ask
Does SGHMC use every observation at each step?
Usually not; it estimates the full-data gradient from a minibatch.
Why add friction?
It counteracts the momentum disturbance associated with noisy dynamics.
Does a fixed small step size guarantee exact posterior samples?
No. Discretization and noise-model errors can introduce bias, and practical convergence must be investigated.
Check your understanding
Answer: 1.
What is the injected noise variance in the example?
Answer: It is 0.08 for the specified update.
Answer: 2.
Why is Bhat greater than C invalid in that scalar formula?
Answer: It gives a negative variance.
Answer: 3.
What distinguishes the goal from optimization?
Answer: SGHMC aims to explore a posterior distribution, rather than only identify an optimum.
Related terms
- Hamiltonian Monte Carlo
- Stochastic gradient Langevin dynamics
- Momentum
- Posterior sampling
- Effective sample size
- minibatch
- friction
Learn next
Understand full-data Hamiltonian Monte Carlo and stochastic gradient scaling first. Then study the relationship between friction and diffusion, and compare small-step SGHMC output with a trusted posterior reference on a simple model.
More in Probabilistic machine learning
- Stationary kernel - A stationary kernel is a covariance function whose value depends on the displacement between two inputs, rather than their absolute locations.
- Statistical linear regression - Statistical linear regression, in nonlinear Bayesian estimation, approximates a relationship between uncertain variables by an affine transformation plus an...
- Stein discrepancy - A Stein discrepancy measures how strongly a candidate distribution violates identities that should hold under a target probability distribution.
- Stochastic neighbor embedding - Stochastic neighbor embedding, or SNE, is a nonlinear dimensionality-reduction method that represents objects in a lower-dimensional space by trying to...
- Stochastic variational inference - Stochastic variational inference, or SVI, uses stochastic optimization to fit a tractable approximation to a Bayesian posterior distribution.
- Stochastic weight averaging - Stochastic weight averaging, or SWA, forms a single neural-network model by taking an equal average of selected parameter vectors along a training trajectory.
Sources
Chen, Fox and Guestrin, Stochastic Gradient Hamiltonian Monte Carlo, introduces the method, its idealized noise-friction balance, and practical Algorithm 2: https://proceedings.mlr.press/v32/cheni14.pdf
The one-step calculation and Linden Forecasting case are original teaching examples; the case is fictional. Independent review consulted Algorithm 2 and Section 3.3 for update order, diffusion correction and finite-step bias.
Previous: Stochastic gradient descent Next: Stochastic gradient Langevin dynamics