Its correctly constructed transition can preserve the exact target despite a discretized proposal. A finite simulation still has convergence and Monte Carlo error.

How is it different from unadjusted Langevin sampling?

The unadjusted method omits the acceptance correction and can have step-size-dependent stationary bias.

Does it need the normalizing constant?

Usually not. An evaluable unnormalized target and the required gradient suffice for the standard acceptance calculation.

AI Glossary / Probabilistic machine learning / Advanced

Metropolis-adjusted Langevin algorithm

Definition

The Metropolis-adjusted Langevin algorithm, or MALA, is a Markov chain Monte Carlo method that uses gradients of a log target density to guide proposals, then applies a Metropolis - Hastings acceptance correction. It aims to sample a distribution rather than merely locate its maximum.

What it means

An ordinary random-walk proposal moves without using information about the local shape of the target. MALA combines a move toward higher log density with Gaussian noise, which can make proposals more informed.

The gradient does not eliminate randomness or guarantee acceptance. A finite numerical step only approximates the underlying Langevin dynamics. The acceptance correction accounts for that proposal mechanism so the intended target can remain invariant.

The method requires suitable evaluation of both the target density and its gradient. An unknown constant normalizing the target is usually unnecessary because it cancels in density ratios and vanishes when differentiating the log density.

How it works

In one common convention, choose a positive step parameter h and propose y = x + (h/2) gradient log pi(x) + square-root(h) times z, where z is standard Gaussian noise. The proposal covariance is h times the identity matrix.

Evaluate the Gaussian proposal density q(y given x) and the reverse density q(x given y). Accept with probability min(1, pi(y) q(x given y) divided by pi(x) q(y given x)). Otherwise keep x. The gradient generally makes the proposal asymmetric, so dropping the proposal-density ratio changes the algorithm.

A fixed positive-definite preconditioner can adapt drift and noise to parameter scales. State-dependent versions require additional care. Step size affects acceptance and exploration, while gradients add computational cost. Convergence diagnostics and effective sample size remain necessary for finite runs.

Three practical examples

A Bayesian regression experiment samples continuous coefficients using derivatives of the log posterior.
A latent spatial model uses gradient information to propose changes to a high-dimensional continuous field.
A computational-statistics team compares MALA with random-walk and Hamiltonian samplers on the same smooth target, measuring useful samples per unit of computation.

Think of it this way

Think of a walker who feels the local slope but also takes random steps. A separate checking rule decides whether to keep each proposed move so the walk spends the right amount of time across the landscape.

The analogy must not be interpreted as ordinary hill climbing. Sampling requires visits to lower-density regions too, and the checking rule depends on both directions of the proposal mechanism.

Worked example

Why the acceptance correction matters even for a Gaussian

Target a standard normal distribution on the line, whose log-density gradient is −x. With h = 1, the proposal is y = 0.5x + z, where z is standard normal.

Start at x = 0 and consider a proposed y = 1. The target-density ratio is exp(−1/2), approximately 0.606531. But the proposal is not symmetric: q(1 given 0) has Gaussian mean 0, whereas q(0 given 1) has Gaussian mean 0.5.

Their reverse-to-forward ratio is exp(−0.125 + 0.5) = exp(0.375). Multiplying by the target ratio gives exp(−0.125), approximately 0.882497. A uniform draw of 0.7 therefore accepts this proposal. Using only the target ratio would incorrectly reject it.

Now imagine removing the acceptance correction altogether. The update becomes the autoregressive process x_next = 0.5x + z. Its stationary variance is 1/(1 − 0.25) = 4/3, not the target variance 1.

This simple calculation isolates discretization bias in the unadjusted update. MALA's correction restores target invariance under the appropriate conditions, but it does not make successive samples independent or prove that a short run has converged.

Illustrative case study

A fictional engineering team builds a posterior model for continuous calibration parameters. Random-walk proposals move slowly across differently scaled directions, so the team tests a preconditioned MALA implementation.

It first checks gradients and proposal densities on a Gaussian target with known moments. It verifies that rejection retains the current state and that the reverse proposal uses the gradient at the proposed point.

For the real model, it compares several chains and reports effective sample size relative to runtime. The team does not claim that using gradients automatically resolves multimodality or produces a trustworthy uncertainty estimate from a short run. This scenario describes a validation process rather than measured results.

Common misconceptions and mistakes

Gradient guidance makes proposals symmetric: the proposal mean changes with the current point, so reverse probabilities generally differ.
It is stochastic gradient descent: MALA targets a distribution through a specified acceptance rule, not an optimization objective alone.
Every noisy minibatch gradient preserves exactness automatically: altered proposals and approximate target evaluations require their own valid analysis.
Higher acceptance is always better: tiny steps can be accepted frequently while exploring very slowly.

Business use

MALA can be useful when gradients are available and the posterior is smooth enough for local guidance to help. Its suitability depends on geometry, constraints, multimodality and the cost of derivative evaluations.

Compare methods on the quantities that support the decision, such as tail probabilities or predictive intervals. Parameters constrained to be positive may need transformations with the appropriate density adjustment. A convenient implementation should document its step-size convention because some descriptions use h while others use its square root. Copying a numerical setting across conventions can substantially alter behavior.

Check your understanding

A MALA proposal has a higher target density than the current point. Must it be accepted?

Answer: No. Acceptance also includes the reverse-to-forward proposal-density ratio. A higher target density alone does not determine the final ratio.

Related terms

Preconditioning
Hamiltonian Monte Carlo
Stationary distribution
Effective sample size
Langevin dynamics
Metropolis–Hastings
log-density gradient
unadjusted Langevin algorithm

Learn next

Derive MALA's Gaussian proposal for a standard normal target and calculate both proposal directions. Compare its accepted chain with the unadjusted autoregressive update. Then study parameter transformations and preconditioning before applying it to a complicated posterior.

Preconditioning

Sources

MCMCLib documentation, Metropolis-adjusted Langevin Algorithm, gradient-based Gaussian proposals and acceptance correction: https://mcmclib.readthedocs.io/en/latest/api/mala.html

The standard-normal calculation and fictional calibration case are original teaching illustrations. This entry explicitly uses proposal variance h; notation for step parameters differs across implementations.

Previous: Metrology study Next: Metropolis-Hastings