Is adaptation the same as learning model parameters?

No. Here it concerns the sampling procedure's tuning parameters; posterior draws concern the model parameters.

Must adaptation continue forever?

No. Many workflows restrict history-based tuning to warmup.

Is warmup simply disposable sampling?

It can perform active tuning as well as help the sampler move away from its initial state.

AI Glossary / Probabilistic machine learning / Advanced

Adaptive Markov chain Monte Carlo

Definition

Adaptive Markov chain Monte Carlo, or adaptive MCMC, changes aspects of a sampling algorithm using information gathered during its run. It may adjust proposal scales, covariance estimates, or other tuning parameters to explore a fixed target distribution more effectively. Valid adaptation requires suitable rules: changing a sampler in response to its history does not automatically preserve correct sampling behavior.

What it means

A Bayesian model can define a posterior distribution that is difficult to sample directly. MCMC explores that distribution through dependent steps. Poor tuning can produce tiny movements or frequent rejected proposals, making useful information expensive to obtain.
Adaptation attempts to learn more suitable settings from the sampler's own experience. It tunes how the algorithm moves; it need not change the posterior being studied. The distinction matters because altering a prior or likelihood during the run changes the inferential problem, whereas adapting a proposal aims to solve the existing problem more efficiently.

How it works

A family of transition kernels shares a target distribution. An adaptation rule uses accumulated information to choose which kernel operates next. Because choices can depend on earlier states, the sampled states alone need not form a time-homogeneous Markov chain.
One practical strategy adapts during warmup, freezes tuning parameters, and then collects draws for estimation. Another permits continuing adaptation under conditions supporting convergence. A widely used sufficient framework combines diminishing adaptation, meaning successive kernels become close in probability, with containment, which controls their convergence times along the run. These conditions are not universal necessities or something established merely by watching a trace plot.
In Stan's Hamiltonian samplers, warmup can tune step size and estimate a metric. This is one application of adaptation, not the definition of every adaptive MCMC method.

Three practical examples

An analyst fitting a hierarchical sales model uses warmup to learn scales appropriate to its parameters. The retained sample is then examined for effective sample size and disagreements across chains.
An engineering team calibrates a simulator whose parameters have strong posterior correlation. An adaptive proposal covariance can help align proposed moves with that geometry, subject to a valid algorithm and numerical safeguards.
A researcher compares two parameterizations of a probabilistic model. Each run receives its own adaptation period, and efficiency is assessed through Monte Carlo precision per unit of computing time rather than raw iteration count alone.

Think of it this way

Think of learning an appropriate walking stride for a route. Very small steps make little progress, while excessively large jumps may repeatedly fail. Adjusting the stride can help, but it does not change the destination. Nor does a comfortable stride prove that every part of a complicated route has been explored.

Worked example

Consider an original toy warmup rule for a random-walk Metropolis proposal scale s. Update its logarithm by log(s_new) = log(s_old) + gamma × (a − a_target), where a is one for an accepted proposal and zero for a rejection. Choose a_target = 0.4 purely for illustration; it is not a recommended universal target.
Start with s = 1. An accepted proposal and gamma = 0.5 give log(s_new) = 0 + 0.5 × (1 − 0.4) = 0.3. Hence s_new = exp(0.3), approximately 1.3499. The scale increases after this acceptance.
For the next update, suppose the proposal is rejected and gamma = 0.25. The new log scale is 0.3 + 0.25 × (0 − 0.4) = 0.2, giving s approximately 1.2214. The scale decreases from its previous value but remains above the starting value.
If these were the only two warmup updates, freezing s afterward would stop this adaptation. That would not prove that two warmup steps were adequate or that subsequent samples had converged. This calculation demonstrates the arithmetic of a tuning rule; it is neither a complete sampler nor a proof of its validity. In particular, shrinking gamma alone does not establish all convergence conditions.

Illustrative case study

Fictional forecasting company HarborEstimate runs a Bayesian demand model and notices that one chain moves slowly. Its first response is to keep increasing the number of iterations. A review instead examines parameter scaling, initialization, adaptation settings, and the model's geometry.
The team revises a problematic parameterization, runs fresh warmup for each chain, and checks retained draws using multiple diagnostics. It records warmup separately from the estimation sample and reports Monte Carlo uncertainty for the quantities used in planning. This fictional workflow illustrates investigation; it reports no verified improvement and does not suggest that tuning alone can repair an unsuitable statistical model.

Common misconceptions and mistakes

High acceptance is not necessarily good mixing: a sampler making nearly invisible moves may accept almost everything. Likewise, an attractive target acceptance rate is not a certificate of convergence.
Another mistake is assuming that any history-dependent sequence of individually valid kernels must remain valid. Adaptive selection can break that reasoning. Continuing adaptation requires appropriate theory, and stopping adaptation still leaves ordinary convergence and mixing questions. Finally, adaptation does not make retained draws independent, discover every separated mode, or remove the need to examine model assumptions.

Business use

Use established adaptive samplers to reduce manual tuning when posterior computation is a genuine bottleneck. Keep the model, software settings, random seeds, warmup length, retained draws, and diagnostic results together. Evaluate whether the uncertainty in decision-relevant estimates is sufficiently small, rather than celebrating a large number of generated draws. If diagnostics remain poor, investigate parameterization and model structure before assuming additional computing time will solve the problem.

Check your understanding

Why use a logarithmic scale update?

Answer: Exponentiating it keeps the proposal scale positive.

Why did the second worked update reduce the scale?

Answer: The observed acceptance indicator was below the chosen target.

Does every valid fixed kernel guarantee a valid adaptive combination?

Answer: No; the selection mechanism and its assumptions matter.

Related terms

Markov chain Monte Carlo
Proposal distribution
Effective sample size
Hamiltonian Monte Carlo
Metropolis–Hastings
warmup
diminishing adaptation
containment

Learn next

Study the Metropolis - Hastings acceptance rule and how proposal geometry affects exploration. Then distinguish sampler adaptation, convergence diagnostics, and Monte Carlo uncertainty when interpreting a Bayesian computation.

Monte Carlo

Sources

Roberts and Rosenthal, Coupling and Ergodicity of Adaptive MCMC, 2007 manuscript: https://www2.stat.duke.edu/~scs/Courses/Stat376/Papers/AdaptiveMC/RobRosCouplingErgod2007.pdf Stan Reference Manual, MCMC Sampling and automatic parameter tuning: https://mc-stan.org/docs/reference-manual/mcmc.html

Previous: Adaptive Learning Rate Methods Next: Adaptive neuro fuzzy inference system