Does every Markov chain perform useful MCMC?

No. Its stationary distribution and exploration properties must suit the intended target.

Does MCMC return the best parameter value?

Its main purpose is sampling a distribution. An optimization routine addresses a different objective.

Can more iterations help?

Often, but additional iterations may be ineffective when the chain cannot reach important regions or the implementation targets the wrong distribution.

AI Glossary / Graphs and symbolic AI / Intermediate

Markov chain Monte Carlo

Definition

Markov chain Monte Carlo, or MCMC, is a family of methods for estimating quantities from a probability distribution by generating a dependent sequence of samples. The method constructs a Markov chain whose stationary distribution is the desired target. With suitable conditions and enough exploration, sample averages approximate expectations under that distribution.

What it means

Many AI and statistical models describe plausible values for several unknown quantities together. Calculating every probability or integrating over every possibility can be impractical. MCMC replaces some of that calculation with a carefully designed simulation.

The chain's states may be whole parameter vectors rather than individual observations. Its simulation steps are computational steps, not necessarily minutes, days, or real changes in a business. The goal is generally to characterize uncertainty, rather than to find only the single most likely state. A sample that frequently revisits a region can be doing exactly what its target distribution requires.

How it works

Specify the target distribution, often using a density known only up to a normalizing constant. Choose an update rule that preserves it. Gibbs sampling updates variables from their conditional distributions; Metropolis - Hastings proposes changes and accepts or rejects them using a correction rule. These are different members of the same broad family.

Initialize the chain and run updates. Under appropriate convergence conditions, the initial state becomes less influential and the distribution of states approaches the target. Use retained draws to estimate means, probabilities, intervals, or other summaries.

Check whether the simulation explores the relevant regions adequately. Multiple chains, trace plots, effective sample sizes, Monte Carlo standard errors, and suitable convergence diagnostics provide evidence about reliability. They cannot prove that every important region has been found in a finite run.

Three practical examples

Demand modeling: sample uncertain seasonal effects and store-specific parameters, then propagate those draws into forecasts of future order volumes.
Reliability analysis: explore plausible failure-rate parameters in a hierarchical model, preserving uncertainty for groups with limited observations.
Image modeling: update hidden labels or latent variables in a probabilistic model when directly enumerating all possible configurations is too expensive.

Think of it this way

Think of a surveyor walking through a landscape with rules designed to spend the right proportion of time in each region. The resulting route can reveal averages across the landscape without listing every location beforehand.

The analogy is limited: an ordinary walk does not automatically have the desired distribution. Poorly designed moves can trap the surveyor in one valley, and a long route alone does not establish representative coverage.

Worked example

Consider a tiny target distribution with states Low and High, having probabilities 0.75 and 0.25. Construct a chain that moves from Low to High with probability 0.1 and otherwise remains Low. From High, it moves to Low with probability 0.3 and otherwise remains High.

Check stationarity: if the current distribution is the target, the next probability of Low is 0.75×0.9+0.25×0.3=0.75. The target is therefore preserved. Both states communicate and both have self-transitions, so this finite chain is irreducible and aperiodic.

Assign values 2 and 10 to Low and High. The exact target mean is 0.75×2+0.25×10=4. A long, adequately explored run can estimate it by averaging its state values. If a hypothetical retained sample has 760 Low and 240 High visits, its average is 3.92. That count is an illustration, not an executed simulation, and the 1,000 dependent visits must not be treated as 1,000 independent observations when assessing simulation error.

Illustrative case study

Fictional case: a distributor fits a Bayesian demand model with separate effects for branches. One chain appears stable but rarely moves between two different explanations of a seasonal pattern. Another initialization settles in the alternative region.

The analyst investigates the disagreement, revises the parameterization, and reruns the model before presenting forecast intervals. The team also checks forecasts against held-out observations. This separates a computation problem from a model-validation problem: even accurate sampling would not establish that the demand model describes future business conditions. No numerical performance improvement is claimed in this fictional case.

Common misconceptions and mistakes

MCMC is not synonymous with independent random sampling, and repeated states are not automatically an error. In rejection-based methods, recording the unchanged state is part of the correct chain.

Discarding a fixed number of initial draws does not guarantee convergence. Nor does obtaining a large raw sample count resolve strong dependence or failure to move between modes. Thinning is not a universal remedy for poor sampling. Better transitions, parameterization, initialization checks, or a different inference method may be needed.

Separate Monte Carlo error from posterior uncertainty: one concerns numerical estimation, while the other reflects uncertainty within the assumed model.

Business use

MCMC can support decisions where a single estimate hides important uncertainty or dependencies. For instance, a capacity planner can evaluate several policies across sampled parameter settings rather than treating estimated demand as certain.

The practical cost includes computation, diagnosis, and model review. Reports should identify the target model, sampler, retained draws, relevant diagnostics, and numerical precision of decision-critical summaries. Compare the value of detailed uncertainty modeling with simpler alternatives when the decision does not require a complex posterior calculation.

Check your understanding

two chains remain in different regions and give different means. Is averaging them enough to establish a reliable answer?

Answer: no. Their disagreement signals inadequate exploration or another problem. Investigate the sampler and model before trusting a pooled result; equal pooling need not assign the regions their correct probabilities.

Related terms

Markov Chain
Stationary distribution
Gibbs sampling
Bayesian inference
Effective sample size
Metropolis–Hastings
posterior distribution
Monte Carlo error
mixing

Learn next

verify the two-state stationarity calculation, then examine how changing both movement probabilities affects persistence while preserving the same target ratio. Study Gibbs sampling and Metropolis - Hastings, followed by practical diagnostics and the distinction between sampling uncertainty and model uncertainty.

Gibbs sampling

Sources

UC Berkeley Stat210A, “Markov Chain Monte Carlo,” chain construction and Gibbs sampling: https://stat210a.berkeley.edu/fall-2025/reader/bayes-computation.html. Stan Reference Manual, “Posterior Analysis,” convergence and dependent-draw diagnostics: https://mc-stan.org/docs/reference-manual/analysis.html. Relevant primary teaching and implementation documentation was read. The two-state arithmetic and hypothetical sample counts are original illustrations.

Previous: Markov Chain Next: Markov decision process