Is acceptance probability the same as posterior probability?

No. In Metropolis-Hastings it is a transition decision probability constructed from target and proposal quantities. It is not the posterior probability that a model or candidate is correct.

Should every sampler aim for the same acceptance rate?

No. Useful rates depend on the algorithm, proposal, dimension and target geometry. A single universal percentage is inappropriate.

What happens after a rejected Metropolis-Hastings proposal?

The current state is retained as the next state of the chain. Rejection does not mean the iteration disappears.

AI Glossary / Statistics and probability / Intermediate

Acceptance probability

Definition

Acceptance probability is the probability that an algorithm accepts a proposed candidate, given its current state and decision rule. In sampling and stochastic search, it determines whether a proposed move becomes the next state. The formula depends on the algorithm; it is not automatically the probability that the candidate is correct or optimal.

What it means

Many algorithms separate proposing a candidate from deciding whether to use it. A proposal can be computationally convenient without having exactly the desired distribution or quality. An acceptance rule then controls which proposed moves are taken.

In Metropolis-Hastings sampling, the acceptance probability helps construct a Markov chain with a specified target distribution. In simulated annealing, a related acceptance mechanism can allow worse moves during optimization. These uses have different objectives. The phrase also appears in other statistical settings, so the surrounding algorithm should always be identified.

How it works

For Metropolis-Hastings, let x be the current state, y the proposal, pi the target density up to a common normalizing constant, and q(y|x) the proposal density. The standard acceptance probability is min(1, pi(y)q(x|y) divided by pi(x)q(y|x)), wherever the ratio is well defined.

Draw a uniform random number U between zero and one and accept the proposal if U is below this probability. Otherwise the chain remains at x. That repeated state is part of the chain and should not simply be discarded from its recorded trajectory.

For symmetric proposals, the proposal terms cancel, leaving the target-density ratio capped at one. Asymmetric proposals require the correction. An implementation should handle support restrictions and numerical calculations carefully, commonly working with log densities. A valid acceptance formula alone does not establish that a finite run has mixed well or produced accurate estimates.

Three practical examples

1. A Bayesian analyst uses Metropolis-Hastings to explore a posterior distribution whose normalizing constant is difficult to calculate.

2. An optimization experiment uses simulated annealing to accept some temporarily worse candidates, with the probability depending on the objective change and temperature.

3. A simulation developer inspects acceptance behavior alongside movement and effective sample size to diagnose a proposal mechanism that may be exploring too slowly.

Think of it this way

Think of a candidate move passing through a probabilistic gate. The proposal mechanism brings the candidate to the gate; the acceptance rule determines its chance of passing. A gate that opens frequently is not necessarily doing a better job, because the candidates may be almost identical to the current state.

Worked example

Consider an original Metropolis-Hastings example with positive unnormalized target values pi(x) = 4 and pi(y) = 2. First assume a symmetric proposal. The acceptance probability is min(1, 2/4) = 0.5. If the uniform draw is 0.3, accept y; if it is 0.8, keep x.

Now use an asymmetric proposal with q(y|x) = 0.4 and q(x|y) = 0.1. The acceptance probability becomes min(1, (2 x 0.1)/(4 x 0.4)) = 0.2/1.6 = 0.125. A draw of 0.3 now leads to rejection.

The target values did not change, but the proposal correction did. Multiplying both target values by any common positive constant would leave the ratio unchanged. The example illustrates a single decision; it does not describe the long-run distribution of an inadequately constructed chain or prove that the proposed state is more useful for an optimization objective.

Illustrative case study

Fictional case study: SampleForge builds a posterior sampler for a forecasting experiment. A developer records only accepted proposals and reports a very high acceptance rate after reducing the proposal step size.

A review finds two separate issues. Omitting repeated states changes the recorded chain, and tiny steps can lead to slow exploration even when most proposals are accepted. The team restores the full state sequence and examines trace plots and effective sample size alongside acceptance statistics. It also checks that the proposal correction matches the actual proposal mechanism.

This fictional case illustrates implementation and diagnostic concerns. It does not establish a universal target acceptance rate or claim that the revised sampler has converged without further evidence.

Common misconceptions and mistakes

Acceptance probability for one move is different from the observed acceptance rate across many moves. The latter is a summary of realized decisions and depends on the visited states and proposal distribution.

Another mistake is accepting only moves to higher target density when the objective is sampling. Such a rule generally destroys the intended distribution. Conversely, an acceptance probability of one does not prove that the proposal is optimal or that the chain explores efficiently. Do not omit asymmetric proposal terms or interpret a target density value as a probability without considering its mathematical role.

Business use

For businesses using simulation or probabilistic models, understanding acceptance probability helps distinguish a functioning algorithm from a misleading diagnostic. The relevant question is whether the full procedure produces trustworthy estimates for the intended purpose.

Document the target, proposal and acceptance rule together. Verify calculations on small examples with known behavior before applying them to larger models. Monitor exploration and estimation uncertainty rather than optimizing acceptance rate alone. If proposals or tuning rules change during a run, ensure the adaptation method preserves the required sampling properties instead of assuming any adjustment is harmless.

Check your understanding

With a symmetric proposal, the current unnormalized target value is 3 and the proposed value is 9. What is the Metropolis acceptance probability?

Answer: min(1, 9/3) = 1. The proposal is accepted, but this does not mean its posterior probability is 100% or that it is an optimum.

Related terms

Proposal distribution
Detailed balance
Markov chain Monte Carlo
Metropolis-Hastings algorithm
target distribution
acceptance rate
simulated annealing and effective sample size

Learn next

calculate acceptance probabilities for symmetric and asymmetric proposals on a small discrete state space. Then compare chains that accept frequently but move little with chains that make larger moves, recording all states and examining estimation accuracy.

State space

Sources

Primary sources: AIMA author-maintained exercises, Metropolis-Hastings acceptance rule: https://aimacode.github.io/aima-exercises/question_bank/

Stephens, The Metropolis Hastings Algorithm: https://stephens999.github.io/fiveMinuteStats/MH_intro.html

Bouchard-Côté, Metropolis-Hastings and slice samplers: https://www.stat.ubc.ca/~bouchard/courses/stat520-sp2020-21/T16-mh-through-slice.html

Previous: Acceptable quality limit Next: Acceptance sampling