Does the target need to be normalized?

No. Its common normalizing factor cancels in the acceptance ratio.

Can the proposal depend on the current state?

A standard independence sampler uses a fixed proposal independent of the current state. A state-dependent proposal belongs to a more general Metropolis - Hastings construction.

Is it rejection sampling?

No. Rejections retain the current chain state, and the acceptance calculation compares target-to-proposal ratios at current and proposed states.

AI Glossary / Probabilistic machine learning / Advanced

Independence Metropolis sampler

Definition

An independence Metropolis sampler is a Metropolis - Hastings algorithm that proposes each candidate from a fixed distribution independent of the chain's current state. An acceptance rule corrects the proposal so that the intended target distribution is stationary under suitable conditions.

What it means

The independence refers to the proposal mechanism, not generally to the sequence of retained states. A rejected candidate leaves the chain at its previous state, creating dependence and repeated values.

This method is useful when a tractable approximation to the target distribution is available. The approximation supplies candidates, while the acceptance step accounts for the difference between proposal and target.

A good proposal should cover the target's important regions. Being close around the main peak may be insufficient if the proposal assigns too little probability to a tail or another mode. The sampler can become sticky after reaching a region that the proposal rarely visits.

How it works

Let pi be the target density or mass function and q the fixed proposal. From current state x, draw candidate y from q. Accept with probability min(1, pi(y)q(x)/(pi(x)q(y))). Otherwise retain x as the next chain state.

The target's unknown normalizing constant cancels in the ratio, so an unnormalized target can be used. The proposal probabilities still matter: independence does not make q(x) and q(y) equal.

Writing w(z) = pi(z)/q(z), the acceptance probability becomes min(1, w(y)/w(x)). This connects the algorithm to importance weights, but the output is a Markov chain rather than a collection of independently weighted proposal samples. Support and convergence conditions must be checked for the intended target.

Three practical examples

A Bayesian analyst uses a fitted approximation as a global proposal for a posterior whose density is evaluable up to normalization.
A small discrete-model study checks a sampling implementation against exactly calculable target probabilities.
An inference researcher compares an independent proposal with a local random-walk proposal on a multimodal target.

Think of it this way

Think of suggesting destinations from the same travel catalog regardless of where a traveler currently stands. A separate acceptance rule decides whether the traveler moves or stays.

The suggestions can be independent while the traveler's recorded locations remain dependent. The analogy explains the naming; real sampling probabilities require the precise acceptance formula, not a subjective preference for a destination.

Worked example

Checking a two-state sampler

Suppose the target probabilities are pi(A) = 0.8 and pi(B) = 0.2. Choose the fixed proposal q(A) = q(B) = 0.5.

From A, a proposal of B is accepted with probability (0.2 × 0.5)/(0.8 × 0.5) = 0.25. Since B is proposed half the time, the total transition probability from A to B is 0.5 × 0.25 = 0.125.

From B, a proposal of A is accepted with probability 1, so the transition probability from B to A is 0.5. The stationary flows balance: 0.8 × 0.125 = 0.1 and 0.2 × 0.5 = 0.1.

The resulting transition rows are A to (A, B) = (0.875, 0.125) and B to (A, B) = (0.5, 0.5). Their stationary distribution is the desired (0.8, 0.2), even though the proposal is uniform.

Recording only changes of state would alternate A and B and incorrectly give equal frequencies. Repeated states are part of the sample path and must be retained or represented with their correct multiplicities.

Illustrative case study

Fictional case: assessing a posterior approximation

A fictional research team has a convenient approximate posterior and wants to use it as an independence proposal. The proposal reproduces the center of a reference example well, but the chain occasionally remains at one tail value for a long time.

The analyst checks target-to-proposal ratios and finds that the proposal underrepresents that tail. The team revises the proposal and compares multiple chains, effective sample sizes and the posterior quantities needed for its analysis.

It does not interpret high acceptance in the central region as proof that the full target has been explored. This fictional case highlights proposal coverage and chain behavior rather than claiming a universally successful correction strategy.

Common misconceptions and mistakes

An independent proposal does not generate independent retained samples except in special cases, such as proposing exactly from the target with every candidate accepted.

The acceptance rule is not simply min(1, pi(y)/pi(x)) unless the proposal terms cancel. A nonuniform independent proposal is generally not symmetric between two states.

Rejecting a candidate does not mean removing an iteration from the record. Dropping repeated states changes the empirical distribution. Likewise, a valid stationary distribution is not proof that a short run has converged or that estimates have negligible Monte Carlo error.

Business use

Use an independence sampler when there is a plausible global proposal and reliable target evaluations. Test it first on a small case where exact quantities are available. Record the proposal and target versions so sampling results can be reproduced.

Assess the chain's exploration and the uncertainty in estimates that drive the business calculation. Compare diagnostics across multiple starts, and investigate unusually large target-to-proposal ratios. If the approximation misses important regions, extending a short run may be less effective than redesigning the proposal or choosing another sampler.

Check your understanding

Let pi(A) = 0.8, pi(B) = 0.2, q(A) = 0.6 and q(B) = 0.4. What is the acceptance probability for a proposed move from A to B?

Answer: It is min(1, 0.2 × 0.6 divided by 0.8 × 0.4) = 0.375. This is conditional on B having been proposed.

Related terms

Learn next

Compare the retained chain with its proposal sequence in a two-state example. Then examine how a proposal that underweights a target tail can create long runs of repeated states.

Sources

Author-written technical notes: Gautam Iyer, Metropolis Hastings Revisited, independence-sampler construction and proposal-coverage condition: https://www.math.cmu.edu/~gautam/c/2024-387/notes/09-metropolis-hastings2.html

Official implementation reference: SAS, Markov Chain Monte Carlo Method: https://support.sas.com/documentation/cdl/en/statug/66103/HTML/default/statug_introbayes_sect016.htm

The transition calculation and research-team case are original illustrations.

Previous: Independence Next: Independent component analysis