AI Glossary / Probabilistic machine learning / Advanced

Adaptive resampling

Definition

Adaptive resampling, in sequential Monte Carlo and particle filtering, means deciding whether to resample a population of weighted particles using its current condition rather than resampling at every step. A common trigger is a low effective sample size estimated from the normalized weights. The aim is to address weight concentration while avoiding unnecessary resampling and its additional randomness.

What it means

A particle is a candidate state or trajectory, accompanied by a weight indicating its contribution to the current approximation. After new information arrives, a few particles may receive nearly all the weight. Keeping many negligible-weight particles can waste subsequent computation.
Resampling selects ancestors according to a specified scheme, usually giving high-weight particles more descendants and resetting offspring weights equally. Adaptive resampling chooses when to do this. It does not necessarily change the number of particles, and it is distinct from adaptively rebalancing a training dataset. Here the term refers specifically to sequential probabilistic inference.

How it works

After a weighting step, normalize nonnegative weights so that they sum to one. A common diagnostic is estimated ESS = 1 / sum(w_i squared). Compare it with a chosen threshold, such as half the population size. If the trigger fires, apply a resampling scheme; otherwise retain the weighted population.
The trigger and the resampling algorithm are separate choices. Multinomial, stratified, and systematic schemes can all be used within an adaptive schedule. TensorFlow Probability, for example, documents a criterion based on ESS falling below half the particle count.
The weight-based ESS is a useful proxy, not a count of independent or distinct states. Equal offspring weights can restore its numerical maximum while many offspring still share the same ancestor.

Three practical examples

• A warehouse robot tracks its position with particles. A distinctive sensor observation concentrates the weights, triggering resampling; less informative observations may leave them balanced enough to skip it.
• An equipment-monitoring system follows an unobserved wear state. The team uses a weight-concentration trigger and separately checks whether its transition model represents plausible changes in wear.
• A demand-forecasting prototype updates latent demand states as observations arrive. It records how often resampling occurs alongside forecast accuracy and particle diversity, rather than treating the trigger count as a performance score.

Think of it this way

Imagine allocating a fixed number of research assistants among competing explanations. If evidence strongly favors a few explanations, reallocating effort may be sensible. Doing so after every tiny update could waste effort and prematurely discard alternatives. Reallocating assistants also does not create new evidence or recover an explanation that nobody retained.

Worked example

Take an original four-particle example with states A, B, C, and D and normalized weights 0.7, 0.1, 0.1, and 0.1. The squared weights sum to 0.49 + 0.01 + 0.01 + 0.01 = 0.52. Estimated ESS is 1/0.52, approximately 1.9231.
Use the explicit rule resample when ESS is strictly below N/2. Here N = 4 and the threshold is 2, so the rule triggers. For comparison, equal weights 0.25 each would give ESS = 4 and would not trigger.
To illustrate one multinomial resampling realization, partition the unit interval using cumulative weights 0.7, 0.8, 0.9, and 1.0. Suppose four independent uniform draws happen to be 0.12, 0.43, 0.76, and 0.95. The selected ancestors are A, A, B, and D. Each offspring receives weight 0.25.
The new weight-based ESS is 4, but only three distinct ancestor labels remain, and C is absent. Expected offspring counts before seeing the random draws were 4 times each weight: 2.8, 0.4, 0.4, and 0.4. The realized counts 2, 1, 0, and 1 need not equal those expectations. This example demonstrates a trigger and a random outcome; it does not establish improved estimation accuracy.

Illustrative case study

Fictional monitoring company WearWatch resamples its equipment-state particles after every observation. During quiet periods, repeated selection produces many related trajectories even though the weights were reasonably balanced. The team investigates an adaptive trigger rather than assuming that more frequent resampling is always beneficial.
It compares schedules using the same simulated test scenarios and records weight concentration, ancestor diversity, computation time, and errors against known simulated states. When every schedule struggles after an abrupt regime change, it revisits the transition model instead of merely adjusting the threshold. This fictional case describes an evaluation approach and reports no measured commercial result.

Common misconceptions and mistakes

Resampling does not automatically add new locations to the particle population. Ordinary selection duplicates or removes existing particles; propagation or suitable rejuvenation steps provide other mechanisms for movement. An apparently healthy ESS immediately after resampling therefore does not prove broad coverage of the target.
Another error is calculating the formula with unnormalized weights without the corresponding normalization factor. The general equivalent expression is (sum of weights) squared divided by the sum of squared weights. Thresholds also require a stated comparison rule: below and at-or-below differ at equality. Finally, an adaptive schedule cannot compensate for a likelihood or transition model that systematically excludes the true state.

Business use

Adaptive resampling can help manage computation in sequential uncertainty estimates used for tracking, forecasting, and monitoring. Choose the diagnostic and threshold through task-relevant evaluation, retain logs of resampling events, and inspect diversity as well as weights. Keep the particle count, resampling scheme, random seeds, and model version available for reproducibility. Assess whether estimates remain useful during unusual observations, since average performance can conceal failures precisely when decisions become most consequential.

People also ask

Is half the particle count always the right threshold?

No. It is a common choice, including a documented software default, rather than a universal optimum.

Does the method require changing population size?

No. The worked example keeps four particles throughout.

Is this ESS the same as MCMC ESS?

They address related efficiency concerns but use different calculations; autocorrelation-based MCMC diagnostics should not be substituted blindly.

Check your understanding

Why does the example trigger resampling?

Answer: Its estimated ESS is about 1.9231, below the chosen threshold of two.

Why does ESS become four afterward?

Answer: The offspring have equal weights.

Why is that not proof of four independent states?

Answer: Two offspring share ancestor A, and ancestry introduces dependence and duplication concerns.

Related terms

• Sequential Monte Carlo
• Effective sample size
• Multinomial resampling
• Systematic resampling
• particle filter
• importance weights
• weight degeneracy
• particle impoverishment

Learn next

Compare resampling schemes while holding the trigger fixed. Then study how propagation, proposal quality, and rejuvenation affect diversity, and distinguish improvements in a diagnostic from improvements in the final estimate.

More in Probabilistic machine learning

• Acyclic directed mixed graph - An acyclic directed mixed graph, or ADMG, contains directed edges such as X → Y and bidirected edges such as X ↔ Y, with no cycle made entirely of...
• Adaptive Markov chain Monte Carlo - Adaptive Markov chain Monte Carlo, or adaptive MCMC, changes aspects of a sampling algorithm using information gathered during its run.
• Adaptive rejection sampling - Adaptive rejection sampling, or ARS, draws from a target distribution by constructing and refining an upper envelope used for rejection sampling.
• Advantage actor-critic - Advantage actor-critic is a reinforcement-learning approach in which an actor learns a policy and a critic helps estimate whether an action performed better...
• Alpha divergence - Alpha divergence is a parameterized family of measures for comparing probability distributions. Changing alpha changes how discrepancies are penalized.
• Amortized variational inference - Amortized variational inference learns a shared function that maps observations or other relevant context to parameters of approximate posterior distributions.

Sources

Martino, Elvira and Louzada, Effective Sample Size for Importance Sampling Based on Discrepancy Measures: https://arxiv.org/pdf/1602.03572 TensorFlow Probability, SequentialMonteCarlo API, resampling criteria and schemes: https://www.tensorflow.org/probability/api_docs/python/tfp/experimental/mcmc/SequentialMonteCarlo

Previous: Adaptive rejection sampling Next: Adaptive resonance theory