AI Glossary / Probabilistic machine learning / Advanced
Sequential Monte Carlo approximate Bayesian computation
Definition
Sequential Monte Carlo approximate Bayesian computation, or SMC-ABC, uses successive populations of weighted parameter samples to approximate Bayesian inference when simulation is feasible but the likelihood is difficult to evaluate. Populations typically move through decreasing discrepancy tolerances, concentrating simulations on parameter regions that produce data resembling the observations.
What it means
Approximate Bayesian computation compares simulated data with observed data instead of directly calculating a likelihood. Usually the comparison uses summary statistics and a distance measure. Parameters are retained when the simulated summaries are sufficiently close to the observed summaries under a stated tolerance.
Basic rejection ABC can waste many simulations by repeatedly drawing from a broad prior. Sequential population methods reuse information from earlier accepted particles to construct more useful proposals. Here sequential commonly refers to a sequence of tolerances or targets, not necessarily the arrival of time-series observations. The approximation still depends on summaries, tolerance and simulator assumptions.
How it works
Begin with a prior, simulator, summary statistics, discrepancy and initial tolerance. Generate an initial particle population using an appropriate ABC procedure. At later stages, propose parameters using the previous weighted population, often by resampling a particle and perturbing it. Simulate new data and apply the tighter acceptance criterion.
Accepted particles require weights consistent with the chosen sequential scheme. In a common population importance-sampling construction, the proposal density is the weighted mixture of perturbation kernels centered at the previous particles. An accepted particle receives weight proportional to prior density divided by that mixture proposal density, followed by normalization.
Other SMC-ABC variants use different target transitions or move steps, so their weighting rules must be derived rather than copied indiscriminately. Tolerances may follow a preset schedule or an adaptive rule. Monitor particle diversity, weight concentration and simulation cost as the target becomes harder to reach.
Three practical examples
1. Ecological simulation: infer parameters of a simulator whose population dynamics can be generated but whose likelihood is inconvenient. Summary statistics must retain information relevant to the intended parameter questions.
2. Queueing models: compare simulated waiting-time patterns with observed service data. Check whether the simulator can reproduce the key patterns before interpreting a narrow parameter population.
3. Synthetic inference benchmarks: compare SMC-ABC with an exact posterior in a small tractable model. This separates approximation from algorithmic error before attempting a complex likelihood-free application.
Think of it this way
Think of searching for plausible recipes by simulating their results, then concentrating later trials near recipes that previously looked promising. Tighter comparisons progressively restrict acceptable results. The analogy explains adaptive search, but the statistical method also needs correct weights; repeatedly favoring successful recipes without correction does not automatically produce a posterior distribution.
Worked example
Use a two-value parameter space only to illustrate one accepted-population weighting step. The prior gives theta_A and theta_B probability 0.5 each. Suppose the current proposal distribution gives them probabilities 0.8 and 0.2. One proposed particle of each type passes the current ABC criterion.
Their unnormalized importance weights are 0.5/0.8=0.625 for A and 0.5/0.2=2.5 for B. Normalizing by their sum 3.125 gives weights 0.2 and 0.8. Equal weights would ignore how much more frequently the proposal tends to produce A. These are the weights of this two-particle realization, not a claim that the exact posterior probabilities are 0.2 and 0.8.
Separately, suppose the observed scalar summary is 10 and two simulated summaries are 9.8 and 10.4. With absolute discrepancy and tolerance 0.5, both pass. At tolerance 0.3, only 9.8 passes. A real next population generally involves new proposals and simulations with its own correct weighting; it is not fully described by simply deleting the second particle. The toy calculation isolates proposal correction and tolerance, rather than implementing a complete SMC-ABC algorithm.
Illustrative case study
Fictional case: Alder Simulation fits a queueing simulator to service records. Its first sequential implementation keeps equally weighted accepted particles after proposing from a concentrated previous population. A reviewer asks for the proposal density and finds that the required importance correction was omitted.
The team fixes the specified algorithm and tests it on a simple case with a known reference posterior. It then checks whether the waiting-time summaries capture both typical delays and long tails. This fictional case illustrates algorithm and summary validation, not a claim that a tighter tolerance ensures an adequate service model.
Common misconceptions and mistakes
Likelihood-free does not mean assumption-free or data-free. The simulator defines a statistical model, the prior influences inference and the summaries determine what similarity means. A very small tolerance cannot recover information discarded by insufficient summaries.
Increasing particle count reduces some Monte Carlo error but does not remove tolerance bias or simulator misspecification. A collapsed population can miss plausible modes, and high effective sample size alone cannot prove that all modes were found. Reusing earlier particles without valid weights or move kernels can bias the result. Parameters that explain similar summaries may also remain poorly identifiable.
Business use
Consider SMC-ABC when the simulator is credible and direct likelihood-based inference is genuinely impractical. Budget simulator calls as well as particle-processing overhead. Document random seeds, tolerances, summary scaling and proposal construction so the approximation is reproducible.
Assess sensitivity to summaries and tolerances, and compare replicated simulations with observations beyond the statistics used for acceptance. Report the output as an approximate posterior under the chosen model and comparison rule. For operational decisions, examine whether uncertainty in poorly identified parameters changes the recommendation instead of relying only on a single best-fitting particle.
People also ask
Does sequential mean online observations?
Not necessarily. Many SMC-ABC methods sequence populations through increasingly strict tolerances for one fixed dataset.
Why are weights needed?
Proposals concentrated around earlier particles are generally not draws from the prior or final target; weights correct the specified proposal-target mismatch.
Does zero tolerance always give the full-data posterior?
Not if information has been lost through insufficient summaries, and exact matching may be computationally infeasible for continuous data.
Check your understanding
Why does the accepted B particle receive four times A's weight in the example?
Answer: The prior treats them equally, but B was four times less likely under the proposal. The prior-to-proposal correction compensates for that difference.
Related terms
- Approximate Bayesian computation
- Effective sample size
- particle population
- importance weight
- discrepancy
- tolerance schedule
- summary statistic
- simulator
Learn next
Implement rejection ABC for a small model with known likelihood, then compare a correctly weighted sequential population scheme. Study sensitivity to summaries and tolerance separately from Monte Carlo variability.
More in Probabilistic machine learning
- Semi-amortized variational inference - Semi-amortized variational inference combines an inference network's fast prediction of variational parameters with additional optimization for each...
- Semisupervised variational autoencoder - A semisupervised variational autoencoder is a latent-variable generative model trained using both labeled and unlabeled observations.
- Sequential model-based optimization - Sequential model-based optimization, or SMBO, is an approach to optimizing an expensive objective by repeatedly fitting a surrogate model, using it to...
- Shampoo optimizer - Shampoo is an adaptive gradient optimizer that uses the matrix or tensor structure of model parameters to build manageable preconditioners.
- Slice sampling - Slice sampling is a Markov chain Monte Carlo method that samples a distribution by introducing an auxiliary height beneath its unnormalized density curve.
- Sliced score matching - Sliced score matching is a method for learning a probability distribution's score by comparing score vectors along random directions.
Sources
Beaumont, Cornuet, Marin and Robert, Adaptive Approximate Bayesian Computation: https://arxiv.org/pdf/0805.2256
The population proposal correction and decreasing-tolerance rationale were checked in the original paper. The discrete weighting example and fictional queueing case are original illustrations. Independent review consulted Section 3.1 for the mixture-proposal importance correction.
Previous: Sequential Monte Carlo Next: Sequential pattern mining