Is the entire posterior exact?

Generally no; particles approximate part of it.

Must the conditional calculation use a Kalman filter?

No; other tractable structures are possible.

AI Glossary / Time series and signal processing / Advanced

Rao-Blackwellized particle filter

Definition

A Rao-Blackwellized particle filter estimates a changing hidden state by sampling some variables while calculating the conditional distribution of others analytically. It exploits a model's structure so that quantities with a tractable conditional solution need not also be represented by random point samples.

What it means

A particle can carry more than one guessed state value. It can hold a sampled history together with a conditional probability distribution for another part of the state. Combining these distributions with particle weights produces a mixture representation.

The analytical part is exact only under the assumed conditional model. The sampled part remains an approximation, and a misspecified model can still give misleading estimates. Marginalizing a variable does not mean ignoring it or assuming its uncertainty is zero.

How it works

Split the hidden variables into a sampled component and a conditionally tractable component. Propagate the sampled component, update its associated conditional distribution using the new observation, and calculate appropriate importance weights. Normalize the weights and resample when the chosen algorithm requires it.

In a conditionally linear Gaussian model, a particle can carry a Kalman-filter mean and covariance. Other tractable structures can use other exact conditional calculations. The essential requirement is conditional tractability, not that the entire system is Gaussian.

Analytical updates remove one source of Monte Carlo variation, but they also cost computation. Compare accuracy at a realistic computing budget; fewer sampled dimensions do not guarantee faster execution in every implementation.

Three practical examples

1. Robot tracking: sample alternative motion or location histories while maintaining a conditional distribution for associated continuous quantities.

2. Equipment monitoring: represent alternative operating regimes and conditionally estimate a changing numerical state within each regime.

3. Sequential business models: maintain alternative latent scenarios while updating conditionally tractable quantities as observations arrive. The mathematical structure must justify the split.

Think of it this way

Imagine keeping several possible explanations for a delivery delay. For each explanation, you retain a whole estimated arrival-time distribution rather than drawing one arrival time and forgetting its uncertainty. The explanations have weights, and the final forecast combines their distributions.

Worked example

Consider an invented filter output for a machine's hidden load, X. At one time point, simplify its particle mixture to two components. Component A has normalized weight 0.6 and a conditional Gaussian distribution with mean 10 and variance 4. Component B has weight 0.4, mean 16 and variance 9.

These are stipulated posterior components after processing observations. They are not raw observations, and the example does not claim to derive their weights from a sensor reading. A real filter may retain many particles with different histories.

The mixture mean is the weighted average of the conditional means:

E[X] = 0.6 × 10 + 0.4 × 16 = 12.4.

The filter can calculate this value directly. It does not need to draw a load from each Gaussian merely to estimate this mean for the given mixture.

However, reporting only the two conditional means would lose information about uncertainty within each component. Calculate the average conditional variance:

0.6 × 4 + 0.4 × 9 = 6.

Next calculate the variance of the conditional means around the mixture mean:

0.6 × (10 - 12.4)^2 + 0.4 × (16 - 12.4)^2 = 8.64.

The total mixture variance is the sum, 6 + 8.64 = 14.64. Thus the state remains uncertain even though its mixture mean was computed without sampling the conditional load values. Using 8.64 alone would omit within-component uncertainty; using 6 alone would omit uncertainty between the components.

A second calculation confirms the result. The conditional second moments are variance plus squared mean: 4 + 100 = 104 for A and 9 + 256 = 265 for B. Their weighted average is 0.6 × 104 + 0.4 × 265 = 168.4. Subtracting the squared mixture mean gives 168.4 - 12.4^2 = 14.64.

To see the sampling distinction, imagine drawing one independent load value from each conditional Gaussian and forming 0.6 times the first draw plus 0.4 times the second. Conditional on these fixed components, that random estimator has expectation 12.4 and variance 0.6^2 × 4 + 0.4^2 × 9 = 2.88. Replacing those draws with their conditional expectations removes that additional sampling variance for this calculation.

The 2.88 describes variation in this particular mean estimator across repeated draws. The 14.64 describes uncertainty about the hidden load under the mixture. They answer different questions. Neither value measures errors caused by an incorrect machine model, and the fixed-component comparison is not a complete performance analysis of a running particle filter.

If later evidence changes the weights, the mixture summaries change. Keeping all component parameters fixed but setting the weights to 0.5 and 0.5 gives mean 13, average conditional variance 6.5 and between-component variance 9, hence total variance 15.5.

Illustrative case study

Fictional Alder Maintenance initially stores one sampled load per operating hypothesis. Its reported average fluctuates between repeated runs even when the inputs remain unchanged.

The team uses a model with tractable conditional load distributions and retains their means and variances instead. It checks the mixture summaries with the calculations above, then benchmarks the complete filter across repeated runs. The invented case illustrates an implementation choice, not measured evidence that a particular monitoring system became more accurate.

Common misconceptions and mistakes

"Marginalized variables have no uncertainty." Their conditional distributions retain it.

"Each particle is just a point in the complete state space." Here it can carry a conditional distribution.

"Lower sampling variance means the physical state is less uncertain." Estimation noise and state uncertainty differ.

"The method is always faster." Conditional calculations can be expensive.

Business use

Ask which variables are sampled, which are handled analytically and which assumptions make that calculation valid. Benchmark accuracy, memory and latency together. Preserve component uncertainty when producing operational forecasts, and test how results change under plausible model errors.

Check your understanding

What is the example's mixture mean?

Answer: 12.4.

Why is its variance greater than 6?

Answer: Different component means add 8.64.

What does 2.88 measure?

Answer: Extra variation from the specified random mean estimator.

Related terms

Sequential Monte Carlo
Marginalization
Conditional distribution
Kalman filter
Particle filter
importance weight
mixture model
state estimation

Learn next

Keep the original weights and means, but set both conditional variances to one. Recalculate total state variance and the extra variance of the two-draw mean estimator. Explain why neither change removes the difference between component means.

Sources

Arnaud Doucet, Nando de Freitas, Kevin Murphy and Stuart Russell, Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks, UAI 2000: https://people.eecs.berkeley.edu/~pabbeel/cs287-fa12/optreadings/DoucetdeFreitasMurphyRussell_RaoBlackwellizedParticleFilters.pdf

Nando de Freitas, research software descriptions, Rao Blackwellised Particle Filtering for dynamic mixtures of Gaussians: https://www.cs.ubc.ca/~nando/software.html

Previous: Rao-Blackwellization Next: Rapidly exploring dense trees