Is the assumed family always Gaussian?

No. Gaussian families are common, but other tractable families can be chosen.

Is a Kalman filter always approximate?

In the standard linear-Gaussian setting with the appropriate assumptions, its filtering distribution is exact; nonlinear or non-Gaussian settings motivate approximation.

Does matching two moments preserve a mixture?

It preserves the selected moments, not the mixture structure.

Why can order matter?

Each update operates on an already compressed approximation, so discarded information cannot automatically be recovered.

AI Glossary / Time series and signal processing / Advanced

Assumed-density filter

Definition

An assumed-density filter is a sequential Bayesian approximation method that keeps the evolving probability distribution within a chosen manageable family. After incorporating new information, it replaces the resulting distribution with an approximation from that family. A common Gaussian version matches the updated distribution's mean and covariance before processing the next observation.

What it means

Bayesian updating can produce distributions that become increasingly complicated as observations arrive. Retaining every detail may be too expensive for an online system. Assumed-density filtering controls that complexity by repeatedly compressing the distribution into a fixed representation.

The word assumed concerns the form of the approximation, not a claim that the true posterior must have that shape. A Gaussian approximation, for instance, records a mean and covariance while losing features such as multiple separated peaks. This trade-off can make inference practical, but information lost at an early step can matter when later evidence arrives.

How it works

Choose an approximating family and initialize its parameters. For a static hidden quantity, combine the current approximate distribution with the next likelihood contribution and normalize. Then project the result back into the chosen family. In a dynamic state model, a prediction through the state-transition model precedes the observation update, and approximations may be needed in those stages.

For an unrestricted Gaussian approximation using forward KL projection, matching the target mean and covariance gives the projection when the required moments exist. Computing those moments can itself require approximation. Carry the resulting parameters into the next step. Record the projection rule, numerical integration method and state assumptions, because different choices can produce different filters and uncertainty estimates.

Three practical examples

1. Tracking: A robotics team uses a compact Gaussian state distribution to update a position estimate from streaming measurements. It tests the approximation when measurements become ambiguous or multimodal.

2. Online learning: A model updates approximate parameter beliefs as observations arrive, avoiding storage of an increasingly complicated exact posterior. The team checks whether processing order materially affects results.

3. Sensor fusion: An analytics system combines uncertain observations from several sources. An assumed-density representation makes repeated updates manageable while diagnostics identify situations where a richer distribution may be necessary.

Think of it this way

Think of replacing a detailed landscape with a summary of its center and spread after each new observation. That summary is easy to carry forward, but two distant hills can be turned into one broad hill between them. The next update then starts from the summary, not from the original landscape that was discarded.

Worked example

Consider an original illustration of the projection step alone. Suppose an updated one-dimensional density is a mixture: weight 0.75 on a Gaussian with mean 0 and variance 1, and weight 0.25 on a Gaussian with mean 4 and variance 1. Write N(mean, variance), so the second parameter is not a standard deviation.

The mixture mean is 0.75 × 0 + 0.25 × 4 = 1. Its second moment is 0.75(1 + 0²) + 0.25(1 + 4²) = 0.75 + 4.25 = 5. The variance is second moment minus squared mean: 5 - 1² = 4.

A moment-matched Gaussian approximation is therefore N(1, 4), with standard deviation 2. Simply averaging the component variances would incorrectly give 1 because it would ignore separation between component means. The replacement preserves these first two moments but not the mixture's full shape. This example does not derive the mixture from a complete state-transition and observation model or demonstrate an entire tracking run.

Illustrative case study

Fictional case: Clearpath Robotics tracks a moving device using a sensor that sometimes has two plausible location interpretations. A single-Gaussian filter reports a position between the alternatives. Although its covariance is relatively large, downstream software initially treats the mean as a likely physical location.

The team introduces test cases with ambiguous observations and compares the compact approximation with a richer reference calculation. It documents when the single-Gaussian summary is acceptable and when the system should preserve multiple hypotheses or request another measurement. The lesson is not that Gaussian approximations are unusable, but that a convenient distribution summary must be interpreted in light of what it discards.

Common misconceptions and mistakes

Assumed-density filtering does not mean all observed data are Gaussian. A Gaussian posterior approximation can be used with a non-Gaussian likelihood. Matching mean and variance does not match every probability, tail or mode of the updated distribution.

ADF is not generally exact, and its sequence of projections can make results order-sensitive. Reordering independent observations in an approximate batch calculation can therefore matter even when the exact posterior would be unchanged. Expectation propagation adds iterative refinement and should not be treated as identical to a one-pass ADF procedure. Finally, a covariance matrix produced by software is not automatically evidence of well-calibrated uncertainty.

Business use

The practical benefit is bounded representation complexity for systems that must repeatedly update beliefs. This can matter in real-time monitoring, embedded analytics and streaming models where full posterior computation would be too slow or memory-intensive.

A business evaluation should measure latency and memory alongside estimation quality and uncertainty calibration. Include difficult cases, delayed measurements, outliers and competing hypotheses in validation. If decisions depend on tail probabilities or rare events, check those quantities explicitly. Agreement in average error alone may conceal an approximation that gives misleading confidence in the cases most important to the application.

Check your understanding

What variance does the worked example produce?

Answer: Four.

Why is it larger than either component variance?

Answer: The different component means add between-component variation.

What is carried into the next Gaussian update?

Answer: The approximate mean and covariance, together with the specified model and update procedure.

Related terms

Bayesian filtering
Kalman filter
Expectation propagation
posterior distribution
moment matching
KL divergence
Gaussian approximation
particle filter

Learn next

Learn next how exact Bayesian prediction and measurement updates work, then examine Gaussian moment projection. Compare ADF with particle filtering and expectation propagation on problems where multimodality or processing order matters.

Expectation propagation

Sources

Minka, Expectation Propagation for Approximate Bayesian Inference, Section 2 on assumed-density filtering: https://vismod.media.mit.edu/pub/tpminka/papers/minka-ep-uai.pdf . The primary author text explains sequential projection and its information-loss limitation. This entry states the Gaussian projection convention explicitly; the mixture calculation is an original, independently checked illustration of one approximation step.

Previous: Assortativity Next: Asymmetric cointegration