Does FFBS return the most likely sequence?

No. It draws sequences according to their posterior probabilities.

Is it restricted to discrete states?

No. Linear Gaussian models have Gaussian backward conditionals, and other models have additional variants.

Why sample backward after filtering forward?

Filtering stores the information needed to construct backward conditionals while preserving the posterior dependence between neighboring states.

AI Glossary / Probabilistic machine learning / Intermediate

Forward filtering backward sampling

Definition

Forward filtering backward sampling (FFBS) is a method for drawing complete hidden-state sequences from their posterior distribution given observed data. It first computes filtering distributions in time order, then samples the final state and works backward, drawing each earlier state conditional on the already sampled next state and the relevant observations.

What it means

When a process has unobserved states, uncertainty concerns an entire history, not just each date separately. A machine may have been in normal or stressed operation, for example, and adjacent states are related by its transition model. FFBS generates plausible histories that preserve those relationships.

This differs from reporting a posterior mean trajectory, selecting a single most probable sequence or independently sampling each time's marginal distribution. The result of one FFBS run is one possible posterior history. Repeated runs allow estimation of quantities that depend on several times together, such as the duration of a state or the chance that a transition occurred.

How it works

For a first-order state-space model with fixed parameters, filter forward to obtain p(x_t | y_1:t). Draw x_T from the final filtering distribution, which is also the final-time smoothing distribution. Then move backward using p(x_t | x_(t+1), y_1:T), proportional to p(x_t | y_1:t) times p(x_(t+1) | x_t).

The Markov assumptions explain why the next sampled state and filtering information are sufficient for the backward conditional. Later observations already influence the sampled future state. They should not be inserted again as an independent likelihood contribution.

Finite-state hidden Markov models allow categorical backward draws, and linear Gaussian state-space models allow Gaussian draws. These versions can be exact for their specified models and fixed parameters. Nonlinear or non-Gaussian extensions may use approximations or particles. If parameters are uncertain, one can incorporate FFBS within a broader sampler rather than pretending the parameters are known.

Three practical examples

1. Operations analysis: sample possible histories of a machine's hidden operating regime and examine how long stressed periods may have lasted. Report the uncertainty rather than treating one sampled history as the definitive record.

2. Demand modeling: draw latent demand-level trajectories from a state-space model and propagate them into later calculations. Keep observation noise distinct from uncertainty in the underlying level.

3. Sequence research: sample hidden labels consistent with an observed sequence and transition rules. This is useful when several plausible labelings should be considered rather than only the highest-scoring one.

Think of it this way

Think of reconstructing several plausible routes through a sequence of checkpoints. After choosing a possible final checkpoint, earlier choices must be compatible with how the route could have reached it. The analogy captures dependence across time. FFBS uses conditional probabilities, not an investigator's intuition, and a plausible sampled route is not necessarily the route that actually occurred.

Worked example

Consider two binary states at times one and two. After the first observation, suppose the filter assigns probability 0.6 to state 0 and 0.4 to state 1. The transition probability of next state 1 is 0.2 from state 0 and 0.7 from state 1. Thus its predicted probability is 0.6 × 0.2 + 0.4 × 0.7 = 0.4.

Suppose the second observation has likelihood 0.2 in state 0 and 0.9 in state 1. The final posterior probability of state 1 is 0.4 × 0.9 / (0.6 × 0.2 + 0.4 × 0.9) = 0.75.

If the backward sampler draws final state 1, the earlier weights are 0.6 × 0.2 = 0.12 for state 0 and 0.4 × 0.7 = 0.28 for state 1. Normalizing gives earlier-state probability 0.7 for state 1. The joint posterior probability of the path (1, 1) is therefore 0.75 × 0.7 = 0.525.

For final state 0, the earlier-state-1 conditional probability is 0.12/0.6 = 0.2. Hence the earlier smoothed marginal is 0.75 × 0.7 + 0.25 × 0.2 = 0.575. Independently sampling the two marginals would assign path (1, 1) probability 0.575 × 0.75 = 0.43125, which is wrong for this model.

Illustrative case study

Fictional case: Maple Maintenance estimates periods of high equipment load from incomplete sensor records. Its analyst samples whole regime histories and calculates stressed-period durations on each history. The results are summarized as a distribution rather than a single reconstructed sequence.

Engineers compare these estimates with known interventions and investigate how alternative transition assumptions affect them. They avoid presenting a random draw as an observed fault history. This fictional example illustrates preserving sequence uncertainty; it is not evidence that the chosen model diagnoses real equipment correctly.

Common misconceptions and mistakes

FFBS does not run the physical process backward. The backward conditional is derived from the forward model and filtering results. Sampling each state independently from its smoothed marginal loses dependence and can produce implausible or impossible sequences.

Exact sampling conditional on fixed parameters does not eliminate uncertainty about those parameters or validate the model. Approximate filters can produce approximate samples. Numerical normalization matters, especially for long sequences or extremely small likelihoods. Random seeds support reproducibility but do not substitute for sufficient sampling or model checks.

Business use

Use FFBS when a downstream question depends on whole histories, such as how many transitions happened or how long a regime persisted. Preserve the sampled sequences long enough to compute those joint quantities; separate time-by-time intervals cannot reconstruct every dependency.

Document whether parameters are fixed, estimated or jointly sampled. Assess sensitivity to transition probabilities and observation assumptions. When FFBS is embedded in MCMC, examine convergence of the overall chain rather than assuming the state-block draw makes every iteration independent.

Check your understanding

In the example, why is independent marginal sampling incorrect?

Answer: It gives path (1, 1) probability 0.43125 instead of 0.525 because it ignores the posterior relationship between the two states.

Related terms

Hidden Markov model
Filtering
Smoothing
Posterior sampling
Gibbs sampling
state-space model
simulation smoother
Markov dependence

Learn next

Enumerate every path in a short hidden Markov model and compare its exact posterior with frequencies from backward sampling. Then study the Gaussian backward conditional and how state sampling fits within parameter inference.

Hidden Markov model

Sources

Sokolov, Bayes, Artificial Intelligence, and Deep Learning, forecasting chapter: https://vsokolov.org/html/_book/15-forecasting.html

astsa authors' FFBS documentation: https://search.r-project.org/CRAN/refmans/astsa/html/ffbs.html

Independent review consulted Sokolov’s backward conditional derivation and Stoffer’s astsa documentation on posterior sequence sampling with fixed parameters. Original probabilities verified.

Previous: Forward checking Next: Forward-looking centers