Does the model need known transition probabilities?

The inference calculation needs model parameters. They may be specified beforehand or estimated separately.

Is forward-backward identical to Baum-Welch training?

No. Forward-backward inference supplies quantities used in Baum-Welch, which iteratively updates model parameters.

Can smoothing remove all uncertainty?

No. Ambiguous observations and overlapping state behavior can leave substantial posterior uncertainty.

AI Glossary / Graphs and symbolic AI / Intermediate

Forward-backward algorithm

Definition

The forward-backward algorithm computes the probability distribution of each hidden state in a sequence after considering observations from the whole sequence. In a hidden Markov model, it combines information arriving from earlier observations with information supplied by later observations.

What it means

A hidden state is something the system cannot observe directly, such as the operating condition behind a sequence of sensor readings. An observation offers indirect evidence about that state.

Filtering estimates the current state using observations available up to that time. Smoothing revisits an earlier state using observations that arrived afterward. Forward-backward is an efficient way to obtain these smoothed marginal distributions across a sequence.

The result is a distribution at each time, not necessarily the single most probable complete state path. Finding that path is a different problem commonly handled by the Viterbi algorithm. Selecting the most probable state independently at each time need not recover the most probable path.

How it works

The forward pass propagates state probabilities through the transition model and incorporates each observation's likelihood. Depending on notation, forward messages are stored as joint probabilities or normalized filtering distributions.

The backward pass starts at the end and summarizes the likelihood of later observations conditional on each possible earlier state. At each time, the forward and backward information is multiplied state by state and normalized to obtain the smoothed distribution.

For a standard dense model with K states and T observations, the usual computation takes order T times K squared operations. Implementations use scaling or log-domain calculations to avoid numerical underflow on long sequences. These calculations are exact for the specified finite-state model, but model mismatch can still make its probabilities poor descriptions of reality.

Three practical examples

Equipment analysis: A team revises earlier operating-state estimates after reviewing a complete sensor sequence.
Sequence labeling: A probabilistic language model calculates uncertainty about a token's hidden label using surrounding observations.
Movement analysis: A system retrospectively estimates likely hidden motion states from a sequence of noisy position-related observations.

Think of it this way

Think of rereading an earlier scene after learning what happens later in a story. Later evidence can change how you interpret the earlier scene without changing what actually happened. The analogy explains retrospective inference; the algorithm uses explicit probabilities and conditional-independence assumptions rather than narrative judgment.

Worked example

Consider hidden states A and B at times 1 and 2. Initially, each state at time 1 has probability 0.5. The state stays the same with probability 0.8 and switches with probability 0.2.

The observation can be yes or no. A produces yes with probability 0.9, while B produces yes with probability 0.2. Therefore their no probabilities are 0.1 and 0.8.

Observe yes at time 1. The unnormalized forward values are 0.5 × 0.9 = 0.45 for A and 0.5 × 0.2 = 0.10 for B. Filtering gives P(A at time 1 | yes) = 0.45 / 0.55, approximately 0.818.

Now observe no at time 2. The backward likelihood for A at time 1 is 0.8 × 0.1 + 0.2 × 0.8 = 0.24. For B it is 0.2 × 0.1 + 0.8 × 0.8 = 0.66.

Multiply forward and backward values: A receives 0.45 × 0.24 = 0.108; B receives 0.10 × 0.66 = 0.066. Normalize to obtain P(A at time 1 | yes, no) = 0.108 / 0.174, approximately 0.621.

The later no observation reduces the earlier estimate from about 81.8 percent to 62.1 percent. It does not prove that state 1 was B; it changes the distribution under the model.

Illustrative case study

Fictional case: Willow Sensors reviews a machine's recorded operating data after each test session. Its dashboard originally labels earlier states using only the evidence available at those times.

The team adds a retrospective view using forward-backward smoothing. It keeps that view distinct from the live filtering display, because the smoothed results depend on later observations that were unavailable during operation.

During evaluation, the team checks the transition assumptions and sensor likelihoods against held-out sessions. It also compares model confidence with independent annotations where available. The case illustrates how later evidence can improve retrospective interpretation while avoiding an unfair claim that the same information supported earlier live decisions.

Common misconceptions and mistakes

Backward messages predict the past by reversing causation. They calculate likelihoods of later evidence conditional on earlier states.
Smoothing and filtering use the same information. Smoothing uses later observations as well.
The algorithm returns the most likely full path. Its primary output here is the marginal distribution at each time.
Exact inference guarantees an accurate real-world model. Exactness concerns the calculation for the supplied assumptions.

Business use

Forward-backward is useful for retrospective analysis of sequences and as an inference component in some model-training procedures. It allows an application to retain uncertainty rather than reporting only a sequence of hard labels.

Specify the observation window and distinguish retrospective estimates from live outputs. Check numerical stability, impossible observation sequences and model calibration. When benchmarking a real-time service, avoid using smoothed labels as if their later evidence were available at prediction time. That would change the information available to the task.

Check your understanding

An estimate for time 5 uses observations through time 10. Is it a filtering estimate for time 5?

Answer: No. It is a smoothed estimate, because it uses observations that occur after time 5.

Related terms

Learn next

Enumerate the four possible two-state paths in the worked example and calculate their joint probabilities with the observations. Sum the paths beginning in A and verify the smoothed result. Then compare that marginal calculation with choosing the most probable entire path.

Sources

Russell and Norvig provide forward-backward smoothing pseudocode: https://aima.eecs.berkeley.edu/algorithms.pdf

Carnegie Mellon's course notes derive the forward and backward messages: https://www.cs.cmu.edu/~15281/coursenotes/hmm_pf/index.html

Rabiner's tutorial develops hidden Markov model inference and estimation: https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf

The two-state calculation and Willow Sensors case are original illustrations.

Previous: Forward algorithm Next: Forward-chaining