Can a factorial HMM be expressed as an ordinary HMM?

Yes, by using a state for every component combination, but that may create a much larger state space and obscure useful transition structure.

Are its observations necessarily sums?

No. Additive emissions are a useful formulation, while other joint observation models can be defined.

Is exact inference always impossible?

No. Small models can be tractable. The difficulty is how exact computation scales with the number and sizes of the hidden components.

AI Glossary / Graphs and symbolic AI / Advanced

Factorial HMM

Definition

A factorial hidden Markov model, or factorial HMM, represents a time-varying hidden state as several component state variables rather than one categorical variable. In the standard form, these components follow separate Markov chains, while each observation can depend on their combined states.

What it means

An ordinary HMM uses one hidden state at each time step. If the situation involves several underlying processes, a single state may need to encode every combination of their settings. A factorial HMM makes those components explicit and places structure on their transitions.

For example, an aggregate measurement might depend on two devices, each switching between operating modes. Separate chains describe each device's persistence, and the observation model describes their combined measurement. Factorial refers to the combined state space, not the factorial operation or factorial experimental design.

How it works

Each hidden chain has an initial distribution and transition probabilities. In the standard uncoupled transition structure, a chain's next state depends on its own previous state rather than directly on the other chains' previous states. The observation at a time step depends jointly on the current component states.

This reduces transition parameterization compared with an unrestricted HMM over all combined states. With M components each having K states, the combined state space still contains K raised to M combinations. Compact transition parameters therefore do not eliminate the challenge of inference.

Observations can make the hidden chains statistically dependent in the posterior, even when their prior dynamics are independent. Exact inference may be feasible for small models, while larger models often require sampling or variational approximations. An additive Gaussian observation model is one important formulation, but the broader idea is not limited to one particular emission distribution.

Three practical examples

1. A monitoring model represents several devices whose operating states contribute to a combined reading. Separate temporal chains encode persistence, while the emission model connects their combined settings to the measurement.

2. A sequence-analysis experiment models several latent musical characteristics evolving over time. Factorial structure provides a way to combine hidden components rather than allocating an unrelated state to every possible combination.

3. A synthetic signal study uses separate hidden sources to generate observations. Researchers can compare recovered state estimates with the known simulated states while examining how overlapping contributions create ambiguity.

Think of it this way

Think of several dimmer switches contributing to the brightness measured by one light sensor. Each switch has its own pattern of changes, but the sensor sees the combined effect. Knowing the brightness may not reveal which switch caused it. The analogy explains shared observations; a factorial HMM additionally specifies probability distributions, time steps and precise conditional-independence assumptions.

Worked example

Consider two binary hidden chains A and B, initially independent with probability 0.5 of being one. For this teaching example, the observation is noiseless: Y = A + B. Before observing Y, the four combined states 00, 01, 10 and 11 each have probability 0.25.

Now observe Y = 1. Only states 01 and 10 can produce that value. Each receives posterior probability 0.5; 00 and 11 receive zero. The separate posterior marginals remain P(A = 1) = P(B = 1) = 0.5, but A and B are no longer independent. Multiplying those marginals would incorrectly assign 0.25 to impossible combinations.

To add temporal dynamics, suppose each chain stays in its current state with probability 0.9, independently of the other chain's transition given their current states. From combined state 10, the probability of remaining 10 is 0.9 × 0.9 = 0.81, while switching to 01 is 0.1 × 0.1 = 0.01. These calculations illustrate transition factorization and posterior coupling; they are not a full fitted model or an inference benchmark.

Illustrative case study

Fictional case study: Silverleaf Lab models a combined machine reading using two hidden operating processes. The team initially tries to infer each process independently from the same reading. A small simulated example reveals that one observed value can be explained by different combinations, so independent answers can contradict the shared measurement.

The team adopts a joint observation model and evaluates an inference method that accounts for the resulting coupling. It reports uncertainty when several explanations remain plausible. This illustrates model design and ambiguity, without claiming successful device identification or a measured improvement on real data.

Common misconceptions and mistakes

Independent hidden transitions do not imply independent hidden states after conditioning on observations. This is the central trap demonstrated by the worked example. A factorial HMM is also different from several unrelated HMMs fitted to separate data streams: a shared observation model links the components during inference.

The structure does not guarantee that learned components correspond to meaningful physical sources. Different parameter settings or component labelings may explain similar observations. Finally, replacing the full state with several variables reduces some parameters but leaves exponentially many joint configurations, so computational claims must specify the inference method and model size.

Business use

Use the concept when a sequence plausibly reflects several persistent hidden processes and the proposed transition structure is defensible. Define the sampling interval, component states and observation model. Check whether processes really can evolve independently in the prior or whether cross-process interactions require a different dynamic network.

Evaluate prediction, state estimation and uncertainty on data relevant to the intended decision. Where possible, use controlled or labeled examples to test whether the components are identifiable. Record approximation choices during inference and learning. A plausible decomposition should be treated as a model interpretation supported by evidence, not automatically as a discovered causal mechanism.

Check your understanding

After Y = 1 in the worked example, what is P(A = 1, B = 1)?

Answer: Zero, because that combination would produce Y = 2. The product of separate marginals would give 0.25, showing why those marginals cannot be treated as independent.

Related terms

Hidden Markov model
Dynamic Bayesian network
Latent variable
Markov Chain
Gibbs sampling
emission model
posterior dependence
variational inference

Learn next

Study ordinary HMM filtering first. Then build a two-chain example with a shared observation and enumerate the joint states, comparing exact posterior probabilities with a product-of-marginals approximation.

Sources

Primary source: Zoubin Ghahramani and Michael I. Jordan, “Factorial Hidden Markov Models,” Machine Learning, 1997: https://mlg.eng.cam.ac.uk/pub/pdf/GhaJor97a.pdf . The source defines the model and its inference challenges. The binary-chain calculations and fictional case here are original teaching examples.

Previous: Factorial design Next: Factoring