Must observations be Gaussian?

No. Their distribution can be chosen for the data, such as a Bernoulli model for binary observations.

Does first-order mean the model cannot remember history?

No. The inferred latent state can carry information accumulated from earlier observations.

Are hidden states exactly recovered?

Generally not. They are inferred under modeling assumptions, commonly with an approximate posterior, and may not have a unique interpretation.

AI Glossary / Probabilistic machine learning / Intermediate

Deep Markov model

Definition

A deep Markov model is a probabilistic sequence model with latent states whose transition or observation distributions are parameterized by neural networks. In a common formulation, each hidden state depends on the previous hidden state, and each observation depends on its corresponding state. The hidden states remain uncertain rather than being fixed neural activations.

What it means

Many sequences contain an underlying condition that cannot be measured directly. A sensor reading, for example, may reflect a hidden operating state plus measurement noise. The model describes how that state changes and how it produces observations.
The word “deep” refers to neural functions within the probabilistic model, not to the number of previous time steps that directly determine the next state. A first-order latent transition can still support complicated observed behavior. After the hidden states are integrated out, the observed sequence need not itself satisfy a first-order Markov assumption.

How it works

Specify an initial state distribution, a transition distribution p(z_t | z_(t−1)), and an observation distribution p(x_t | z_t). Neural networks can produce transition means, variances, or parameters of the observation distribution. Additional known inputs may be included when the chosen model requires them.
An inference network approximates the distribution of hidden states given observations. Training commonly optimizes a variational lower bound, balancing explanatory fit with consistency between approximate inferred states and the generative model.
Distinguish filtering from smoothing. Filtering uses observations available up to the current time; smoothing can use later observations to infer earlier states. When evaluating a forecast, later observations must not quietly enter the inference procedure. Generate future states and observations from information genuinely available at the forecast time.

Three practical examples

Equipment sequences: infer an uncertain operating condition from noisy measurements and predict future readings under the fitted transition model.
Activity measurements: represent changing latent patterns behind repeated observations without assuming that every measurement directly reveals the hidden state.
Missing sequence values: estimate distributions for unobserved measurements using surrounding observations, while clearly labeling this as retrospective reconstruction when future observations are included.

Think of it this way

Think of watching a machine through a partially fogged window. The machine has an evolving condition, while each view provides imperfect evidence about it. Later views can help reinterpret an earlier one. The analogy is limited: learned hidden coordinates need not correspond to named mechanical conditions, and good prediction alone does not establish that the coordinates describe physical causes.

Worked example

Consider an original scalar illustration. Let the next state obey z₂ | z₁ ~ Normal(tanh(z₁), 0.04), where 0.04 is a variance. Let the next observation be x₂ = 2z₂ + e, with independent e ~ Normal(0, 0.09).
For this calculation only, assume z₁ is known to equal 0. Since tanh(0) = 0, z₂ has mean 0 and variance 0.04. The observation mean is therefore 0, and its variance is 2² × 0.04 + 0.09 = 0.25. Its standard deviation is 0.5.
If we replace z₂ by its mean before predicting x₂, we retain only measurement variance 0.09, giving standard deviation 0.3. That shortcut omits the uncertainty from the state transition.
This is a small nonlinear state-space calculation, not a trained deep network. In a full model, uncertain z₁ and nonlinear observation functions can make the predictive distribution more complicated. The calculation deliberately conditions on a known starting state so its variance accounting can be checked exactly.

Illustrative case study

Fictional case: Fern Systems studies hourly equipment measurements. Its analysts train a deep Markov model and use retrospective smoothing to inspect gaps in historical records. They separately assess forecasting with an inference procedure that sees only the available past.
The retrospective reconstructions look more accurate than forecasts, as expected because they use extra observations. The team reports those tasks separately rather than presenting reconstruction scores as evidence of forecasting performance. It also checks whether latent states are stable across model fits before assigning them operational names. No business improvement is claimed by this illustrative case.

Common misconceptions and mistakes

A Markov assumption about latent states does not imply that only the most recent measured value matters to prediction. Earlier measurements can improve inference about the current hidden state.
An ordinary deterministic recurrent network is not automatically a deep Markov model. The probabilistic state and observation structure is central here.
Do not treat inferred latent states as directly observed facts, or interpret neural transition relationships as causal effects without additional assumptions and evidence.
Finally, using a bidirectional inference network for retrospective analysis can be appropriate, but using unavailable future observations to evaluate a live forecast introduces leakage. The intended information boundary must be explicit.

Business use

The business value lies in modeling sequential uncertainty and separating changing hidden conditions from noisy measurements. Possible uses include forecasting and retrospective reconstruction, provided the evaluation matches the proposed use.
Compare with simpler state-space models and other sequence predictors. Review forecast accuracy, predictive interval behavior, missing-data assumptions, computation, and sensitivity to the latent-state design. Where outputs influence operations, communicate whether a result is a forecast, a filtered estimate, or a smoothed historical estimate. These distinctions determine what information the result legitimately uses.

Check your understanding

A model estimates Monday's hidden state using observations from Monday through Friday. Is that estimate a Monday-time forecast?

Answer: No. It is a retrospective estimate using future information relative to Monday. It can be useful for smoothing or analysis, but it must not be scored as though that information was available on Monday.

Related terms

Markov property
Filtering
Smoothing
Data leakage
State-space model
latent state
transition distribution
observation model
variational inference

Learn next

Start with a linear Gaussian state-space model to distinguish transitions from measurements. Next compare filtering and smoothing information sets. Then study neural transition models and structured variational inference, checking which distributions are approximated and which observations the inference network can access.

Sources

Krishnan, Shalit, and Sontag, Structured Inference Networks for Nonlinear State Space Models, original generative and inference formulations: https://arxiv.org/pdf/1609.09869 Independent source review: Original DMM generative graph, structured posterior factorization and past/future inference-network discussion were read.

Previous: Deep linguistic processing Next: Deep model