Is the recurrent state directly observed?

No. It is a learned internal representation rather than a sensor measurement.

Can the model predict without new images?

Yes. It can advance through its transition prior under supplied actions, although prediction errors may accumulate.

Is a deterministic recurrent update incompatible with uncertain futures?

No. Stochastic latent samples can differ, and those differences influence later recurrent updates.

AI Glossary / Probabilistic machine learning / Advanced

Recurrent state space model

Definition

A recurrent state space model is a learned sequence model that combines a deterministic recurrent memory with a stochastic latent state. In the widely used PlaNet formulation, these components jointly support predictions of future observations and rewards, including rollouts without receiving a new observation at every step.

What it means

An image or sensor reading rarely reveals everything needed to predict the future. A single picture of a moving object, for example, may not reveal its velocity. A sequence model can retain information from earlier readings while representing uncertainty about hidden aspects of the situation.
The deterministic component supplies a structured memory update. The stochastic component allows different possible latent outcomes instead of forcing every history and action to produce one fixed future. This entry focuses on the RSSM architecture introduced for latent planning; other models may use related terminology with different equations.

How it works

In a common notation, update recurrent memory as h_t = f(h_(t-1), z_(t-1), a_(t-1)). The transition prior p(z_t | h_t) predicts a distribution over the current stochastic state before using the current observation. When observation o_t is available, an inference network produces an approximate posterior q(z_t | h_t, o_t).
Observation and reward models use the combined state (h_t, z_t). During imagined rollouts, sample from the transition prior and advance memory under candidate actions. During observation-conditioned training, use the inference distribution and train reconstruction or prediction terms together with a KL term connecting posterior and prior.
This is a model of dynamics, not by itself a complete action-selection rule. A planner or learned policy must use it. A deterministic memory also need not be identical across imagined trajectories: previous sampled latent states can produce different subsequent memory values.

Three practical examples

Simulated control: a robot model summarizes recent camera frames, then evaluates candidate action sequences through latent rollouts rather than rendering every future frame.
Process forecasting: a sequence model combines remembered operating history with uncertainty about unobserved conditions when predicting future sensor behavior.
Agent research: a team compares policies trained using imagined experiences and checks whether their apparent success survives evaluation in the actual simulator rather than only inside the learned model.

Think of it this way

Think of a navigator keeping a running notebook while maintaining several plausible explanations for what lies beyond the next corner. The notebook summarizes the route so far; uncertain explanations can branch into different predicted futures. The analogy does not make those imagined futures observations or prove that the notebook contains every fact needed for a good decision.

Worked example

Consider an original scalar toy RSSM, chosen for arithmetic rather than realism. Let h_t = 0.5 h_(t-1) + z_(t-1) + a_(t-1). Its transition prior is z_t | h_t ~ Normal(h_t, 1). A simple observation model is o_t | z_t ~ Normal(z_t, 1), which is a valid special case that ignores h_t once z_t is known.
Starting from h_0 = 0, z_0 = 0, and action a_0 = 1 gives h_1 = 1 and prior z_1 ~ Normal(1, 1). If observation o_1 = 3 arrives, exact Gaussian conditioning for this toy model gives z_1 | o_1 ~ Normal(2, 1/2). The posterior mean averages the prior mean and observation because their variances are equal.
Without o_1, imagined rollouts must retain the prior distribution. Suppose one prior draw is z_1 = 0 and another is z_1 = 2. With next action a_1 = 0, their memories become h_2 = 0.5 and 2.5 respectively. The update is deterministic given its inputs, yet different sampled histories produce different memories.
Real neural RSSMs generally use approximate inference. The exact update here is a transparent illustration, not a claim that their learned encoders perform exact Bayes updates.

Illustrative case study

In a fictional project, Alder Automation trains a camera-based control agent in a factory simulator. The model predicts plausible images, but the policy repeatedly exploits a simulated shortcut that fails in the simulator's actual dynamics. The team discovers that long imagined rollouts drift into poorly supported states.
It shortens the planning horizon, expands evaluation across unfamiliar starting conditions, and compares predicted rewards with realized rewards after each executed action. It also checks observation-conditioned state estimates separately from open-loop forecasts. This case illustrates why a useful world model must support decisions accurately, rather than merely produce visually convincing reconstructions.

Common misconceptions and mistakes

An RSSM is not an ordinary recurrent network with a decorative random output. Its stochastic state participates in the dynamics and inference process. The transition prior and observation-conditioned posterior serve different purposes; using future observations in an imagined rollout would invalidate the comparison.
Latent variation does not automatically quantify uncertainty about model parameters or reveal unfamiliar environments reliably. Good reconstruction does not guarantee accurate control-relevant predictions. Another mistake is assuming that all RSSMs use Gaussian stochastic states or the same training objective: later variants may change these choices while retaining the broad deterministic-and-stochastic structure.

Business use

Consider an RSSM when partial observations, temporal memory, and multistep prediction are central to the application. Define which inputs and actions are available at each time and what the model is expected to predict. Separate reconstruction quality, open-loop prediction quality, and downstream decision performance in the evaluation.
Record rollout horizons and test for accumulating error. Compare predicted rewards and constraint-relevant events with actual outcomes. Maintain an independent evaluation environment where the policy cannot gain credit merely by exploiting weaknesses in its own model. Any physical deployment requires its own operating limits and validation beyond this model architecture.

Check your understanding

In the toy example, should a rollout made before seeing o_1 = 3 use Normal(1, 1) or Normal(2, 1/2) for z_1?

Answer: Normal(1, 1). The latter distribution uses the observation and belongs to the observation-conditioned update. Using it prematurely would introduce unavailable information.

Related terms

Recurrent neural network
Model-based reinforcement learning
State-space model
latent state
world model
variational inference
transition prior
filtering posterior
imagined rollout

Learn next

Implement the scalar example and compare trajectories sampled before and after observing o_1. Then study how a learned encoder approximates filtering and how a planner evaluates action sequences using predicted rewards.

Sources

Hafner and colleagues, Learning Latent Dynamics for Planning from Pixels, original PlaNet paper, Section 3 and equation4: https://proceedings.mlr.press/v97/hafner19a/hafner19a.pdf Hafner and colleagues, Dream to Control, original author abstract on learning behavior through latent imagination: https://arxiv.org/abs/1912.01603 PlaNet Section 3, equations 3–4, independently consulted for the stochastic/deterministic transition and inference structure.

Previous: Recurrent neural network Next: Recursive best-first search