Does a PSR eliminate uncertainty?

No. Probabilities explicitly represent uncertainty about future observations.

Is it the same as a POMDP belief state?

No. Both can support prediction in partially observed systems, but they represent information differently: one through hidden-state beliefs, the other through observable-test predictions.

Must tests involve only one step?

No. Multi-step action-observation tests are central to the representational idea.

AI Glossary / AI and machine learning / Advanced

Predictive state representation

Definition

A predictive state representation, or PSR, represents the state of a controlled dynamical system using predictions about future observations under specified action sequences. Instead of naming hidden states, it describes what the system is expected to reveal when particular tests are performed.

What it means

A system's state should contain the information needed to predict what happens next. In a hidden-state model, that information may be expressed as beliefs about unobserved conditions. A PSR instead uses a set of observable, action-conditional predictions.

The predictions depend on the history of actions and observations. They are not merely guesses about the next measurement: carefully chosen tests can involve several future steps. The approach is a representation of uncertainty and dynamics, not a claim that a system can directly observe every relevant condition.

How it works

The model defines tests involving actions and possible observations. A vector records the predicted probabilities of selected core tests given the current history. After an action and observation, an update rule changes that vector.

The representation must support predictions for the other tests of interest. Learning a suitable representation and accurate updates is a substantive modeling problem. Arbitrarily collecting a few forecasts does not automatically create a sufficient PSR. Data coverage, valid probabilities and the distinction between chosen actions and passive observations all matter when applying the idea.

Three practical examples

1. A robotics research model can represent navigation state through predictions of sensor outcomes after specified movements, without assigning names to every hidden environmental configuration.

2. A controlled-machine simulation can compare hidden-state and predictive representations for forecasting the consequences of permitted actions.

3. A teaching exercise can construct two histories with the same immediate observation but different future predictions, showing why the latest observation alone may be an inadequate state description.

Think of it this way

Imagine describing a sealed device by what it is likely to do when you press certain sequences of buttons. Those test predictions can be useful even if you do not know the exact internal configuration.

The analogy does not remove the need for a model. A small collection of tests may fail to distinguish devices that behave differently under other sequences. Choosing informative tests is part of the challenge.

Worked example

Suppose a toy device has two observable outcomes, green and red, after each permitted button press. After one history, the model predicts a green outcome after action A with probability 0.8. After a different history, it predicts 0.3, even though the most recent observed color is green in both cases.

The identical latest observation therefore does not justify treating the two histories as the same predictive state. A second test, such as observing green and then red after a specified two-action sequence, could provide additional information.

These numbers illustrate action-conditional predictions only. They do not establish a complete PSR or its update equations. A valid representation must support the required predictions consistently across histories and tests.

Illustrative case study

Fictional robotics group TurnPath models a simulated vehicle that sometimes encounters visually similar corridors. A model using only the latest image confuses situations with different movement outcomes. The researchers investigate a predictive representation based on short action-observation tests.

They discover that their training trajectories rarely include one important turning sequence. The corresponding predictions are unreliable despite good results on familiar routes. The team expands the experiment's action coverage and compares prediction quality on held-out trajectories. The case illustrates the importance of informative experience, not a guaranteed advantage over every hidden-state model.

Common misconceptions and mistakes

A PSR is not simply a list of recent observations. Its state components are predictions, conditioned on history, about future tests. Nor is it equivalent to a general dashboard of forecast values without a consistent dynamical update model.

Avoid interpreting action-conditional predictions as ordinary correlations from uncontrolled records. The data-collection process affects what can be learned about actions. Another mistake is assuming theoretical representational compactness guarantees easy learning or superior practical performance. The choice of tests, numerical estimation and available experience can make the problem difficult.

Business use

PSRs are mainly relevant to research involving partially observed, controlled systems. A technical proposal should explain why predictive state is preferable for its particular task and how the chosen tests connect to observable operational behavior.

Ask how predictions are checked after unfamiliar action sequences, how the representation is updated and whether the data covers the intended control policy. Compare with simpler state summaries and established hidden-state approaches. A representation's conceptual elegance is useful only if it supports dependable prediction or control under the conditions that matter.

Check your understanding

Two histories end with the same sensor reading. Must they have the same predictive state?

Answer: No. Earlier actions and observations can imply different future outcomes. A useful state representation preserves the distinctions needed for the predictions and decisions under consideration.

Related terms

Belief state
Dynamical system
Reinforcement learning
Hidden Markov model
Partially observable Markov decision process
system identification
action-conditional prediction
state representation

Learn next

Construct a small controlled system with ambiguous observations. Compare representing its state by recent history, hidden-state probabilities and predictions of future tests. Identify which information each representation retains and how it updates after an observation.

Sources

Littman, Sutton and Singh, Predictive Representations of State: https://papers.nips.cc/paper_files/paper/2001/hash/1e4d36177d71bbb3558e43af9577d70e-Abstract.html. Singh, James and Rudary, Predictive State Representations: A New Theory for Modeling Dynamical Systems: https://arxiv.org/abs/1207.4167. The 2004 author paper was consulted on controlled tests, history-conditioned predictions, core tests and state updates. The device example and TurnPath case are original illustrations.

Previous: Predictive parity Next: Predictive text