Is a belief state the same as a history?

Not exactly. Under suitable model assumptions, it summarizes the history's relevant information for future decisions.

What is belief space?

It is the set of possible belief states. Even a finite physical-state model can have continuously many probability distributions.

Can actions change knowledge without changing the world?

A sensing action can primarily improve information, although real sensing may still consume time or resources.

AI Glossary / Graphs and symbolic AI / Intermediate

Belief state

Definition

A belief state represents an agent's information about the situation it may currently be in. In a nondeterministic setting it can be a set of possible physical states; in a probabilistic setting it is a probability distribution over those states.

What it means

An agent often cannot observe the world directly or completely. A robot may know that it is in one of two similar corridors. A software assistant may know that a request is either waiting or completed because the latest status response was lost.

A belief state preserves those alternatives rather than pretending that the most likely state is certainly the true one. A probability distribution adds relative likelihoods; a set representation records which states remain possible without assigning probabilities.

The belief state is information held by the agent, not the physical state itself. Two agents observing the same world can have different belief states because they have different evidence. The term does not imply consciousness, and it should not be confused with a Dempster - Shafer belief function.

How it works

The agent starts with an initial representation of uncertainty. After an action, it predicts which states could result using a transition model. It then incorporates the new observation using a model of what observations are possible or likely in each state.

For a probabilistic model, the prediction step redistributes probability through the possible transitions. The observation step weights those predicted states by how well they explain the observation and normalizes the result. In a set-based version, the agent retains states consistent with the action and observation under the model.

A policy can choose actions using this uncertainty representation. Sometimes an action is valuable because it gathers information rather than immediately completing the task. Maintaining the full belief can be expensive, so practical systems may use approximations. Their quality depends on both the approximation and the correctness of the transition and observation models.

Three practical examples

A mobile robot maintains several possible locations until a distinctive landmark helps distinguish them.
An interactive assistant keeps more than one interpretation of an ambiguous request and asks a clarifying question when the alternatives imply different actions.
A monitoring system tracks probabilities over equipment modes when its sensors provide noisy or incomplete readings.

Think of it this way

Think of marking several plausible locations on a map after losing sight of a moving object. New movement information shifts the possibilities, and a new observation narrows or reweights them.

The analogy has limits. A probability attached to a location is a model-based assessment, not a physical fraction of the object occupying that place. A confidently concentrated belief can still be wrong if the map or observation model is wrong.

Worked example

Predicting and then updating

An invented robot has two possible locations, Left and Right. Its current belief is 0.6 for Left and 0.4 for Right. After a particular action, it stays in its current location with probability 0.8 and switches with probability 0.2.

The predicted probability of Left is 0.6 × 0.8 + 0.4 × 0.2 = 0.56. Right therefore has probability 0.44.

The robot then observes a red marker. The observation model gives probability 0.9 for this observation in Left and 0.2 in Right. The unnormalized updated weights are 0.56 × 0.9 = 0.504 and 0.44 × 0.2 = 0.088. Their sum is 0.592.

The updated belief in Left is 0.504/0.592, approximately 0.851, with about 0.149 remaining for Right. The observation makes Left more likely but does not establish certainty. Both the motion and sensor assumptions matter to this result.

Illustrative case study

Fictional case: Elm Robotics operates a demonstration cart in two visually similar aisles. Its first planner replaces its location distribution with whichever aisle has the highest probability. A slightly favored but incorrect guess can then lead to a poor route choice.

The team retains the alternatives and introduces a sensing action that views a distinctive marker before choosing an aisle-specific maneuver. It evaluates the action's time cost against the consequences of taking the wrong route. It also tests misleading observations to avoid assuming that sensing always resolves uncertainty.

The lesson is that uncertainty can be relevant to action selection even when one state is more likely than all others. Preserving alternatives enables the planner to consider the value of additional evidence.

Common misconceptions and mistakes

A belief state is the actual world state. It represents the agent's information about that state.
The highest-probability state can always replace the distribution. Decisions may depend on less likely but consequential possibilities.
An observation removes uncertainty completely. Noisy observations often only change relative probabilities.
More confidence proves a better model. A misspecified model can become confidently wrong.
Every belief state must be probabilistic. Set-based representations are useful when only possibility is modeled.

Business use

Belief states help explain why agents need memory and uncertainty management when observations are incomplete. They support decisions about whether to act, wait, inspect or ask for clarification.

For business applications, define the hidden states and test whether the observations actually distinguish them. Evaluate how the policy behaves when evidence is missing or contradictory, not just when the most likely state is correct. If the system approximates its belief, document what information is discarded and whether that affects consequential decisions. The representation is useful only insofar as it supports the intended task.

Check your understanding

An agent believes Location A has probability 0.7 and Location B has probability 0.3. A costly error occurs only if it acts as though A is certain while actually in B. Should it automatically discard B?

Answer: No. The remaining probability and the consequence of error can justify a different action or further observation.

Related terms

Transition model
Uncertainty
Partial observability
POMDP
state estimation
Bayesian update
observation model
value of information

Learn next

Create a two-state example with an initial probability, one uncertain action and one noisy observation. Perform prediction before the observation update and check that the final probabilities sum to one. Then compare a policy using only the most likely state with one that can choose an information-gathering action.

Sources

AIMA glossary, belief state, distinguishes set-based and probabilistic representations: https://github.com/aimacode/aima-glossary/blob/master/glossary.md?plain=1

Anthony Cassandra, Basics of Solving POMDPs, explains belief distributions, belief space and action-observation updates: https://www.pomdp.org/tutorial/pomdp-solving.html

The robot calculation and fictional aisle example are original illustrations.

Previous: Belief revision Next: Bellman equation