Must a transition model be a matrix?

No; matrices are convenient for small finite systems.

Does it have to be learned?

No; it may be specified from physical knowledge or explicit rules.

Can a model include history?

Yes, directly or by expanding the state representation.

AI Glossary / Statistics and probability / Intermediate

Transition model

Definition

A transition model describes how a system moves from its current state to a next state, sometimes depending on an action. It may specify a deterministic next-state rule or a probability distribution over possible outcomes. In probabilistic AI, it provides the dynamics used for prediction, filtering, simulation, and planning.

What it means

Knowing what the world looks like now is not enough to predict what comes next. A robot needs an account of how motion commands change its position; an inventory planner needs an account of how demand and replenishment change stock.
The transition model supplies that account at a defined level of detail and time scale. It describes changes in the modeled state. An observation model instead describes what a sensor reports about that state.

How it works

Specify the state variables and the time interval. A daily model and a monthly model describe different transitions, even when they use identical state labels. The state should contain the information needed for the intended prediction.
For a first-order uncontrolled probabilistic model, write P(s-next|s). For a controlled system, write P(s-next|s,a), conditioning on the action. A first-order Markov assumption says that the current state, with the action when applicable, summarizes the relevant past for the next-state distribution.
In a finite state space, every conditional distribution must assign nonnegative probabilities that sum to one over all possible next states. A transition matrix is one representation. Large or continuous systems may use equations, factored graphical models, simulators, or learned conditional distributions.
A deterministic model is a special case in which the specified state and action determine the next state. Uncertainty may instead come from unknown inputs, coarse state descriptions, imperfect parameters, or inherent randomness in the chosen abstraction.
Combine the transition model with the current state distribution to predict forward. Sum over possible starting states rather than selecting the most likely starting state and discarding the others. Multi-step prediction repeats this operation under a specified sequence of actions or policy.
Distinguish time-homogeneous dynamics from a stationary distribution. Using the same transition rule each day does not require the population's state proportions to remain constant. Time-varying rules can explicitly depend on season, calendar time, or other conditions.
Estimate and validate the model at the level at which decisions will use it. Good one-step accuracy does not guarantee reliable long-horizon simulations, particularly when errors compound or the planned actions differ from those represented in training data.
A transition model alone does not choose an action. Planning also needs objectives, costs, rewards, and constraints. Historical conditional associations between actions and outcomes may be confounded, so a learned table should not automatically be interpreted as an intervention model.

Three practical examples

A warehouse simulator models how a dispatch action changes stock after uncertain demand.
A robot uses motion dynamics to predict location before incorporating a new sensor reading.
A maintenance planner compares equipment-state distributions under stipulated service and no-service actions.

Think of it this way

Think of a board game's movement rules. The board position is the state, the chosen move is an action, and the rules describe possible resulting positions. If a die is involved, the rules include probabilities rather than a single guaranteed destination.

Worked example

Consider an original one-step maintenance example with states ready and degraded. The current belief is 60% ready and 40% degraded. Without service, the chance of being ready next period is 0.80 from ready and 0.20 from degraded. The complementary probabilities lead to degraded.
The predicted ready probability is 0.60 x 0.80 + 0.40 x 0.20 = 0.56. Therefore the degraded probability is 0.44.
Under a stipulated service action, the ready probabilities are 0.95 from ready and 0.70 from degraded. The predicted ready probability becomes 0.60 x 0.95 + 0.40 x 0.70 = 0.85, leaving 0.15 degraded.
Suppose next-period degradation costs 100 units and service costs 20 units immediately, with no other costs or benefits. Expected cost is 44 without service and 20 + 15 = 35 with service, so service is preferred in this one-step toy decision. These are assumed dynamics and costs, not measured maintenance performance or a solution to a multi-period planning problem.

Illustrative case study

Fictional case: Westhaven Logistics creates a two-state model for conveyor availability. It initially fits transitions from routine operating records and uses them to estimate next-shift downtime. Before simulating a new maintenance policy, the team checks whether the historical records contain comparable serviced equipment.
Service had often been reserved for the worst machines, making naive comparisons misleading. The analysts document that limitation and use a controlled pilot to improve the action-conditioned model before relying on its counterfactual maintenance projections.

Common misconceptions and mistakes

Do not confuse a transition model with a transition probability, which is one conditional probability within such a model. A Markov state need not be a single variable. Reusing a transition matrix does not guarantee constant state proportions. A sensor reading is not necessarily the actual state. Finally, a plausible simulation can still be wrong if its dynamics or action effects are poorly specified.

Business use

Transition models support capacity planning, forecasting, maintenance simulation, and agent development. Record state definitions, time units, action semantics, data coverage, and uncertainty. Test whether recommendations change under reasonable alternative dynamics. A model is most useful when decision-makers can see which conclusions follow from data and which depend on assumptions.

Check your understanding

Why is the no-service ready probability 0.56 rather than 0.80?

Answer: The current state is uncertain, so both possible starting states contribute.

Does the service result prove real-world effectiveness?

Answer: No; the example assumes its action-conditioned probabilities.

Related terms

Learn next

Construct a two-state transition matrix and verify its row sums. Predict two steps ahead, then add actions and distinguish a fixed action sequence from a state-dependent policy.

Policy

Sources

Poole and Mackworth, Artificial Intelligence: Foundations of Computational Agents, Decision Processes: https://artint.info/3e/html/ArtInt3e.Ch12.S5.html Russell and Norvig, Probabilistic Reasoning over Time, university-hosted chapter draft: https://personal.utdallas.edu/~vibhav.gogate/ml/2020f/lectures/dbn-chapter.pdf

Previous: Transition kernel Next: Transition probability