Is a transition matrix a transition kernel?

Yes, for a finite state space it is a convenient tabular representation of one.

Is the observation model the same thing?

No. A transition kernel describes state evolution; an observation model describes what is measured given the state.

Does a kernel guarantee convergence to a unique stationary distribution?

No. Such conclusions require additional properties of the process; some kernels have multiple stationary distributions or problematic long-run behavior.

AI Glossary / Graphs and symbolic AI / Advanced

Transition kernel

Definition

A transition kernel specifies a probability distribution over possible next states for each current state. It generalizes a transition matrix so that the same idea can describe finite, countable or continuous state spaces.

What it means

For a current state x and a set of possible next states A, write K(x,A) for the probability that the next state lies in A. The kernel is therefore a rule for conditional uncertainty, not a single predicted next state.

With finitely many states, each row of a transition matrix supplies the probabilities for one current state. With continuous states, sets and intervals matter because an individual exact value can have probability zero even when values near it are likely.

This is the probability and Markov-process meaning of kernel. It differs from a similarity kernel in a support vector machine or an operating-system kernel. A valid transition kernel also does not, by itself, guarantee that the chosen state captures every relevant dependency in the real process.

How it works

For each fixed current state, the kernel must assign a proper probability distribution to next-state events: probabilities are nonnegative, the whole state space has probability one and disjoint events combine additively. In general mathematical settings, an additional measurability condition ensures that these probabilities vary with the current state in a way that supports integration.

A Markov model uses the current state as the information needed for the next transition. A time-homogeneous model uses the same kernel at each step; a time-varying model can use different kernels. The Markov property alone does not imply time homogeneity.

To predict from uncertainty about the current state, combine each possible state's outgoing distribution with its current probability. Multiple-step prediction composes the kernels. For finite states this becomes matrix multiplication; for continuous states the corresponding operation uses integration over intermediate states.

Three practical examples

A queue simulation assigns next-step probabilities to different queue-length states, conditional on the current state.
A reinforcement-learning environment uses a state-and-action transition rule to describe uncertain outcomes of an action.
A Markov chain Monte Carlo sampler uses a designed transition kernel to move between candidate states while targeting a specified distribution.

Think of it this way

Think of a travel desk that gives a different probability-weighted list of tomorrow's destinations depending on where you are today. The complete collection of those lists is the transition rule.

The analogy illustrates conditional distributions, not intentional choice. A kernel need not describe a traveler, and a distribution over destinations is not a guarantee of which destination occurs. It also says nothing by itself about what a sensor will report after arrival.

Worked example

Predicting across two states

Consider an invented system with states Quiet and Busy. From Quiet, the next state is Quiet with probability 0.8 and Busy with probability 0.2. From Busy, those probabilities are 0.3 and 0.7. Each row sums to one.

Suppose the current distribution is 0.6 Quiet and 0.4 Busy. The next probability of Quiet is 0.6 × 0.8 + 0.4 × 0.3 = 0.60. The next probability of Busy is 0.6 × 0.2 + 0.4 × 0.7 = 0.40. This particular distribution is unchanged by one application of the kernel, so it is stationary.

Starting instead with Quiet for certain gives 0.8 Quiet after one step. After two steps, the probability of Quiet is 0.8 × 0.8 + 0.2 × 0.3 = 0.70. The calculation sums both possible intermediate states. It does not multiply 0.8 by itself and ignore paths through Busy.

Illustrative case study

Fictional case: Alder Simulation models workload using Quiet and Busy states. Its first implementation places current-state probabilities in columns but applies a formula written for row vectors. The resulting predictions do not match the team's hand calculations.

The team documents the matrix convention, checks every outgoing distribution and adds simple one-step and two-step examples. It then notices that morning and evening transition patterns differ, so a single unchanged kernel may not represent the full day adequately.

The model is revised to state its time scope explicitly. Reviewers separately inspect the observation process because workload measurements can be noisy. The case illustrates representation and assumption checks, not evidence that two states are sufficient for a real service.

Common misconceptions and mistakes

A transition kernel returns one fixed next state. It usually returns a distribution; deterministic transitions are a special case.
Every kernel has a density. Discrete, mixed or other distributions need not have a density relative to the chosen reference measure.
A stationary distribution means the system never changes state. It means the overall state distribution remains unchanged under the transition rule.

Business use

Transition kernels make the uncertainty assumptions in simulations, planners and probabilistic models explicit. They help teams examine what outcomes a system believes are possible from each condition.

Specify the state definition, time-step length and any action conditioning. Validate probability totals and matrix orientation, then compare predicted transitions with representative observations. If transition behavior changes with context, either enrich the state or model that change explicitly. A mathematically valid kernel can still be a poor description of the operational process.

Check your understanding

A finite-state transition row contains probabilities 0.4, 0.4 and 0.4 for three exhaustive, mutually exclusive next states. Is that a valid row?

Answer: No. The sum is 1.2 rather than one. The specified probabilities need correction before the row can represent a next-state distribution.

Related terms

Learn next

Read Markov chain and stationary distribution. Recalculate the worked example from a starting distribution of 0.2 Quiet and 0.8 Busy, then distinguish the one-step prediction from the transition probabilities themselves.

Sources

Jim Pitman, Berkeley probability-theory lecture notes, Markov kernels and distribution propagation: https://www.stat.berkeley.edu/~pitman/s205s03/lecture1.pdf

Carnegie Mellon statistical-computing course, simulation methods and transition kernels: https://36-750.github.io/methods/simulation/

Relevant definitions and propagation discussions were read. The two-state matrix and fictional case are original illustrations, with their arithmetic checked directly. No fitted transition probabilities or empirical convergence results are claimed.

Previous: Transition constraint Next: Transition model