Do rows or columns have to sum to one?

The chosen convention determines which; both are used in legitimate references, so always state it.

Can a transition probability be zero?

Yes. Zero represents a transition excluded by the model, although an unobserved transition is not necessarily impossible in reality.

Does every stochastic matrix converge to one steady pattern?

No. Reducibility can allow several stationary distributions, and periodicity can prevent convergence from a particular starting state.

AI Glossary / Mathematics and optimization / Intermediate

Stochastic matrix

Definition

A stochastic matrix is a matrix of nonnegative numbers whose rows or columns sum to one, depending on the convention. It represents probability-preserving transitions. In the row-stochastic convention used here, entry Pij is the probability of moving from state i to state j, and every row sums to one. Square stochastic matrices are central to finite-state Markov-chain models.

What it means

Each row is a probability distribution over the next state, conditional on the current state. The matrix itself can be fixed even though individual transitions are random. Stochastic therefore does not mean that the stored numbers must change randomly. A row such as [0.8, 0.2] says that an item currently in the first state has an 80% chance of remaining there and a 20% chance of moving to the second state during the modeled step. The meaning of that step must be specified.

How it works

Represent the current state distribution as a row vector p whose entries are nonnegative and sum to one. The next distribution is pP. Repeated multiplication gives pP², pP³ and later distributions when the transition rule is time homogeneous. A stationary distribution π satisfies πP = π. Finite stochastic matrices have stationary distributions, but uniqueness and convergence from arbitrary starting distributions require additional conditions. For a finite irreducible chain, the stationary distribution is unique; aperiodicity additionally ensures convergence of the state distribution. Under a column-stochastic convention, probability vectors and multiplication order are transposed accordingly.

Three practical examples

An equipment-state model can represent transitions between operating and unavailable states over a fixed observation interval, provided the Markov assumption is suitable.
A learning exercise can model a random walk on a graph by assigning transition probabilities to outgoing edges.
A customer-state analysis can estimate transitions between defined activity states and compare the model's multi-step forecasts with later observations, without treating the matrix as a causal explanation of behavior.

Think of it this way

Think of a sorting station that redistributes a complete batch among destinations according to fixed fractions. Every item must go somewhere, so the outgoing fractions from each origin total one. The analogy captures probability conservation. Individual outcomes need not equal those fractions exactly in a small batch, and a Markov model adds the assumption that the current state contains the information used for the next transition.

Worked example

Use P = [[0.8, 0.2], [0.3, 0.7]] for states A and B. Starting with p0 = [1, 0], one step gives p1 = [0.8, 0.2]. A second gives p2 = [0.8×0.8 + 0.2×0.3, 0.8×0.2 + 0.2×0.7] = [0.70, 0.30]. To find a stationary distribution, let π = [a, 1 − a]. The first component equation is a = 0.8a + 0.3(1 − a), so 0.5a = 0.3 and a = 0.6. Hence π = [0.6, 0.4], and direct multiplication returns the same vector. Starting with 1,000 independent items all in A would give an expected 700 in A after two steps, not a guarantee of exactly 700. For a contrasting matrix Q = [[0, 1], [1, 0]], [0.5, 0.5] is stationary, but starting from [1, 0] alternates forever between [1, 0] and [0, 1]. Existence of a stationary distribution therefore does not alone establish convergence.

Illustrative case study

In a fictional project, Cedar Service models equipment as available or unavailable using daily observations. Analysts estimate transition probabilities, check that each row sums to one and compare predicted state proportions with a later period. They discover that a maintenance-policy change alters the transition pattern, so they avoid presenting the old stationary distribution as a permanent property of the equipment. The case illustrates model maintenance and validation, not a measured reliability improvement.

Common misconceptions and mistakes

A matrix with row sums of one is not stochastic if some entries are negative. Conversely, a row-stochastic matrix need not have column sums of one; satisfying both makes it doubly stochastic. A stationary distribution is not the same as an absorbing state: the overall proportions can remain unchanged while individual items keep moving. A stochastic matrix is not necessarily symmetric, invertible or full rank. Finally, fitting one-step transitions does not prove that memorylessness or time homogeneity is appropriate for the observed process.

Business use

Use a stochastic matrix when states and transition intervals can be defined clearly. Document how probabilities were estimated, including treatment of rare states, missing observations and any smoothing. Check whether the observed history predicts transitions beyond what the current state captures. Compare forecasts over several horizons and revisit the matrix after operating conditions change. If transitions vary with time or context, consider an explicitly varying model rather than repeatedly applying one fixed matrix. Report uncertainty in estimated transition probabilities when data are limited.

Check your understanding

For the worked row-stochastic matrix, should a row distribution p be updated as Pp?

Answer: No. With p represented as a row vector, the update is pP. A column-vector representation would require the corresponding transposed convention.

Related terms

Markov Chain
Transition probability
Stationary distribution
irreducibility
aperiodicity
doubly stochastic matrix

Learn next

compute p3 and p4 for the worked matrix and compare them with [0.6, 0.4]. Repeat with the alternating matrix Q. Then examine the identity matrix, where every initial distribution is stationary, to distinguish stationarity, uniqueness and convergence.

Identity matrix

Sources

Sources: Cornell University, CS 4850 lecture notes on Markov chains, https://www.cs.cornell.edu/courses/cs4850/2022sp/lectures/handouts/markov.pdf ; MIT, Introduction to Stochastic Processes, Lecture 2: Stationary Distribution, https://ocw.mit.edu/courses/18-445-introduction-to-stochastic-processes-spring-2015/2edaa8542469ce692226813a8bd1ec88_MIT18_445S15_lecture2.pdf . The two-state calculations and fictional equipment study are original and consistently use row-stochastic matrices.

Previous: Stochastic hill climbing Next: Stochastic multiarmed bandit