Does stationarity mean no movement?

No. Individual states can change while the distribution remains constant.

Is the stationary distribution always uniform?

No. Transition structure determines its values.

Can a chain have a stationary distribution without converging to it from every start?

Yes; periodic chains provide a simple example.

AI Glossary / Statistics and probability / Intermediate

Stationary distribution

Definition

A stationary distribution is a probability distribution over a system's states that remains unchanged after the system's transition rule is applied. For a finite Markov chain with row-stochastic transition matrix P, a row probability vector pi is stationary when pi P = pi. Its entries are nonnegative and sum to one. This is a property of the distribution, not a claim that individual states stop changing or that every initial distribution converges to it.

What it means

Imagine a large population of systems moving among states according to the same probabilities. At stationarity, the population proportions remain constant even though individual systems continue to move. Probability flowing out of a state is balanced by probability flowing into it overall.

A stationary distribution can describe long-run occupancy under appropriate conditions, but existence, uniqueness, and convergence are different questions. Every finite Markov chain has at least one stationary distribution. A finite irreducible chain has a unique one. Aperiodicity additionally ensures ordinary convergence to it from any initial distribution in that finite irreducible setting.

How it works

Specify the states and transition probabilities, checking that each row of P sums to one. Solve the balance equations pi P = pi together with the normalization equation sum(pi) = 1. One balance equation is redundant, so numerical methods must account for that dependence.

Then examine the chain's structure. Separate closed classes can produce multiple stationary distributions. Periodicity can prevent successive state distributions from converging even when a unique stationary distribution exists. For infinite state spaces, additional recurrence conditions matter; the finite-state existence result cannot be extended automatically to every chain.

If the process starts in a stationary distribution and uses a time-homogeneous Markov transition rule, its marginal distribution stays the same at every step. Estimating long-run proportions from a single trajectory requires appropriate ergodic behavior, not merely solving a matrix equation. In applications, also assess whether the estimated transition rule remains credible over the period of interest.

Three practical examples

1. An operations team models equipment as available or unavailable and studies long-run occupancy under a simplified repair-and-failure model.

2. A search-ranking method uses a random-walk construction on linked pages and interprets its stationary probabilities as ranking scores under that model.

3. A simulation analyst designs a Markov chain with a desired invariant distribution, then checks convergence and mixing before using its samples for estimation.

Think of it this way

Think of water moving between connected tanks. The levels can remain steady while water continues flowing through the pipes. A stationary distribution resembles the steady proportions, while individual state transitions resemble the continuing flows. Steady levels do not imply that an arbitrary starting arrangement reaches them immediately.

Worked example

Consider an original two-state model with states A and B. From A, the next state is A with probability 0.8 and B with probability 0.2. From B, it is A with probability 0.3 and B with probability 0.7. Thus P has rows [0.8, 0.2] and [0.3, 0.7].

Let the stationary probability of A be a. Balance requires a = 0.8a + 0.3(1 - a). Rearranging gives 0.5a = 0.3, so a = 0.6 and pi = [0.6, 0.4]. Verification gives [0.6 times 0.8 + 0.4 times 0.3, 0.6 times 0.2 + 0.4 times 0.7] = [0.6, 0.4].

Starting entirely in A gives distributions [1, 0], [0.8, 0.2], [0.7, 0.3], and [0.65, 0.35]. In this example the probability of A after k steps is 0.6 + 0.4(0.5 to the power k), which approaches 0.6.

For contrast, a chain that switches between A and B with certainty has stationary distribution [0.5, 0.5], but a trajectory started in A alternates forever. Its successive distributions do not converge. The contrast separates the stationary equation from the additional convergence property.

Illustrative case study

In a fictional service operation, analysts model a machine as working or under repair. They calculate a stationary working proportion and initially propose using it as next week's availability forecast.

The manager explains that all machines are currently undergoing a planned overhaul and repair capacity will change during the week. The analysts therefore use the current state distribution and time-specific transition assumptions for the short-term forecast. They retain the stationary calculation as a description of a simplified stable operating regime, not a substitute for modeling the present situation.

Common misconceptions and mistakes

Do not confuse time-homogeneous transition probabilities with a stationary state distribution. A chain can use the same transition matrix at every step while its state probabilities evolve from a nonstationary start.

Stationarity does not imply uniform probabilities, independence between successive observations, or detailed balance. Detailed balance is a stronger condition associated with reversibility. A stationary distribution is also not necessarily unique. In Markov chain Monte Carlo, having the correct invariant distribution does not by itself demonstrate that a finite run has explored it adequately. Slow mixing can leave substantial simulation error.

Business use

Stationary distributions support capacity planning, reliability modeling, network analysis, and simulation. Their business usefulness depends on whether a stable transition model is a reasonable approximation and whether the time horizon is long enough for the intended interpretation. Report the states, transition estimates, structural assumptions, and sensitivity to changes. Avoid presenting long-run proportions as immediate forecasts for a system with unusual initial conditions.

Check your understanding

The transition matrix is the identity matrix. Is there only one stationary distribution?

Answer: No. Every probability distribution is stationary because the transition rule leaves every state unchanged. This illustrates why existence does not imply uniqueness.

Related terms

Markov Chain
Detailed balance
Mixing time
transition matrix
invariant distribution
irreducibility
periodicity
ergodicity

Learn next

Learn transition-matrix multiplication, then distinguish stationary equations from convergence theorems. Study periodic and reducible examples before applying long-run interpretations.

Sources

Poole and Mackworth, Artificial Intelligence, Markov chains lecture: https://artint.info/2e/slides/ch08/lect5.pdf

The authors' primary teaching material was read. Existence and convergence statements here are explicitly qualified for finite chains. The two-state calculations and fictional case are original.

Previous: Stationarity Next: Stationary kernel