Can a transition probability equal one?

Yes, for a certain transition under the model.

Does a stable transition matrix imply stable population proportions?

No; the starting distribution can evolve toward or away from a particular composition.

AI Glossary / Statistics and probability / Intermediate

Transition probability

Definition

A transition probability is the conditional probability that a system moves to a specified state over a specified step or interval, given its starting state and any other stated conditions. In a Markov decision process, those conditions include the action. It is one numerical component of a model of how states change.

What it means

A probability such as 0.30 is incomplete without its conditioning information. It might mean a 30% chance that an active account becomes inactive next month, given that it is active now and follows a particular service policy.
That is different from the fraction of all accounts currently inactive, the probability that an inactive account used to be active, or the probability of ever becoming inactive. States, direction, and time horizon determine what the number means.

How it works

For a discrete-time Markov chain, write p-ij = P(X-next = j|X-current = i). The first-order Markov assumption means that conditioning on the current state makes earlier history unnecessary for this next-state distribution.
Arrange probabilities in a transition matrix when the state space is finite. Under the row convention, row i contains probabilities of all possible next states given starting state i. Each entry lies between zero and one, and each row sums to one.
A diagonal entry represents remaining in the same recorded state at the next observation. It does not necessarily mean that no activity occurred during the interval; intermediate movements may be unobserved.
Time-homogeneous models use the same probabilities at each step. In a changing environment, write probabilities that depend on time or relevant context. A monthly probability cannot be substituted for a daily probability without an appropriate time model.
For a time-homogeneous Markov chain, two-step probabilities sum over intermediate states: p-ij over two steps equals the sum over k of p-ik times p-kj. In matrix notation, the two-step transition matrix is P squared, using matrix multiplication rather than squaring each entry.
Distinguish endpoint probabilities from hitting probabilities. Being inactive two months from now is different from having been inactive at least once during those two months, because a system may leave and then return to its starting state.
Estimate probabilities using suitable data and assumptions. Observed transition counts divided by the number of eligible departures from a starting state provide a simple estimate in a fully observed, homogeneous setting. Sparse counts, censored observation, hidden states, changing policies, or repeated dependence require more careful treatment.
In controlled systems, condition on action as well as state. A policy specifies which actions are selected; combining policy probabilities with action-conditioned transitions produces the state dynamics under that policy. The transition probability itself is not a recommendation about which action is best.

Three practical examples

A subscription model estimates the next-month probability of becoming inactive conditional on current activity status.
A robot model assigns probabilities to intended movement, sideways movement, and remaining in place after a command.
A maintenance simulator records the chance of moving from an operating state to a degraded state within one shift.

Think of it this way

Think of a labeled junction on a map. Each outgoing route has a probability, and the complete set of outgoing possibilities sums to one. Reaching a destination after two junctions requires accounting for every possible intermediate route, not merely doubling one label.

Worked example

Consider an original two-state monthly model with states A and B. From A, the next state is A with probability 0.70 and B with probability 0.30. From B, it is A with probability 0.20 and B with probability 0.80. Both rows sum to one.
Starting in A, the probability of being in B after two months is P(A to A to B) + P(A to B to B) = 0.70 x 0.30 + 0.30 x 0.80 = 0.45. The complementary probability of ending in A is 0.70 x 0.70 + 0.30 x 0.20 = 0.55.
However, the probability of visiting B at least once during the two months is 0.30 + 0.70 x 0.30 = 0.51. The difference, 0.06, is the probability of the path A to B to A.
These calculations assume a first-order, time-homogeneous chain with the stated states observed at monthly boundaries. They illustrate why an endpoint probability and a first-visit question require different calculations.

Illustrative case study

Fictional case: Meadow Subscriptions presents a two-month inactivity forecast to its customer-support team. A first draft labels the endpoint probability as the chance of any inactivity during the period. An analyst notices that some accounts reactivate before the second month ends.
The team reports endpoint inactivity and any-period inactivity separately, preserving the transition model's time units. It also checks whether recent pricing changes make the historical monthly probabilities unsuitable for the upcoming period.

Common misconceptions and mistakes

Do not reverse a conditional probability without additional information and Bayes' rule. Do not add one-step probabilities to obtain a multi-step endpoint probability. An estimated zero from a small sample does not prove impossibility. Row sums, rather than column sums, equal one under the convention used here. Continuous-time transition rates have units of inverse time and are not themselves ordinary transition probabilities.

Business use

Transition probabilities support retention forecasts, operations simulations, reliability analysis, and planning models. Label dashboards with the starting state, destination state, horizon, and policy context. Report uncertainty when estimates are based on limited transitions. Review whether state definitions and observation frequency hide movements that matter to the business question.

Check your understanding

In the example, why is the two-month A-to-B probability 0.45 rather than 0.60?

Answer: It sums the probabilities of two relevant paths.

Which additional path explains the difference between endpoint and any-visit probabilities?

Answer: A to B to A, with probability 0.06.

Related terms

Transition model
Markov Chain
Stationary distribution
Conditional probability
Transition matrix
Chapman-Kolmogorov equation
hitting probability

Learn next

Multiply a small transition matrix by itself and interpret each resulting entry. Then calculate a first-visit probability and compare its event definition with an endpoint forecast.

Sources

Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Markov-chain introduction: https://www.probabilitycourse.com/chapter11/11_2_1_introduction.php Same textbook, probability distributions and multi-step transitions: https://www.probabilitycourse.com/chapter11/11_2_3_probability_distributions.php Poole and Mackworth, Decision Processes: https://artint.info/3e/html/ArtInt3e.Ch12.S5.html

Previous: Transition model Next: Transitive Dependency