Does first-order mean only one state variable?

No. A state can contain several variables. First-order refers to dependence on the previous stage, not the number of components.

Can transitions change with time?

Yes. A time-inhomogeneous process can still satisfy the first-order conditional-dependence property.

Can observations from a hidden Markov model have longer memory?

Yes. Uncertainty about the hidden state can make earlier observations informative about future observations.

AI Glossary / Graphs and symbolic AI / Intermediate

First-order Markov process

Definition

A first-order Markov process is a sequence of random states in which the next state's conditional distribution depends on the current state rather than the full earlier history. Once the current state is known, previous states provide no additional information for predicting the next state under the model. This is the first-order Markov property.

What it means

The property concerns conditional dependence, not an absence of memory in ordinary language. The current state may itself summarize information from the past. Choosing a useful state description is therefore central: a coarse label may fail the Markov assumption even when a richer state works well.

First-order also does not mean linear, deterministic or first in a sequence. A process can have complicated or random transitions while satisfying the property. This entry uses a discrete-time description, where states occur at successive stages. Those stages must be defined consistently for the modeled application.

How it works

For states X0, X1 and so forth, the property says that the probability of X at the next stage, given the entire state history, equals its probability given only the current state. For a finite state set, transition probabilities can be recorded in a matrix whose rows sum to one.

An initial distribution and transition rules define probabilities of state sequences. If the transition rule is the same at each stage, the process is time-homogeneous. That is an additional assumption: first-order processes can also have time-dependent transitions.

Multi-step predictions sum over intermediate states rather than pretending each step is independent. In a hidden Markov model, the hidden state can be first-order Markov while the observed sequence generally is not. A model's claimed state property should therefore not be transferred automatically to raw sensor readings or other observations.

Three practical examples

1. A simple operating-mode model predicts whether a machine is idle or busy at the next observation from its current mode. If elapsed time in the mode strongly affects transitions, those two labels alone may be inadequate.

2. A synthetic weather exercise models the next day's category from today's category. This is a teaching assumption to evaluate, not a claim that actual weather ignores earlier conditions.

3. A robot state model includes position and velocity rather than position alone. The richer description can better support a transition model because current velocity influences what happens next.

Think of it this way

Think of a relay handover in which the current runner carries all information needed for the next leg. Earlier handovers need not be consulted if the current handover is sufficient. The analogy concerns an information summary. It does not mean every real process has such a simple summary, or that history disappears from the way the current state was reached.

Worked example

Consider two states, Idle and Busy. At each stage, an idle system remains Idle with probability 0.8 and becomes Busy with probability 0.2. A busy system becomes Idle with probability 0.3 and remains Busy with probability 0.7. Assume these transition probabilities stay fixed over time.

Starting in Idle, the probability of being Busy after two steps is the sum of two paths. Idle to Idle to Busy contributes 0.8 × 0.2 = 0.16. Idle to Busy to Busy contributes 0.2 × 0.7 = 0.14. The total is 0.30. The complementary probability of Idle is 0.70.

If the system is known to be Busy at the current stage, the next-stage Busy probability is 0.7 whether it was Idle or Busy one stage earlier. That equality expresses the modeled first-order property. If real observations show different probabilities depending on how long the system has been busy, the proposed two-state model may need revision. These invented probabilities demonstrate calculation, not a fitted service model.

Illustrative case study

Fictional case study: Fir Support models request status using only Open and Closed. Its analysts notice that two open requests have different closing patterns depending on whether a response is already awaiting review. Earlier events matter because the current label hides that distinction.

The team considers a richer state including the review stage, then checks whether the revised transition model predicts held-out sequences more adequately. It also tests whether probabilities change across operating periods. The case illustrates state-design and stationarity questions, without claiming that the new labels establish an exact Markov property for the real workflow.

Common misconceptions and mistakes

First-order does not mean successive states are independent. The next state can depend strongly on the current one. It also does not imply that transition probabilities are constant over time; time-homogeneity is separate.

The word memoryless can be misleading if the state already stores historical information. A higher-order process can sometimes be represented as first-order by enlarging the state to include recent history. Finally, accepting the assumption for a hidden state does not justify it for observed outputs. Hidden uncertainty can preserve dependence across many observations even when latent transitions are first-order.

Business use

The Markov assumption can make sequential models manageable by replacing an expanding history with a current-state description. For business use, define states according to the information needed to predict relevant transitions. Check whether duration, recent events, workload or time of day creates systematic differences that the proposed state omits.

Validate transition estimates on appropriate held-out periods and record how the sampling interval affects them. A model can still be useful as an approximation, but communicate that scope. Avoid treating a compact transition matrix as evidence that the underlying process truly forgets its past; the adequacy of the summary is an empirical modeling question.

Check your understanding

Why is the two-step Busy probability 0.30 rather than simply 0.20?

Answer: The model must account for both possible intermediate states. Staying Idle and then becoming Busy contributes 0.16; becoming Busy and staying Busy contributes 0.14.

Related terms

Markov property
Markov Chain
Hidden Markov model
Conditional independence
Stochastic process
transition matrix
state representation
time-homogeneity

Learn next

Calculate two-step predictions in a small transition matrix by enumerating intermediate states. Then compare a coarse state label with one that includes relevant recent history, checking what each representation assumes away.

Sources

Primary sources: Poole and Mackworth, temporal reasoning material: https://artint.info/2e/html2e/ArtInt2e.Ch8.S5.SS1.html ; Ghahramani and Jordan, discussion of first-order hidden-state transitions: https://mlg.eng.cam.ac.uk/pub/pdf/GhaJor97a.pdf . The two-state arithmetic and fictional case are original teaching examples.

Previous: First-order logic Next: First-order reliability method