Can a terminal state still have a representation of future transitions?

Yes. Some mathematical models use an absorbing self-loop with zero further reward to represent the ended episode.

Is every goal state terminal?

Only if the task says reaching that goal ends the episode. Some tasks continue after intermediate goals.

Does a timeout always mean truncation?

No. A deadline can be a genuine task ending. Its role must be defined explicitly.

AI Glossary / Graphs and symbolic AI / Beginner

Terminal states

Definition

Terminal states are states at which a modeled game, task, or episode has ended according to its rules. No further ordinary continuation of that episode is required, and the final outcome can be evaluated. A terminal state may represent success, failure, or a draw. It is different from a state where a search algorithm stops because it has exhausted its own time or depth budget.

What it means

In a game tree, terminal states form genuine endings of play. An outcome function can assign their values directly under the game rules, while nonterminal positions may require further search or estimation.

In an episodic learning environment, termination similarly marks the end of the task's return. A computer may implement the ending by stopping interaction or by using an absorbing representation with no further reward. The representation convention should be explicit.

How it works

Define the conditions that end the task, such as a win, a completed objective, or a specified failure. Include the information needed to recognize those conditions in the state. Some endings depend on history, a counter, or a rule beyond the visible arrangement.

When the system reaches a terminal state, determine its outcome and stop ordinary successor expansion. In minimax-style game search, terminal utility anchors values propagated back through earlier decisions. A depth-limited leaf that is not terminal instead needs an estimate.

For episodic reinforcement learning, a true terminal transition has no future reward beyond the episode, so its bootstrapped continuation value is zero. External truncation can require different treatment. If a finite time horizon is genuinely part of the task, the remaining time may need to be represented so that otherwise identical observations do not conceal different decision situations.

Three practical examples

1. Board games: a state satisfying the defined victory or draw rules ends play, even if unused board spaces remain.

2. Simulated delivery: a task ends when the parcel is delivered or when a specified unrecoverable failure occurs. These are different terminal outcomes rather than one generic success flag.

3. Learning episodes: an environment reports that its task has ended, while separately identifying a run interrupted by an external step limit. The learner uses that distinction when calculating targets.

Think of it this way

Think of reaching the final page of a particular story. The story has ended even if the surrounding world could continue. Closing the book halfway through because a timer rings does not turn that page into the story's actual ending.

The analogy is limited because some tasks are intentionally defined to end at a fixed time. In that case, the deadline belongs to the task itself. The distinction concerns the modeled rules, not whether a clock is involved.

Worked example

Consider a two-player game with a pile of three tokens. Players alternate removing one or two tokens, never more than remain. The player taking the last token wins. Represent a state as (n, player-to-move), where n is the number of tokens remaining. States with n=0 are terminal.

Player A starts. If A takes one, the state becomes (2,B). This is nonterminal because B can take one or two. B can take both and reach (0,A), a terminal state in which B won. If A initially takes two, the state becomes (1,B); B takes the final token and again wins.

Thus both initial choices allow B to force a win under these rules. The terminal state's player-to-move field names the player who would move next if play continued, not the winner. Interpreting that field as the winner would reverse the outcome.

Now stop search after A's first move. The states (2,B) and (1,B) are leaves of that truncated search tree, but neither is terminal in the game. Their values must be estimated or derived from further reasoning. The search boundary and the game's ending are different concepts, even when both appear as leaves in a diagram.

Illustrative case study

Fictional case: EpisodeWorks tests a learning environment for a small navigation task. Its logs use one flag for both reaching the destination and stopping after a fixed external evaluation budget. The learning code treats every flagged transition as having no continuation value.

The team separates task termination from external truncation and records the reason for each ending. It also checks whether any time limit is actually part of the task definition. Reviewers then inspect the target calculations for each case. The scenario illustrates an implementation distinction, not a measured improvement in a deployed agent.

Common misconceptions and mistakes

Terminal does not mean successful. A loss or specified failure can be terminal.

A leaf in a search tree is not necessarily a terminal state. Search can stop early for computational reasons.

No available action does not universally mean termination. Some games permit or require a pass, and some workflows allow waiting. Follow the model's rules.

Do not attach an arbitrary nonzero future value beyond a true episodic terminal state while also claiming the return ends there. Conversely, do not erase continuation value merely because a data-collection process stopped early.

Business use

Correct terminal-state definitions matter for game agents, simulations, workflow models, and reinforcement-learning datasets. They determine when outcomes are final and whether further consequences should influence an evaluation.

Document each terminal reason and its outcome separately. Test endings that occur before resources are exhausted, failure endings, and external interruptions. Confirm that the state includes all rule-relevant history or counters. When comparing agents, use consistent episode definitions; changing an ending condition can change scores without changing the agents' underlying capabilities.

Check your understanding

In the token game, who won in the terminal state (0,A)?

Answer: B, because B just removed the final token. A is merely the player who would have moved next if the game had continued.

Related terms

Terminal test
Utility function
Game tree
Episode
Minimax
absorbing state
truncation
goal state
value function

Learn next

Study terminal tests next. List all reachable states of the three-token game and mark genuine endings. Then draw a search tree cut off after one move and compare its leaves with the terminal set. Explain what information is required to assign each leaf a value.

Search tree

Sources

Primary references: AIMA game-playing slides, https://aima.cs.berkeley.edu/slides-pdf/chapter06.pdf; and Gymnasium, “Handling Time Limits,” https://gymnasium.farama.org/tutorials/gymnasium_basics/handling_time_limits/. Both were opened for terminal utility and the termination/truncation distinction. The token game and fictional case are original hand-checked illustrations.

Previous: Term weight Next: Terminal symbol