Does finite horizon mean no uncertainty?

No. Transitions and observations can be uncertain even when the number of stages is fixed.

Can a finite-horizon model use discounting?

Yes. Discounting and the endpoint are separate parts of the objective.

Is a time-dependent policy incompatible with a stationary state model?

No. Remaining time can change the best action even when physical transition rules stay the same. Time can also be included explicitly in the state representation.

AI Glossary / Graphs and symbolic AI / Intermediate

Finite horizon

Definition

A finite horizon is a limited span of future stages over which a decision process is evaluated or planned. In a finite-horizon model, rewards, costs or goals are considered up to that endpoint, often with a specified terminal value. Best actions can depend on remaining stages.

What it means

Some tasks have a natural end: an event finishes, a campaign closes or a machine has a scheduled operating period. Others continue indefinitely but are planned using a bounded look-ahead window. Both can involve finite-horizon calculations, although an artificial planning cutoff is different from the real task ending.

The horizon counts decision stages, not necessarily equal clock-time intervals. It differs from state-space size and computer runtime limits. A model with only a few states can have a long horizon, and a short-horizon problem can still contain a large or difficult state space.

How it works

Specify the number of remaining decisions, the allowed actions, state transitions and rewards or costs. Define what happens at the endpoint: no further value, a terminal reward, a disposal cost or an estimate of value beyond the planning window.

For a finite Markov decision process with a known model, backward induction starts from the terminal values. At each earlier stage, it compares each action's immediate reward plus the expected value of its next state. The chosen action can change with the stage even when the physical state and transition rules are unchanged.

Discounting is optional in a finite-horizon objective; it can be used but is not what makes the horizon finite. Receding-horizon planning repeatedly solves a bounded problem, executes an initial action and plans again. Its behavior depends on terminal assumptions and updated information, so it should not be confused with executing one fixed finite plan unchanged.

Three practical examples

1. A temporary exhibition plans staffing until its closing date. An action that pays off after the exhibition ends may have little value unless the model explicitly includes a later benefit.

2. A production planner schedules work across a known set of shifts. Inventory remaining after the last shift needs a defined terminal treatment rather than being silently valued at zero.

3. A robot considers a fixed number of future moves before choosing its next action. This is a bounded planning window; the robot may continue operating and replan after moving.

Think of it this way

Think of packing for the final hour before a trip. With an hour left, reorganizing a bag may be worthwhile; with one minute left, finishing the essential items may be better. The analogy explains why remaining time changes choices. A formal horizon additionally specifies stages, outcomes and an objective, rather than relying on an informal sense of urgency.

Worked example

Consider a deterministic machine with states Unprepared and Prepared. In Unprepared, action Work earns 2 reward units and leaves the machine unprepared. Action Prepare earns zero now and changes the state to Prepared. In Prepared, Work earns 5 units per stage and keeps that state. Assume zero terminal value and no discounting.

With one stage remaining in Unprepared, Work yields 2 while Prepare yields 0, so Work is best. With two stages remaining, Work followed by the best final action yields 2 + 2 = 4. Prepare followed by Work yields 0 + 5 = 5, so Prepare is best.

The same physical state therefore has different optimal actions depending on the remaining horizon. If the machine has already reached Prepared with two stages left, working twice yields 10. The calculations depend on the stated zero terminal value: assigning a residual value to a prepared machine after the endpoint could change the final-stage comparison. These invented rewards illustrate planning logic, not a measured production process.

Illustrative case study

Fictional case study: Cedar Operations uses a rolling weekly plan that values only completed work inside the window. A preparation task repeatedly appears unattractive because most of its benefit falls just beyond the cutoff. Replanning moves the cutoff forward, but the same omission can recur.

The team reviews whether its horizon and terminal value capture the real objective. It compares longer windows and explicit residual-value assumptions on small scenarios. The case illustrates a planning artifact, without claiming that a particular remedy is always sufficient or that the revised plan improves actual performance.

Common misconceptions and mistakes

A finite horizon is not the same as an indefinite task that eventually reaches a terminal state after an unknown number of steps. A fixed maximum number of stages is a specific modeling choice. It is also different from an infinite-horizon model with discounting, even though discounting can reduce the practical influence of distant rewards.

Do not assume a stationary policy is always optimal for a finite-horizon problem. Remaining time can matter. Finally, truncating future rewards at zero can favor actions that look good inside the window while deferring costs or benefits beyond it. The endpoint deserves explicit modeling attention.

Business use

Choose a horizon that reflects the decision's real consequences and the reliability of available forecasts. Document terminal costs or benefits, the stage duration and whether the model will be replanned. Compare outcomes under plausible changes to these assumptions rather than treating the chosen endpoint as inevitable.

For implementation, include the stage or remaining time wherever it is needed to select actions correctly. Evaluate the resulting policy over the operating period, not just the optimization window. A bounded plan can make computation manageable, but its usefulness depends on what the cutoff excludes and how later decisions reconnect to the objective.

Check your understanding

Why is Prepare best with two stages remaining but not with one in the worked example?

Answer: Two stages allow its later payoff of 5, exceeding two unprepared Work rewards totaling 4. With only one stage, the zero terminal value leaves no modeled opportunity to benefit from preparation.

Related terms

Markov decision process
Policy
Discount factor
Infinite horizon
Planning horizon
terminal reward
backward induction
receding-horizon planning

Learn next

Solve a two-state, two-stage decision problem backward from its terminal values. Then change the horizon and terminal reward separately, checking which action changes and why. This clarifies the consequences of an artificial cutoff.

Sources

Primary sources: Poole and Mackworth, “Planning Horizon”: https://artint.info/2e/html2e/ArtInt2e.Ch1.S5.SS2.html ; MDPtoolbox finite-horizon documentation: https://search.r-project.org/CRAN/refmans/MDPtoolbox/html/mdp_finite_horizon.html . The worked machine model and fictional case are original teaching examples, not outputs from an executed planning tool.

Previous: Finite-dimensional distribution Next: Finite mixture model