AI Glossary / Reinforcement learning and robotics / Intermediate

Policy

Definition

A policy is a rule that determines how an agent selects actions from the information available to it. In a standard Markov decision process, it maps each state to an action or to a probability distribution over actions. A policy describes behavior; it is separate from the reward that defines the objective and the learning procedure that changes the behavior.

What it means

For an AI agent or robot, a policy answers the practical question of what action to take in the current situation. It may be a simple table, a collection of rules, a mathematical function, or a trained neural network.
The term here does not mean an organization's written governance policy. A reinforcement learning policy can embody a decision strategy, but it is not automatically aligned with business requirements simply because it has been trained. Its inputs, permitted actions, and objective must be specified and evaluated.

How it works

A deterministic policy selects one action for each input state. A stochastic policy assigns probabilities and samples an action from them. In notation, π(a|s) is the probability of choosing action a in state s. For a finite action set, the probabilities must be nonnegative and sum to one over the permitted actions.
A stationary policy uses the same rule whenever the same state occurs. A time-dependent policy may also consider the decision stage or remaining horizon. When observations do not reveal the true state, the decision rule may need memory, a belief state, or a history of observations.
Learning can change a policy directly by adjusting its parameters, or indirectly by changing value estimates used to select actions. A value function estimates the consequences of behavior; it is not itself the action-selection rule. A model of the environment predicts what follows an action and serves yet another role.
The policy's actual behavior includes details such as random sampling, tie-breaking, and action restrictions. Replacing sampling with a highest-probability choice changes the policy, even if the underlying network parameters remain identical.

Three practical examples

• Mobile robots: select whether to proceed, wait, or return for charging based on an appropriate state estimate.
• Scheduling: choose which eligible job to process next using queue conditions and available capacity.
• Interactive assistants: choose between asking a clarifying question, retrieving information, and producing a response, within the system's permitted action set.

Think of it this way

Think of a recipe that tells you what to do when the dough is dry, sticky, or ready. The recipe is the decision rule, while the desired bread quality is the objective and the oven's behavior is part of the environment. Following a recipe consistently does not prove it is a good recipe.

Worked example

Consider a fictional robot policy with two observed battery states, Low and High. At Low, it selects Recharge with probability 0.80 and Work with probability 0.20. At High, it selects Work with probability 0.90 and Idle with probability 0.10. Other action probabilities are zero in this example. Each state's probabilities sum to one.
Suppose an evaluation set contains 300 independent decision opportunities labeled Low and 700 labeled High. These state counts are supplied for the example; they are not derived from a robot's evolving battery dynamics. The policy produces an expected 300 × 0.20 = 60 low-battery Work choices and 700 × 0.90 = 630 high-battery Work choices, for 690 Work choices overall. Expected Recharge and Idle counts are 240 and 70, giving 1,000 expected actions in total.
Now suppose the environment's conditional probability of completing a job after Work is 0.60 at Low and 0.95 at High. The expected number of completed jobs is 60 × 0.60 + 630 × 0.95 = 634.5. A fractional expectation does not mean half a physical job is completed; realized counts are whole numbers and vary.
The probability 0.90 describes action selection at High, whereas 0.95 describes success after that action. Confusing them would mix the policy with the environment's outcome model.
If deployment always chooses the highest-probability action, the resulting deterministic policy recharges at Low and works at High. Under the same supplied state counts, it would make 700 Work choices with 665 expected completions. That is a different policy, not merely a different display of the original one. This one-step comparison does not establish which policy is best over a real battery-dependent trajectory.

Illustrative case study

Fictional case: JuniperAutomation trains a stochastic controller and records its performance in simulation. Before deployment, an engineer changes action selection to always take the most likely action. The team notices that the controller now behaves differently in situations where training intentionally used a mixture of actions.
It treats the deterministic version as a separate candidate, labels its evaluation results accordingly, and checks the effects across relevant conditions. This fictional case illustrates why the action-selection procedure belongs in a policy's specification. It does not claim either version is inherently better or report a real-world outcome.

Common misconceptions and mistakes

A policy is not the same as a reward function, a world model, or a value estimate. A stochastic policy need not select every action equally. A deterministic policy can still produce different outcomes because the environment is random. A policy that looks sensible in familiar states may behave poorly on unfamiliar inputs, and an apparently small implementation change can alter its effective behavior.

Business use

When evaluating an agent, ask what information its policy receives, what actions it can select, and how those selections are made. Keep versioned records of both learned parameters and decision settings. Test the behavior actually intended for use, including action restrictions and fallback handling. Review the policy's decisions against measured outcomes.

People also ask

Does every policy learn?

No. A hand-written rule is also a policy.

Must a policy be a neural network?

No. Tables and simple functions can represent policies.

Is the most likely action always the best action?

No. Its probability reflects the policy's construction or training, not a universal guarantee of value.

Check your understanding

What does π(Work|High) = 0.90 mean?

Answer: The policy selects Work with that probability when High is the input state.

Does it mean Work succeeds 90% of the time?

Answer: No; success belongs to the outcome model.

Why does taking the most likely action create a new policy?

Answer: It changes the action probabilities and therefore the behavior.

Related terms

• Agent
• Action space
• State
• Stochastic policy
• Policy gradient
• deterministic policy
• behavior policy
• target policy
• value function
• reward function

Learn next

Next, compare policies that use the same value estimates but different exploration rules. Then examine how a policy's value depends on both its decisions and the environment's responses.

More in Reinforcement learning and robotics

• Path consistency learning - Path consistency learning, abbreviated PCL, is a reinforcement-learning approach that jointly trains policy and value estimates to satisfy...
• Perceptual robotics - Perceptual robotics studies how principles of perception can inform the design of robots and their interaction with the world.
• Playout policy - A playout policy is the rule used to choose actions during a simulated continuation of a decision process, especially beyond the explicitly explored part of...
• Policy-gradient baseline - A policy-gradient baseline is a reference quantity subtracted from a return or action-value estimate when forming a policy-gradient update.
• Policy improvement - Policy improvement is the process of changing a decision policy using information about the value of its actions.
• Policy iteration - Policy iteration is a method for finding an improved decision policy by repeatedly alternating policy evaluation and policy improvement.

Sources

Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, second edition, Sections 3.1 and 3.5, primary text hosted by Carnegie Mellon: https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf Aalto Dictionary of Machine Learning, policy and MDP entries: https://aaltodictionaryofml.github.io/ADictML_Main.pdf

Previous: Poisson regression Next: Policy capturing