AI Glossary / Reinforcement learning and robotics / Intermediate

Random policy

Definition

A random policy is a reinforcement learning action-selection rule that uses chance to choose what the agent does. A common baseline selects uniformly from the actions currently allowed, although random selection can also use unequal probabilities. The probability distribution and the permitted action set must therefore be specified before two random-policy results can be meaningfully compared.

What it means

A policy connects the agent's situation to its action choice. Calling it random describes how that choice is sampled; it does not mean the environment, rewards, or initial states must also be random. An agent can make random decisions in a completely deterministic simulation.
A fixed random policy is often useful before training begins. Its performance establishes what can happen without using learned action preferences. However, a weak random baseline does not demonstrate that a sophisticated learner is commercially useful. Simple rules, existing operating procedures, and stronger reference policies may provide more demanding comparisons.

How it works

First determine which actions are available in the current state. Then assign a valid probability distribution over them and sample an action. For a uniform policy with k allowed actions, each action has probability 1/k. Repeat this process as the environment changes.
The distribution may depend on the state even when its rule is simple. A simulator that permits four moves in an open location and two moves at a boundary gives different action probabilities in those locations. Selecting from a global action list and rejecting invalid moves can produce different behavior from sampling directly among permitted moves, especially if rejection consumes time or incurs a penalty.
Random action selection does not guarantee useful exploration. Some states require long sequences of coordinated actions and may be extremely unlikely to reach. Evaluation should use multiple independent runs and report variability alongside average return. Record the sampling rule and random seeds so experiments can be reproduced without treating one seed as representative.
A learned stochastic policy also samples actions, but its probabilities may reflect training. In practice, authors often use random policy to mean a simple untrained reference; the surrounding specification resolves that convention.

Three practical examples

• Simulator checks: confirm that actions, rewards, and episode termination work before adding a learning algorithm.
• Benchmarking: measure whether a trained game-playing agent exceeds a declared random-action reference.
• Exploration studies: examine which regions a simple random controller visits and which it rarely reaches.

Think of it this way

Think of drawing an instruction from a bag. Equal numbers of each instruction give uniform selection; different numbers give unequal probabilities. Removing instructions that are unavailable changes the bag, and therefore changes the policy. The drawing process alone does not improve the instructions over time.

Worked example

Consider a fictional one-decision environment with three permitted actions: A pays 8 points, B pays 2, and C pays -1. Every action ends the episode, and rewards are deterministic. A uniform random policy chooses each with probability 1/3.
Its expected return is (8 + 2 - 1)/3 = 3 points. Over 300 independent episodes, the expected total is 900 points and the expected count for each action is 100. Actual counts need not equal those expectations.
The second moment of reward is (64 + 4 + 1)/3 = 23. Subtracting the squared mean gives reward variance 23 - 9 = 14. The standard deviation of the sample mean across 300 independent episodes is sqrt(14/300), approximately 0.216 points. This describes sampling variability under the stated setup, not a guarantee for every experiment.
Now change the sampling probabilities to 0.50 for A, 0.25 for B, and 0.25 for C. Selection is still random, but expected return becomes 0.50 times 8 + 0.25 times 2 - 0.25 times 1 = 4.25 points. The word random alone did not determine the benchmark's performance.
Finally, suppose a different state forbids C. Uniform sampling over the two remaining actions gives expected reward (8 + 2)/2 = 5 points. Comparing this masked policy with the original three-action policy without disclosing the changed action set would be misleading. All calculations concern a one-step task; sequential environments also require accounting for how actions change future states.

Illustrative case study

Fictional case: AlderSim develops a warehouse scheduling simulator. Its first reference samples uniformly among feasible assignments. A later software update silently changes the reference to sample among all assignments and penalize infeasible ones. Average return falls even though the environment's productive opportunities remain unchanged.
The team records the action-filtering rule as part of the policy definition and restores consistent comparisons. It then evaluates a trained agent against both the documented random baseline and a simple priority rule. These invented events illustrate benchmark design, not measured deployment results.

Common misconceptions and mistakes

Common mistakes include assuming random always means uniform, reporting a single lucky run, and interpreting poor random performance as proof of effective learning. Another error is expecting random actions to cover every important state within a practical budget. Random selection must also respect the intended action constraints; it is not a justification for uncontrolled physical experiments.

Business use

Businesses can use random policies as transparent experimental references and simulator diagnostics. Their value lies in revealing how much performance comes from learned choices, environmental structure, or action filtering. Operational decisions still require suitable constraints, stronger comparisons, and evaluation on outcomes that matter beyond a training score.

People also ask

Does a random policy learn?

A fixed random rule does not, although a learning system may use random actions during exploration.

Must every action be equally likely?

No; uniformity is an additional specification.

Is a random policy the same as epsilon-greedy?

No. Epsilon-greedy combines a random component with a greedy component.

Check your understanding

What is the uniform policy's expected one-step return?

Answer: Three points.

Why is 100 selections per action not guaranteed?

Answer: Those are expected counts from random sampling.

What changes when C is forbidden?

Answer: The permitted action set and the uniform distribution over it.

Related terms

• Policy
• Stochastic policy
• Exploration
• Epsilon greedy policy
• Baseline
• action masking
• expected return
• random seed
• Monte Carlo evaluation

Learn next

Next, compare uniform, epsilon-greedy, and learned stochastic policies in the same small simulator. Keep episode rules and available actions fixed, then examine both average return and run-to-run variation.

More in Reinforcement learning and robotics

• Q-learning - Q-learning is a model-free reinforcement learning method that updates estimates of the long-term value of taking actions in states.
• R-learning - R-learning is an off-policy reinforcement-learning control method for continuing tasks with an average-reward objective.
• Random network distillation - Random network distillation (RND) is an exploration method that gives a reinforcement-learning agent an intrinsic reward based on prediction error.
• Reinforcement - Reinforcement is evaluative feedback that influences how an agent learns from its behavior.
• Reinforcement learning - Reinforcement learning is a machine learning approach in which an agent learns how to act from experience evaluated through rewards.
• Reinforcement Learning from Human Feedback - Reinforcement Learning from Human Feedback, commonly abbreviated RLHF, is a family of methods that uses human evaluations to guide reinforcement learning.

Sources

Google for Developers, Machine Learning Glossary, random policy and epsilon greedy policy: https://developers.google.com/machine-learning/glossary#random-policy Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, second edition, Chapters 2 and 3: https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf

Previous: Random perturbation Next: Random (probability) sample