How does it differ from supervised learning?

It learns from evaluated interaction or experience rather than relying only on correct-output labels.

Does it always need online exploration?

No; offline methods use fixed datasets, with important coverage limitations.

Is maximizing immediate reward enough?

Usually not when actions affect future outcomes.

AI Glossary / Reinforcement learning and robotics / Intermediate

Reinforcement learning

Definition

Reinforcement learning is a machine learning approach in which an agent learns how to act from experience evaluated through rewards. Its objective is typically to maximize expected cumulative reward over time. Actions can affect both immediate outcomes and the future situations the agent encounters, making delayed consequences and the tradeoff between exploration and exploitation central concerns.

What it means

A reinforcement learning task is organized around an agent interacting with an environment. The agent receives observations, chooses actions, and receives rewards and further observations. A policy specifies how it chooses actions. Training seeks a policy that performs well under the task's stated objective.
The feedback generally does not supply the best action as a labeled answer for every situation. Instead, the learner must discover useful behavior from evaluated experience. Experience may come from live interaction, a simulator, or a fixed historical dataset, depending on the method and the available evidence.

How it works

The designer specifies what the agent can observe, which actions it may take, how rewards are calculated, and when an episode ends. These choices define the learning problem and can strongly influence the behavior that emerges.
During learning, an agent may estimate values for states or actions, directly adjust a parameterized policy, learn a model of the environment, or combine these approaches. Value estimates summarize expected future reward. A learned model predicts consequences and can support planning. Different methods make different assumptions about data, dynamics, and the information available to the agent.
Exploration gathers information about actions whose consequences are uncertain. Exploitation uses existing knowledge to obtain good outcomes. A learner that exploits too early may miss better behavior; indiscriminate exploration may be costly or inappropriate outside controlled settings.
Reward design requires care because the algorithm optimizes the encoded objective. A score for intermediate activity can encourage behavior that looks productive while missing the intended outcome. Evaluation should therefore examine task success, constraints, failure cases, and performance under changed conditions, not only the training reward.
Learning is not guaranteed to succeed in every environment. Limited observations, scarce experience, delayed feedback, inaccurate simulators, and unstable optimization can all make the problem difficult.

Three practical examples

Games: learn action sequences that increase the chance of winning through simulated play.
Robotics research: train movement or manipulation policies in simulation before carefully evaluated physical trials.
Scheduling experiments: compare decisions whose effects include later congestion, capacity availability, and completion times.

Think of it this way

Think of learning a route through repeated journeys. A shortcut that saves time immediately may lead into a long queue later. The useful lesson concerns the whole trip, and trying alternatives can reveal information that repeatedly taking the familiar route would never provide.

Worked example

Consider a fictional two-stage task starting in state S. Action Quick gives reward 4 and immediately ends the episode. Action Prepare gives reward -1 and moves to state T. At T, the agent can choose Complete, earning 8 and ending the task, or Abandon, earning 0 and ending it. All transitions and rewards are deterministic.
With discount factor 0.90, the return from Quick is 4. The return from Prepare followed by Complete is -1 + 0.90 times 8 = 6.2. Prepare followed by Abandon gives -1. Although Prepare has the worse immediate reward, it supports the best return when followed by Complete.
Now suppose the agent initially uses a uniform random policy at T. It chooses Complete and Abandon with equal probability, so the expected value at T is 0.50 times 8 + 0.50 times 0 = 4. Under that continuation, Prepare has expected return -1 + 0.90 times 4 = 2.6. Quick is better than Prepare under this particular continuation policy.
Learning to choose Complete at T changes the value of preparing at S. This is why improving an isolated first decision without considering later behavior can be misleading. The example does not prescribe a particular training algorithm; it describes the returns that an algorithm would need to learn or otherwise establish.
The discount factor also matters. At 0.50, even Prepare followed by Complete returns -1 + 0.50 times 8 = 3, below Quick's 4. A policy called best is best relative to the specified objective, including how future rewards are weighted. These exact calculations assume the stated deterministic task and do not establish how many samples a real learner needs.

Illustrative case study

Fictional case: BirchScheduling evaluates a learning agent in a production simulator. A reward for starting jobs makes the agent launch many tasks at once, creating congestion and reducing completed work. The team revises the objective to reflect completion and specified operating costs, while retaining explicit capacity constraints.
It compares the revised agent with the existing scheduling rule across ordinary and stressed scenarios. The exercise illustrates the need to connect rewards with intended outcomes and validate behavior beyond a headline score. No real company results or performance gains are claimed.

Common misconceptions and mistakes

Common mistakes include assuming reinforcement learning always requires a robot, treating every reward-maximizing behavior as desirable, and confusing training performance with reliable deployment. Another mistake is assuming it must learn entirely from scratch: demonstrations, pretrained representations, models, and prior data can contribute. Historical data alone can also leave important action consequences unobserved.

Business use

Businesses should consider reinforcement learning when sequential decisions, delayed effects, and repeated experience are central to the problem. A credible project needs a measurable objective, suitable experience, meaningful comparison methods, and an evaluation environment. Simpler prediction or optimization methods may already solve the business problem effectively, so reinforcement learning should earn its added complexity through evidence.

Check your understanding

Which action sequence yields 6.2 at discount 0.90?

Answer: Prepare followed by Complete.

Why is Prepare worth only 2.6 under random continuation?

Answer: The next policy completes only half the time.

What makes Quick preferable at discount 0.50?

Answer: Future completion reward is weighted less heavily.

Related terms

Learn next

Next, implement this two-stage task and compare a random policy with a learned action-value table. Then examine how partial observations, stochastic rewards, and delayed feedback change the learning challenge.

Random policy

Sources

Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, second edition, Chapters 1 and 3: https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf Google for Developers, Machine Learning Glossary, reinforcement learning: https://developers.google.com/machine-learning/glossary#reinforcement-learning

Previous: Reinforcement Next: Reinforcement Learning from Human Feedback