Does deep reinforcement learning need labeled examples of correct actions?

Not necessarily. It learns from rewards and experience, although demonstrations or other supervision may be incorporated.

Is it the same as deep Q-learning?

No. Deep Q-learning is one family within a broader field that includes policy-based and model-based approaches.

Must training happen in the live environment?

No. Training may use simulation or logged experience, but each approach introduces its own assumptions and evaluation challenges.

AI Glossary / Deep learning / Intermediate

Deep reinforcement learning

Definition

Deep reinforcement learning combines reinforcement learning with deep neural networks. An agent learns how to choose actions through rewards associated with its interaction with an environment, while neural networks represent quantities such as action preferences, expected returns, or environment dynamics. The defining problem concerns consequences across a sequence of decisions. It is broader than a single algorithm and does not require that every component be learned from scratch.

What it means

A prediction system might estimate tomorrow's demand. A reinforcement-learning system could decide how much stock to order, knowing that today's order affects future availability and costs. The system must connect actions with outcomes that may arrive later.

Deep networks help when observations are complex or the number of possible situations is too large for a simple lookup table. They do not supply the business objective automatically. Designers still specify observations, available actions, rewards, constraints, and the environment in which learning is possible. A high reward only means success according to that formulation; it can conceal undesirable behavior if the formulation is incomplete.

How it works

An agent observes its situation, selects an action, receives a reward, and encounters a new situation. Repeated experience supplies training data. A policy determines action selection; a value estimate predicts the return associated with a state or action. Some approaches learn a model of the environment for planning.

In deep Q-learning, a network estimates action values. Training compares an estimate with a target combining an observed reward and an estimate of subsequent value. Experience replay reuses stored transitions, and a target network can stabilize the targets. Policy-gradient and actor - critic methods use different update procedures.

Evaluation requires separate runs, repeated training seeds, realistic conditions, and comparison with credible alternatives. Successful training curves alone are insufficient evidence of dependable decisions.

Three practical examples

1. Simulated robot control: a team trains a controller to reach objects, then examines collision rates and transfer to conditions outside the training simulator.

2. Warehouse scheduling: a research prototype assigns available jobs while accounting for queues, delays, and machine availability over subsequent decisions.

3. Game learning: a student trains an agent using observations, legal actions, and game rewards, comparing different exploration strategies under an equal interaction budget.

Think of it this way

Think of learning a board game by playing and reviewing outcomes, instead of receiving a correct move for every position. A seemingly weak move might create a later advantage. The analogy has limits: a neural agent does not necessarily understand the game's purpose, and it can exploit accidental scoring rules. Experience in one environment may teach habits that fail when the rules or observations change.

Worked example

Consider a simplified Q-learning transition. The network currently assigns value 4 to a chosen action. The action produces reward 2. At the next nonterminal state, the largest target-network action value is 5. With discount factor 0.9, the training target is 2 + 0.9 × 5 = 6.5.

The target exceeds the current estimate by 2.5. A squared-error illustration would have error 2.5² = 6.25; actual implementations may use another loss. Training adjusts network weights toward reducing this discrepancy, potentially changing estimates for other states too.

If the transition instead ends the episode, with no future rewards, the target is just 2. This arithmetic illustrates one update target, not a complete training run, convergence guarantee, or recommended hyperparameter choice.

Illustrative case study

Fictional case: North Dock builds a simulator for assigning loading bays. Its first reward counts completed loads, and the agent repeatedly favors short jobs while long jobs wait. The high score therefore hides a service failure.

The team adds explicit waiting-time evaluation, revisits its objective, and compares the revised agent with a dispatch rule using identical demand scenarios. It also tests blocked bays and unfamiliar arrival patterns. A supervisor reviews proposed assignments before any operational trial. The lesson is that reward design, constraints, and independent evaluation are part of the project, rather than details solved by choosing a larger network.

Common misconceptions and mistakes

Do not confuse reward with a label identifying the correct action. Reward usually evaluates an outcome and may provide little information about which earlier decision caused it.

Avoid assuming more exploration is always harmless: actions can consume resources or damage equipment. A simulator also has assumptions that require testing.

Do not compare methods using only their best training run, omit interaction costs, or treat a higher simulated return as proof of improvement in deployment. Distribution changes and incomplete observations can undermine apparently strong policies.

Business use

Business value is most plausible when decisions repeat, consequences extend over time, and a credible learning environment or suitable logged experience exists. Useful questions include whether simpler optimization or rules already solve the problem and whether rewards capture the actual service objective.

Estimate data collection, simulation, compute, monitoring, and intervention costs. Evaluate operational measures such as delay, resource use, and constraint violations alongside return. Agree who can override the agent and how performance will be assessed after demand or equipment changes. The neural architecture is only one part of that decision system.

Check your understanding

Why can an agent with the highest reward still be unsuitable for a business process?

Answer: The reward may omit important outcomes or allow shortcuts. Independent measures, realistic tests, and operational constraints are needed to establish whether the policy achieves the intended objective.

Related terms

Learn next

Specify a small sequential task using observations, actions, rewards, and episode boundaries. Calculate targets for one continuing and one terminal transition. Then design an evaluation metric that could reveal a failure hidden by reward, and compare the learner with a simple rule under the same budget.

Evaluation metric

Sources

Primary sources: Mnih and colleagues, Playing Atari with Deep Reinforcement Learning, https://arxiv.org/abs/1312.5602 ; PyTorch, Reinforcement Learning (DQN) Tutorial, https://docs.pytorch.org/tutorials/intermediate/reinforcement_q_learning.html . The primary paper abstract and tutorial mechanism were consulted. The business scenarios and arithmetic are original teaching illustrations, not reported deployments or reproduced experimental results.

Previous: Deep Q-Network Next: Deep tomographic reconstruction