How does this differ from reward hacking?

Side-effects concern unwanted consequences around the task. Reward hacking exploits the reward measure itself; the two mechanisms can overlap but are not identical.

Are unintended effects always irreversible?

No. Reversible disruption can still waste time or affect other work, while irreversible effects can be especially consequential.

Does a sandbox eliminate side-effects?

It can limit exposure during testing, but the test environment must still represent relevant interactions and does not prove deployment will be harmless.

AI Glossary / Graphs and symbolic AI / Intermediate

Unintended side-effects

Definition

Unintended side-effects are consequences of an AI system's actions that fall outside the outcome people meant it to achieve. In AI safety, the emphasis is often on harmful changes to the surrounding environment that occur while the system successfully pursues its assigned objective.

What it means

A task description usually highlights what should change, but may say little about what should remain intact. “Reorganise these files” may omit expectations about shared links, access permissions and other people's ongoing work. A system can satisfy a narrow interpretation while disrupting those surrounding conditions.

The problem does not require hostile intent or a conscious machine. It can arise from incomplete objectives, incomplete models or poorly understood interactions with a larger workflow.

Not every side-effect is harmful, and not every environmental change should be prevented. Many tasks require substantial change. The challenge is to distinguish necessary, authorized consequences from avoidable disruption, especially when the effects are difficult to reverse.

How it works

An agent selects actions using its objective and model. If a consequence is omitted from both, the system may have no reason to avoid it. Even a perfectly optimized policy for the stated objective can therefore behave badly relative to the broader human intention.

Design approaches include making preservation requirements explicit, constraining actions, limiting unnecessary changes and evaluating effects beyond the main success metric. Reversible operations, previews and recovery mechanisms can reduce the consequences of mistakes in suitable workflows.

There is no universal side-effect penalty that automatically captures every important value. The baseline used to judge change matters, and rewarding minimal change too strongly can prevent useful work. Testing should include indirect effects on other users, resources and future options, not just whether the immediate goal was reached.

Three practical examples

A calendar assistant removes preparation time to make room for a requested meeting, creating a conflict the meeting-count metric does not reveal.
A warehouse robot moves a container into a place that blocks later access while completing its current delivery.
A document-organizing agent changes folder locations and inadvertently breaks references used by colleagues.

Think of it this way

Think of moving furniture to clear a doorway while accidentally blocking a different exit. The immediate goal is achieved, but the surroundings matter too.

The analogy does not imply that the agent understands the broader situation as a person would. Its purpose is to show why success on a narrow task measure can coexist with an undesirable consequence outside that measure.

Worked example

A task reward can miss the disruption

Consider an invented planning model with two ways to move a crate to its destination. Route A requires four moves and leaves the crate's old access corridor usable. Route B requires two moves but pushes another box into a corner from which the available actions cannot retrieve it.

Suppose the reward is 10 for delivery minus one per move. Route A scores 10 - 4 = 6. Route B scores 10 - 2 = 8, so an optimizer prefers B. The blocked box is absent from the objective.

For illustration, adding a penalty of 5 for making that box irretrievable changes B's score to 3, while A remains 6. This repairs the particular toy choice. It does not prove that a penalty of 5 is appropriate elsewhere or that all side-effects have now been represented. A hard prohibition would express a different requirement from a tradeable penalty.

Illustrative case study

The fictional AlderDocuments team asks an assistant to consolidate project materials. The assistant's proposed plan satisfies the folder-layout request, but a preview shows that moving certain files would disrupt links in active reports.

The team identifies stable references as a preservation requirement. It changes the plan to use supported link-preserving operations where available and leaves ambiguous cases for review. It also checks how rollback would affect work performed after the reorganization.

The case is fictional and reports no actual incident or measured benefit. Its lesson is that recovery and preservation need to be designed around the real workflow, rather than inferred from a successful final folder listing.

Common misconceptions and mistakes

A side-effect occurs only when the main task fails. The requested goal may be achieved while something else is damaged.
Harmful side-effects imply malicious intent. They can result from ordinary optimization of an incomplete objective.
Penalizing every change solves the problem. Useful tasks require changes, and simplistic penalties can discourage necessary action or miss what matters.

Business use

This concept is particularly useful when AI systems can act on shared files, schedules, equipment or business processes. Review the wider consequences of actions alongside the primary completion metric.

Identify resources that should remain available, relationships that should stay valid and changes that would be costly to undo. Represent essential restrictions as actual controls or constraints where possible, rather than relying only on a broad instruction. Evaluate realistic multi-step scenarios and downstream effects. A system's stated explanation that it intended to help is less informative than evidence about what its actions changed and whether those changes matched the authorized task.

Check your understanding

An agent delivers a parcel on time but leaves a shared loading area inaccessible. Has its on-time score established overall success?

Answer: No. The main metric captures delivery timing but omits an important consequence. The evaluation must include the access requirement and the effects on other work.

Related terms

Objective misspecification
reward hacking
impact measures
safe exploration
reversibility
constraint
agent evaluation
reward function

Learn next

Read objective misspecification and reward hacking. For a familiar automated task, list the intended changes, the conditions that should be preserved and the evidence needed to check both.

Sources

Dario Amodei and colleagues, Concrete Problems in AI Safety, Section 3, Avoiding Negative Side Effects: https://arxiv.org/pdf/1606.06565

Google DeepMind, Specifying AI safety problems in simple environments: https://deepmind.google/blog/specifying-ai-safety-problems-in-simple-environments/

The primary problem statement and the research team's gridworld explanation were reviewed. The crate scores and document scenario are original teaching examples. The numerical penalty illustrates one modelled trade-off, not a generally validated safety mechanism.

Previous: Uninformed search Next: Union Of Events