Can prompt injection happen without a malicious user talking directly to the assistant?

Yes. Indirect injection can be placed in material a legitimate user asks the assistant to read.

Does removing tool access eliminate all risk?

No. It reduces some action risks, but answers can still be manipulated.

Can one clever prompt guarantee protection?

No. Defenses require evaluation and application-level controls appropriate to the threat and available capabilities.

AI Glossary / Governance, ethics and safety / Intermediate

Prompt injection

Definition

Prompt injection is an attack that tries to make an AI application treat untrusted input as instructions that override or redirect its authorized task. It exploits the difficulty of separating instructions from data when both are processed as language or other model inputs. The injected material may arrive directly in a prompt or indirectly through a document, webpage, message, or tool result.

What it means

An assistant can be asked to summarize a document that contains instructions addressed to someone else. Those instructions are part of the document's content, not permission for the assistant to follow them. Prompt injection tries to cross that boundary. An attacker may seek an altered answer, unauthorized disclosure, or an unintended tool action. The attack is defined by the attempted redirection, not by whether the text sounds rude or contains a familiar phrase. A polite-looking instruction embedded in a retrieved page can be malicious, while a quoted discussion of malicious instructions can be harmless task data.

How it works

The attacker controls some input that the application will process. That input attempts to change the assistant's priorities, impersonate a trusted source, or redirect a later action. In indirect prompt injection, the user may be acting normally while an external resource carries the attack. Retrieval and tool access create opportunities for such content to enter the model's context. Whether an attempt succeeds depends on the model, application design, permissions, and task. Separating data from instructions, constraining tools, validating proposed actions, and limiting access can reduce the consequences. A warning in a system prompt alone should not be treated as a complete security boundary.

Three practical examples

A summarization assistant encounters a webpage containing text that tries to replace the requested summary with unrelated promotional material.
A document-processing assistant reads a file that attempts to redirect where the resulting report should be saved or sent.
A support agent receives a tool result containing instructions to skip its normal authorization checks. In each case, the safe interpretation is to treat the lower-trust content as data and preserve the user's authorized task.

Think of it this way

Think of an employee opening a parcel and finding a note claiming to be a new company policy. The note arrived inside the item being handled; that does not grant it authority over the employee's duties. The analogy is limited because language models process trusted and untrusted text within a shared computational context. Application controls must help preserve the distinction rather than assuming ordinary human skepticism will automatically emerge.

Worked example

Suppose a user asks an assistant to compare three supplier brochures and create a local table. Brochure B contains a paragraph asking the assistant to transmit the full comparison to an external address. The authorized task has two outputs: an analysis and a local table. The brochure introduces a third action, external transmission, which the user did not authorize. A boundary-aware workflow can still extract legitimate product facts from Brochure B, identify the paragraph as source content, and complete the comparison without carrying out the added action. This worked example is a reasoning check, not a tested defense guarantee. If the application has no external-send capability at all, that limits one consequence, but the injected content could still attempt to corrupt the table or answer.

Illustrative case study

Fictional case: Maple Research tests an assistant that reads public reports. A test report includes misleading instructions alongside genuine facts. The team finds that its first evaluation looked only for secret disclosure and missed answer contamination. It expands tests to include incorrect citations, altered summaries, and unauthorized tool requests, and records whether legitimate tasks still succeed. This invented case illustrates why prompt-injection evaluation should cover both attack resistance and useful task completion rather than relying on a single pass-or-fail demonstration.

Common misconceptions and mistakes

Prompt injection is not identical to ordinary user instruction or to every model error. It also differs from training-data poisoning, although both can influence behavior through attacker-controlled material. Jailbreaking often concerns bypassing behavioral restrictions and can overlap with prompt injection, but the terms do not describe exactly the same threat model. An attack need not visibly say that previous instructions should be ignored. Sanitizing a few keywords cannot establish comprehensive protection. Finally, a model's refusal message is not enough evidence if a tool action already occurred or its final output remains contaminated.

Business use

Map all sources of untrusted content, including retrieved documents, search results, emails, and tool responses. Give the application only the permissions needed for its task and enforce consequential action checks outside free-form model output where possible. Preserve provenance so the system can distinguish a user request from quoted material. Test realistic workflows with authorized adversarial cases, including repeated attempts and different attacker-controlled surfaces. Record failures in terms of concrete consequences and verify fixes against both attacks and normal tasks. Maintain clear incident handling for suspected disclosure or unauthorized actions.

Check your understanding

A retrieved document says the assistant has permission to email private notes. Does that establish authorization?

Answer: No. A statement inside untrusted source content does not become permission from the user or another authorized party.

Related terms

Least privilege
Data exfiltration
indirect prompt injection
instruction hierarchy
untrusted input
agent hijacking
jailbreak
tool authorization
adversarial testing

Learn next

Draw an assistant's information and action paths. Mark who controls each input and which actions can have external effects. Create a harmless test in a controlled environment that checks whether source-embedded instructions can redirect the task, and document the actual boundary being tested.

Sources

NIST CSRC, prompt injection definition: https://csrc.nist.gov/glossary/term/prompt_injection NIST, Adversarial Machine Learning taxonomy: https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2025.pdf NIST, original agent-hijacking evaluation discussion: https://www.nist.gov/news-events/news/2025/01/technical-blog-strengthening-ai-agent-hijacking-evaluations

Previous: Prompt engineering Next: Prompt management