AI Glossary / Governance, ethics and safety / Beginner
Hallucination
Definition
In AI, a hallucination is generated content that is false, unsupported or inconsistent with the information it is supposed to reflect, often presented convincingly. Examples include an invented quotation or a summary that adds a decision nobody made. The term describes an output problem; it does not mean the system experiences human perception.
What it means
A response can sound polished and still contain a fabricated detail. Correct sentences around that detail can make the mistake harder to notice. Reading an answer for fluency therefore differs from checking its evidence.
There are two useful questions. Is the statement accurate in the world? And does the statement faithfully represent the supplied material? A summary might add a generally true observation that the original document never made. That can still be an unfaithful summary if the task required reporting only the document's contents.
An unsupported claim is not automatically disproved. It may be true but lack evidence in the available sources. Marking that distinction helps reviewers avoid replacing one unjustified certainty with another. Deliberately requested fiction is also different from presenting an invented detail as fact.
How it works
Language generation learns patterns that support plausible responses. The process does not automatically check every name, number or citation against an authoritative record. Missing information, confusing context and learned associations can produce a convincing continuation that is wrong.
Consider a request for a source supporting a claim. A model may produce the familiar shape of a citation: authors, title, journal and year. That shape is not proof that the publication exists. Even a real source may fail to support the attached claim.
Retrieval can supply relevant evidence before generation, but it can retrieve the wrong passage, miss a qualification or provide outdated material. The generator may then misuse what it receives. Supplying sources is a useful control, not a guarantee that hallucination disappears.
Three practical examples
Meeting summaries: an assistant reports that a manager approved a budget when the transcript only records a proposal. Check the decision against the actual discussion.
Customer service: a draft promises a return period that the company's policy does not offer. Check the applicable policy version before sending it.
Personal research: an answer names a book and quotation that cannot be found in the stated source. Verify the title and passage independently instead of treating the citation's formatting as evidence.
Think of it this way
Think of a beautifully labeled map containing an invented bridge. The presentation may look dependable, and most roads may be correct, but the false connection matters when someone relies on it.
The analogy concerns the output, not an intention to deceive. A language model need not have beliefs or intentions for its answer to mislead a reader.
Worked example
Suppose a reviewer breaks an AI-written briefing into twenty distinct factual claims. These are invented teaching figures. Fifteen claims are supported by the agreed reference documents, three contradict them and two cannot be checked from those documents.
The supported share is 15 ÷ 20 = 75%. The contradicted share is 3 ÷ 20 = 15%, while the unverified share is 2 ÷ 20 = 10%. The three shares total 100%.
Calling all five remaining claims proven false would be wrong: two are unresolved. Calling the briefing safe because most claims are supported would also be premature. A single incorrect deadline could matter more than several correct background facts. This simple claim audit is inspired by factual-precision evaluation, but it is not a universal hallucination score or a substitute for judging consequences.
Illustrative case study
A fictional equipment supplier uses an assistant to draft answers from its product manuals. A customer asks whether a particular accessory fits an older machine. The retrieved passage describes the newest model, but the draft confidently says the accessory fits both.
A support specialist catches the model mismatch and checks the older manual. The team changes its workflow to require matching model identifiers before drafting compatibility answers and to escalate missing evidence.
This invented case shows that a source-backed workflow can still fail through incorrect matching. The lesson is to inspect how evidence supports the specific claim, rather than assume that adding retrieval solves the problem.
Common misconceptions and mistakes
Confidence means correctness. A confident tone is a writing feature, not a reliable measurement of factual accuracy.
Asking the same model to check itself is independent verification. It may repeat the same mistake; use an external source or a suitable separate check.
Any error is a hallucination. A broken calculator integration or a transcription error may have a different cause. Identify the actual failure so the remedy addresses it.
One correct citation validates the whole answer. Each material claim needs appropriate support, especially when the answer combines several sources.
Business use
Define which outputs need verification and who can perform it. For factual drafting, require traceable evidence for material claims and a clear way to say that information is unavailable. Separate source-backed facts from suggestions and assumptions.
Keep a small test set reflecting real customer questions, including questions the documents cannot answer. Record unsupported additions as well as obvious falsehoods. Review recurring patterns such as invented dates or mismatched product versions. The goal is an accountable process that catches consequential errors before customers rely on them.
People also ask
Can a hallucination include a real web address?
Yes. A real page can be cited for something it does not say.
Does a low creativity setting prevent it?
No. More repeatable wording does not establish factual correctness.
Should every unsupported claim be deleted?
It should be verified, clearly qualified or removed according to the task. An unresolved factual assertion should not silently become a confirmed fact.
Check your understanding
An answer cites a real manual, but the cited passage concerns a different product. Has the claim been verified?
Answer: No. Source existence and source relevance are separate. Check the exact product and passage; if the evidence is missing, preserve that uncertainty rather than invent compatibility.
Related terms
Learn next
Practice a claim-by-claim review of one short generated answer. Label claims supported, contradicted or unresolved, then explain which would matter most if acted upon. Learn retrieval-augmented generation next to understand where evidence enters the workflow.
More in Governance, ethics and safety
- Group attribution bias - Group attribution bias is the tendency, in this glossary's machine-learning sense, to assume that a characteristic observed in one individual also applies...
- Group fairness - Group fairness assesses how an AI system's decisions, errors, or benefits compare across defined groups of people.
- Groupthink Bias - Groupthink Bias is a pattern in which pressure to agree with a group discourages independent criticism and contributes to poor decisions.
- Harmful bias - Harmful bias is a systematic pattern in an AI system or its surrounding process that contributes to unjustified disadvantage, exclusion, misrepresentation...
- High-risk artificial intelligence system - A high-risk artificial intelligence system is an AI system placed in a category requiring stronger safeguards because of its intended use and potential...
- Historical bias - Historical bias is bias already present in social conditions, institutions, or earlier decisions that becomes reflected in data used by an AI system.
Sources
NIST, Generative AI Profile, section 2.2: https://doi.org/10.6028/NIST.AI.600-1 . Min and colleagues, FActScore: https://arxiv.org/abs/2305.14251 . Lewis and colleagues, Retrieval-Augmented Generation: https://arxiv.org/abs/2005.11401 . Primary sources consulted 15 October 2026. The claim audit and fictional case are original; no benchmark result is implied.
Previous: Half-normal distribution Next: Hamiltonian Monte Carlo