AI Glossary / AI and machine learning / Intermediate
Unsupported-claim rate
Definition
Unsupported-claim rate is the proportion of evaluated claims in a response that lack adequate grounding under a defined evidence standard. It is commonly expressed as a percentage and used to assess factual support in generated text.
What it means
A fluent answer can contain several checkable statements. Some may be supported by the provided documents, while others introduce details those documents do not establish. Unsupported-claim rate measures that gap at the claim level.
Unsupported does not always mean false. A statement may happen to be true but remain unsupported by the evidence the system was allowed or expected to use. Conversely, a citation can be present without actually supporting the attached claim.
The metric requires a clear definition of a claim and of acceptable support. A sentence can contain several claims, and a broad statement may need more than one passage to substantiate it.
For business readers, the rate is useful for checking whether generated reports stay within their evidence. It should be considered alongside completeness, usefulness and the severity of any unsupported statements.
How it works
Evaluators first identify the factual claims within a response using a consistent procedure. They define which statements are in scope, such as externally checkable assertions rather than clearly labeled opinions or hypothetical examples.
Each claim is compared with the permitted evidence. Reviewers record whether it is supported, contradicted, unsupported or uncertain under the chosen rubric. The reporting rule specifies how uncertain cases enter the final count.
The basic rate divides unsupported claims by all evaluated claims and multiplies by 100. Automated evaluators can assist, but their judgments need calibration against careful human review.
Across many responses, the team also specifies aggregation. Pooling all claims weights claim-heavy answers more strongly, while averaging response rates weights each response equally. Those summaries can differ substantially.
Three practical examples
- Research briefings: check whether each factual statement is grounded in the cited source material.
- Knowledge assistants: evaluate answers expected to rely on an approved internal document collection.
- Meeting summaries: inspect whether the summary introduces decisions, commitments or numbers absent from the meeting record.
Think of it this way
Think of checking the receipts behind an expense report. Every reported expense needs an appropriate supporting record, and attaching an unrelated receipt does not resolve the gap.
The analogy has a limit. Textual support can be partial or interpretive rather than a simple one-to-one match. Evaluators need a rubric for paraphrases, combined evidence and claims stronger than the source permits.
Worked example
Calculating and interpreting the rate
Suppose an invented answer contains 12 evaluated factual claims. Reviewers find adequate support for nine and no adequate support for three. Its unsupported-claim rate is 3 / 12 times 100 = 25%.
Now imagine a second answer contains two claims, one unsupported. Its rate is 50%. Averaging the two response rates gives 37.5%, while pooling their claims gives 4 / 14, or about 28.6%.
Both calculations are possible summaries, but they answer different weighting questions. A report should state which one it uses.
Neither number captures severity by itself. An unsupported opening hour and an unsupported claim about a binding commitment may have very different consequences. The example illustrates measurement choices, not acceptable quality thresholds.
Illustrative case study
A fictional consulting team adds citations to every paragraph of an AI-generated briefing. Reviewers initially assume the document is well grounded, but several citations support only one of the paragraph's multiple claims.
The team evaluates individual claims and records the exact supporting passages. It also checks omitted facts, because a very short answer can achieve a low unsupported-claim rate by saying almost nothing.
The lesson is that citation appearance and evidence coverage are different. A useful quality process checks what each source supports and whether the response still answers the user's question.
Common misconceptions and mistakes
- Every unsupported claim is definitely false. Lack of grounding and factual falsity are related but distinct judgments.
- A citation guarantees support. The source may be irrelevant, outdated or weaker than the claim.
- A zero rate proves the answer is excellent. The answer may omit important information or avoid the task entirely.
Business use
Unsupported-claim rate can support evaluation of grounded assistants, summarization and research workflows. It provides a concrete way to compare versions under a consistent evidence standard.
Use representative tasks, preserve reviewer disagreements and examine high-consequence errors separately. Pair the metric with completeness and task-success checks. When reporting improvements, keep the claim-splitting rules, evidence scope and aggregation method fixed so the comparison remains meaningful.
People also ask
What happens if a response contains no factual claims?
The rate needs an explicit undefined or not-applicable treatment; it should not silently be counted as perfect performance.
Can an AI judge calculate the rate?
It can assist, but its claim extraction and support judgments can be wrong and need validation.
Is it the same as citation precision?
No. Citation precision evaluates whether citations support their associated claims, while unsupported-claim rate focuses on claims lacking adequate grounding.
Check your understanding
An answer has eight evaluated claims and two lack support. What is its unsupported-claim rate, and does that prove those two claims are false?
Answer: The rate is 25%. It shows missing support under the rubric, not necessarily factual falsity.
Related terms
- Grounding
- Hallucination
- Citation precision
- Citation recall
- Factuality
- evaluation rubric
Learn next
Read grounding and citation precision. Split a short generated answer into factual claims, attach supporting passages and record how ambiguous or partially supported claims are handled.
More in AI and machine learning
- Unlabeled example - An unlabeled example is one record containing input information without a known target answer for the task at hand.
- Unsupervised Clustering - Unsupervised clustering groups examples by patterns in their inputs without using predefined target labels to determine the groups.
- Unsupervised learning - Unsupervised learning finds structure in data without using supplied target labels to guide the fitting objective.
- Upweighting - Upweighting increases the contribution of selected examples to a calculation or learning objective.
- VACUUM - VACUUM is a data-quality mnemonic referring to a Valid, Accurate, Consistent, Uniform and Unified Model.
- Value imputation - Value imputation replaces missing data values with chosen or estimated substitutes.
Sources
Google, Machine Learning Glossary, unsupported-claim rate: https://developers.google.com/machine-learning/glossary#unsupported-claim_rate_ucr. Core ratio definition consulted. Aggregation arithmetic and review procedures are original explanatory analysis.
Previous: Unsupervised translation Next: Unvoiced sound