AI Glossary / Causal inference and experimentation / Intermediate
Citation precision
Definition
Citation precision measures how well the citations an AI system supplies support the statements to which they are attached, under a specified evaluation rule. It focuses on the quality of provided citations rather than how many statements have evidence. A link can be real and relevant to the topic yet fail to support the particular claim. Different benchmarks handle partial support, jointly supporting sources, and redundant citations differently, so the scoring protocol must be named.
What it means
When an answer includes several references, readers expect each reference to help justify the associated statement. Citation precision asks whether those references earn their place. It is different from citation recall, which examines evidence coverage across statements. An answer can cite one statement perfectly while leaving many other claims unsupported. Conversely, every statement may have a useful citation while additional irrelevant references make the citation list noisy. Neither metric alone establishes the answer's overall usefulness or truth.
How it works
Identify the statements and map each citation occurrence to the statement it is intended to support. Retrieve the actual cited material and evaluate support at the relevant passage level. Under a simple individually supporting rule, divide the number of supporting citation occurrences by all evaluated occurrences. More sophisticated protocols allow evidence that works jointly. ALCE, for example, assesses full statement support and detects citations that neither support the statement alone nor contribute necessary evidence to the remaining cited set. State the unit, support rule, aggregation, and treatment of inaccessible citations.
Three practical examples
- Research summaries: check that references support the specific findings or limitations attributed to them.
- Policy assistants: assess whether a linked policy passage actually authorizes the stated procedure, rather than merely mentioning the subject.
- Product documentation: verify that citations support a feature description for the named product version and context.
Think of it this way
Think of placing evidence cards beside each sentence in a report. Citation precision checks whether the cards belong beside that sentence and help justify it. The analogy stops where possession of a card is treated as proof: a source can be outdated, mistaken, or quoted outside its scope even when its text appears to support the wording.
Worked example
Consider a fictional answer containing three separately evaluated statements. Statement A has two citations: one directly supports it, and one discusses a different topic. Statement B has one citation that directly supports it. Statement C has two citations that each independently support it. Under an explicitly simple rule that gives credit to each independently supporting citation occurrence, four of five citations qualify. Citation precision is therefore 4/5 = 80%.
Removing the irrelevant citation raises precision to 4/4 = 100%, but does not add any evidence for a new statement. Now imagine adding a fourth factual statement with no citation. Precision remains 100% because the set of supplied citations has not changed, while statement-level citation coverage falls from three of three to three of four. This demonstrates why precision must be paired with recall. If two sources each supported only part of a compound statement, this simple individual-support rule would be insufficient; the evaluator would need a declared joint-support protocol rather than silently changing the calculation.
Illustrative case study
A fictional knowledge-base team discovers that its assistant appends several broad overview pages to answers. The pages are authentic, but some do not support the detailed claims beside them. The team reviews citation-to-statement mappings and checks the relevant passages instead of counting valid URLs. It removes irrelevant attachments and revises unsupported wording. The improvement is recorded as better citation precision, while answer completeness and source quality remain separate review dimensions. The team avoids claiming that a cleaner reference list proves every answer is correct.
Common misconceptions and mistakes
Citation precision is not the fraction of links that open successfully. Topic relevance is weaker than support for an exact statement. Counting unique documents instead of citation occurrences can change the denominator when one source is cited several times. Redundant but genuinely supporting sources are not necessarily errors; some protocols intentionally allow them. An automated entailment judge can also be wrong, especially with tables, numerical comparisons, conflicting evidence, or long passages. Its score should not be treated as an unquestionable audit finding.
Business use
Use citation precision to reduce misleading references and make AI-generated answers easier to verify. Keep claim boundaries, source versions, cited passages, and evaluator decisions available for review. Inspect failures by type: irrelevant source, wrong scope, incorrect attribution, missing passage, or unsupported inference. Pair precision with evidence coverage and answer correctness. Where several sources jointly establish a result, explain that relationship clearly to readers and use an evaluator capable of assessing the combined evidence rather than requiring every source to prove everything independently.
People also ask
Can citation precision be high when an answer is incomplete?
Yes. The references that are supplied may be accurate even if the answer omits important information.
Does citing an authoritative organization guarantee support?
No. The particular material must support the particular claim, including its date, scope, and qualifiers.
What if the answer has no citations?
The denominator is empty, so a convention is required. It should not be presented as perfect citation quality merely because no bad citation was supplied.
Check your understanding
An answer cites a genuine document about energy use to support a specific percentage that the document never reports. Should the citation receive credit?
Answer: Not merely because the topic matches. The source must support that percentage or a clearly justified derivation under the stated evaluation rule.
Related terms
- Citation recall
- Entailment
- Retrieval-augmented generation
- attribution
- grounded generation
- source reliability
- claim verification
- joint evidence
- factual consistency
- citation mapping
Learn next
Take a short cited answer and map every reference occurrence to an exact claim. Check whether each source supports the wording, then distinguish missing evidence from irrelevant evidence. Study citation recall next, followed by source reliability and numerical verification, so a precise citation list is not mistaken for complete or correct coverage.
More in Causal inference and experimentation
- Causality - Causality concerns relationships in which changes in one part of a system can produce changes in another.
- Causation - Causation is a cause-and-effect relationship in which one event, condition or intervention contributes to producing another.
- Character N-gram F-score - Character N-gram F-score evaluates generated text by combining precision and recall of matching character sequences against reference text.
- Cohen's kappa - Cohen's kappa is a statistic measuring agreement between two raters who assign categorical labels to the same items, adjusted for agreement expected under a...
- Collider - A collider is a variable on a causal path where two arrowheads meet, as in X→Z←Y.
- Computer experiment - A computer experiment is a planned study that runs a computational model at selected input settings and analyses its outputs.
Sources
Gao and colleagues, Enabling Large Language Models to Generate Text with Citations: https://aclanthology.org/2023.emnlp-main.398/ ALCE original evaluation implementation: https://github.com/princeton-nlp/ALCE/blob/main/eval.py ALCE project documentation: https://github.com/princeton-nlp/ALCE
Previous: Circumscription Next: Citation recall