Is recall@k the same as hit rate at k?

They coincide per query when exactly one reference item is relevant. With several relevant items, finding one is a hit but may represent low recall.

Must all relevant items be in the returned list?

Not for collection-based recall. Missing relevant collection items remain in the denominator and count against coverage.

Can recall@k equal one when k is small?

Yes, if all reference-relevant items are included in that prefix. The conclusion applies only to the defined reference set.

AI Glossary / Causal inference and experimentation / Intermediate

Recall at k (recall@k)

Definition

Recall at k, written recall@k or R@k, is the fraction of a defined set of relevant items that appears among the first k ranked results. It measures how much relevant material is covered within a limited result budget. The denominator is the total relevant set specified by the evaluation protocol. Some descriptions restrict that set to a returned list, while standard collection-based retrieval uses all judged relevant items in the evaluation collection; the scope must be explicit.

What it means

A system can rank many documents or products, but an application may keep only a short candidate list. Recall@k asks how much of the desired material survives that cutoff.

This is particularly useful in a retrieval pipeline. If the first stage fails to include a relevant document among its candidates, a later ranking stage cannot promote that missing document. High candidate-stage recall can therefore matter even when the initial list contains irrelevant items.

Recall@k differs from precision@k. Precision divides relevant top-k items by k; recall divides them by the reference total of relevant items. The two describe different aspects of the same list: cleanliness and coverage.

How it works

Define the query or user, candidate universe, and relevance labels. Select a cutoff k and count the distinct relevant items among the first k results. Divide that count by the number of relevant items in the declared reference set.

For multiple queries, calculate a value per query and state the aggregation method. Equal-query averaging can differ from pooling relevant-item counts across queries. Specify treatment of queries with no relevant reference items, because their ordinary recall denominator is zero.

Document incomplete judgments, duplicates, and candidate filtering. If the denominator includes only relevant items already present somewhere in a returned list, the result measures coverage within that list; it cannot establish recall over a larger collection. If each query has exactly one relevant reference item, recall@k reduces to whether that item appears in the top k.

Three practical examples

1. Retrieval-augmented answering: developers measure whether the needed source documents enter the first-stage candidate set before a reranker or answer generator uses them.

2. Catalog recommendation: a benchmark evaluates how many of a user's held-out relevant items appear in a fixed-size recommendation list, with the reference construction stated explicitly.

3. Record matching: analysts check whether the correct matching record is included among the first ten candidates offered to a human reviewer.

Think of it this way

Think of packing a shortlist of tools for a job. Recall@k asks how many of the tools genuinely needed are included in the first k packed items. It does not ask how many unnecessary tools were packed. The analogy is limited because relevance may be uncertain or incomplete, and the reference list of needed tools determines what “coverage” can legitimately mean.

Worked example

Suppose a fictional query has eight relevant documents in a fully judged evaluation collection. A system's first five results contain three distinct relevant documents. Recall@5 is 3/8 = 37.5%, while precision@5 is 3/5 = 60%.

Among the first ten results, six documents are relevant. Recall@10 becomes 6/8 = 75%. The system's full returned list contains only those six relevant documents, so two relevant collection documents were never returned.

If a report instead divides by the six relevant items in the returned list, its within-list recall@5 is 3/6 = 50% and within-list recall@10 is 100%. Those calculations are valid for the stated narrower denominator, but they do not mean the system found all eight relevant collection documents. This example shows why the denominator's scope is essential when comparing reported recall@k scores.

Illustrative case study

Fictional case: Linden Knowledge tests a two-stage search pipeline. The reranker usually places useful documents near the top when they are supplied, but some queries receive no useful answer because the first-stage retriever never includes the needed source.

The team builds a judged query set and measures candidate recall at several cutoffs. It compares the added coverage with reranking cost and checks whether filters exclude valid documents. It evaluates final answer quality separately. The case illustrates candidate-stage coverage rather than a real system result or a guarantee that increasing the candidate list solves every answering problem.

Common misconceptions and mistakes

Do not divide by k when calculating recall@k; that produces precision@k. Likewise, do not silently change the denominator from all reference-relevant items to only those the system happened to retrieve. That can conceal misses outside the returned list.

Increasing k cannot reduce ordinary recall for a fixed ranking and fixed relevance set, because the selected prefix only grows. However, comparing separately generated lists of different sizes may involve other changes and needs care.

Unjudged items are not automatically irrelevant. If relevance labels are incomplete, report the evaluation convention and limitation. Duplicate copies of one relevant document should not inflate coverage of distinct reference items unless the task explicitly defines a different unit.

Business use

Recall@k helps teams choose candidate-list sizes and assess whether early filtering discards useful material. It can connect evaluation to a concrete resource limit, such as how many records a reviewer or reranker can process.

Report k, the candidate universe, relevance source, and denominator definition alongside the score. Consider cost and precision at the same cutoff. A large list may achieve stronger coverage but require more processing or review. The useful operating choice depends on what later stages can handle and what missing an item costs.

Check your understanding

A query has five reference-relevant items, and two appear among the first four results. What are recall@4 and precision@4?

Answer: Recall@4 is 2/5 = 40%, while precision@4 is 2/4 = 50%. They share the numerator but answer different questions.

Related terms

Recall
Precision at k (precision@k)
Candidate generation
Retrieval-augmented generation
precision at k
reranking
relevance judgments
hit rate
evaluation collection

Learn next

Create a ranked list and a separate complete relevant-item set. Calculate recall@3 and recall@6, then repeat using only relevant items present in the returned list as the denominator. Explain the different claims those numbers support.

Recall

Sources

Google, Machine Learning Glossary, recall at k, using a returned-list denominator: https://developers.google.com/machine-learning/glossary#recall-at-k

Manning, Raghavan, and Schütze, Introduction to Information Retrieval, collection-based recall: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-unranked-retrieval-sets-1.html

The sources use different reference scopes, which this entry distinguishes explicitly. Arithmetic, case, and exercise are original. Editorial review should preserve the denominator clarification.

Previous: Recall Next: Recall bias