AI Glossary / Causal inference and experimentation / Beginner

Recall

Definition

Recall is the fraction of actual positive cases that a system correctly identifies as positive. It is calculated as true positives divided by true positives plus false negatives. In retrieval, it is the fraction of all relevant items in the defined reference collection that are retrieved. Recall measures coverage of the target cases. It does not tell you how many selected results are false alarms; that is a different question addressed by precision.

What it means

Imagine a system looking for messages that require a return authorization. Recall asks how many of the messages that genuinely require authorization the system finds. A missed target message reduces recall even if the system's other selections are correct.

This focus makes recall useful when omissions matter. However, selecting everything can produce complete recall while overwhelming users with irrelevant results. A recall score therefore needs context about false positives, workload, and the purpose of the search or classification.

The target condition and reference labels determine the denominator. Without a credible estimate or labeled set of actual positives, a claim that the system found most target cases may be unsupported.

How it works

Define the positive class and obtain reference labels for an appropriate evaluation sample. Count actual positives that the model predicts positive as true positives, or TP. Count actual positives predicted negative as false negatives, or FN.

Calculate recall = TP/(TP + FN). For a fixed labeled dataset, changing a score threshold can change which positives are found. Under a rule that selects scores at or above the threshold, lowering the threshold expands the selected set and cannot reduce recall on that same dataset.

Report the class, threshold, and sampling conditions. When evaluating several classes, specify whether recall is reported separately, averaged equally across classes, or combined using another convention. If the reference set contains no positive cases, the basic ratio has a zero denominator, so any software-assigned value needs explanation.

Three practical examples

1. Support triage: recall measures what proportion of genuinely urgent messages are routed to the urgent queue, revealing how many are overlooked.

2. Document search: recall measures how much of a defined relevant document set is found, which matters when users need broad coverage rather than one good answer.

3. Visual inspection: recall measures the share of items with a specified defect that the detector flags, alongside the burden of unnecessary inspections.

Think of it this way

Think of collecting all the blue marbles from a box. Recall asks what fraction of the blue marbles ended up in your collection, regardless of how many other colors you collected too. The analogy is limited because real-world positives are not always clearly visible or completely labeled. The apparent denominator may itself require careful review and may not represent every future case.

Worked example

Suppose a fictional dataset contains 300 messages, including 60 actual return-authorization requests. A model identifies 45 of those requests and misses 15. It also flags 30 ordinary messages.

Recall is 45/(45 + 15) = 75%. The 30 false positives do not enter that formula. They do affect precision, which is 45/(45 + 30) = 60%.

Now a looser threshold finds 54 of the 60 requests but flags 66 ordinary messages. Recall rises to 90%, while precision becomes 54/(54 + 66) = 45%. Review volume grows from 75 to 120 messages. This invented comparison shows why a recall improvement must be interpreted alongside workload and false alarms. It also shows that recall alone does not specify the operational consequence: finding nine additional requests required reviewing forty-five additional messages in this example.

Illustrative case study

Fictional case: Chestnut Records wants a search tool that helps staff find all files related to a project. A pilot receives praise because the first few results are relevant. A careful reference review later identifies many relevant files with older terminology that the system missed.

The team adds a recall-oriented test set containing both old and current vocabulary. It reports which relevant files remain unfound and checks whether expanding retrieval makes review workloads manageable. This illustrates why a clean first page does not establish comprehensive coverage. No real organization or measured improvement is claimed.

Common misconceptions and mistakes

Recall is not the percentage of predictions that are correct. That would involve a different denominator. It is also not precision: precision examines selected cases, whereas recall examines actual positives.

Perfect recall can be achieved trivially by predicting every example positive when positives are present. Such a system may be unusable because it provides no filtering. Conversely, a high-precision system can have low recall if it selects only easy cases.

Do not assume incomplete reference labels establish complete coverage. Unlabeled positives may be treated as negatives or omitted from evaluation. In retrieval, the total relevant set can be difficult to determine, so explain the judgment procedure and the scope of any recall estimate. Small positive counts also make estimates uncertain.

Business use

Recall connects model evaluation to missed opportunities, overlooked requests, and incomplete searches. It is useful when the organization must understand what the system fails to surface.

Set recall goals in conjunction with error consequences and review capacity. Include the number of actual positives, false negatives, and alerts rather than only a percentage. Inspect misses to learn whether particular input types or groups are systematically overlooked. A coverage target is more actionable when teams know which cases remain outside the system's effective reach.

People also ask

Is recall the same as sensitivity?

In a binary classification setting, sensitivity and true-positive rate are common names for the same ratio.

Can recall be calculated from positive predictions alone?

No. You also need the actual positives that were missed, or an appropriate reference total.

Does higher recall always mean a better system?

No. It may come with many more false positives or greater cost. The relevant trade-off depends on the task.

Check your understanding

A reference set contains 80 relevant documents. A search returns 50 documents, of which 40 are relevant. What is recall?

Answer: Recall is 40/80 = 50%. The returned-list size of 50 is used for precision, which would be 40/50 = 80%.

Related terms

  • Precision
  • False negative
  • True positive
  • Positive class
  • Recall at k (recall@k)
  • Confusion matrix
  • sensitivity
  • recall at k
  • retrieval evaluation

Learn next

Construct two result sets that retrieve the same number of relevant items but contain different numbers of irrelevant items. Calculate recall and precision for each, then explain why recall stays fixed while review effort changes.

  • Precision

More in Causal inference and experimentation

  • Random experiment - A random experiment is a process with an uncertain outcome drawn from a defined set of possible outcomes, called its sample space.
  • Randomized weighted majority algorithm - The randomized weighted majority algorithm combines a set of predictors by choosing predictions randomly according to their current weights, then reducing...
  • Ranking loss - Ranking loss, in the multilabel evaluation sense used here, measures the fraction of relevant-versus-irrelevant label pairs that a model orders incorrectly...
  • Remediation - Remediation is corrective action taken to address an identified weakness, failure or harmful outcome.
  • ROC (receiver operating characteristic) Curve - A receiver operating characteristic curve, or ROC curve, plots a classifier's true-positive rate against its false-positive rate as the decision threshold...
  • ROUGE-L - ROUGE-L is a member of the ROUGE text-evaluation family that compares a candidate with a reference using a longest common subsequence, or LCS.

Sources

Google, Classification: Accuracy, recall, precision, and related metrics: https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall

Manning, Raghavan, and Schütze, Introduction to Information Retrieval, evaluation of unranked retrieval sets: https://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-unranked-retrieval-sets-1.html

These primary references support the classification and retrieval definitions. Calculations, case, and exercise are original teaching material.

Previous: Reassignment method Next: Recall at k (recall@k)

  • Home
  • Daily AI Updates
  • Blog
  • AIFBL Academy
  • AI Glossary
  • Products and Tools
  • Store
  • Free CFO AI Prompt Pack
  • About Shihan Sheriff
  • Privacy Policy
  • Terms of Use
  • Refund Policy
  • Cookie Policy
  • Earnings Disclaimer