Why is the positive-class definition important?

Reversing which event counts as positive changes the meaning of precision and recall and can change the PR summary.

Can two models have similar PR area but different useful thresholds?

Yes. Their curves can differ or cross in the region relevant to the application.

Is average precision always equal to PR AUC?

Only if “PR AUC” has explicitly been defined to mean that particular AP convention; trapezoidal area generally differs.

AI Glossary / Causal inference and experimentation / Intermediate

Area under the PR curve

Definition

Area under the PR curve summarizes a model's precision-recall tradeoff as its decision threshold changes. Precision measures the fraction of flagged cases that are truly positive, while recall measures the fraction of actual positives that were found. The summary is often called PR AUC or AUPRC. Its numerical value depends on how the curve is constructed and integrated; average precision is a related summary that is not generally identical to trapezoidal PR area.

What it means

A scoring model ranks examples before a threshold turns those scores into positive or negative decisions. At a strict threshold, it may flag only a few high-scoring examples. Lowering the threshold can find more positives while also adding false positives. The PR curve plots the resulting precision against recall. It focuses attention on positive detections and missed positives, which can be useful when positive events are uncommon. However, it does not directly encode the cost of an error or identify the best operating threshold.

How it works

Collect reference labels and unthresholded prediction scores on appropriate held-out data. Sweep score thresholds, calculate true positives, false positives, and false negatives, and derive precision and recall at each threshold. Then apply a named summary convention. Trapezoidal integration connects recorded points with straight lines in PR space. Non-interpolated average precision instead weights precision values by increases in recall. Ties, endpoint conventions, sample weights, and multiclass or multilabel averaging can also affect reported values, so a bare “PR AUC” label is insufficient for reproducible comparison.

Three practical examples

Rare-defect detection: compare how well models prioritize actual defects while accounting for the false alarms among flagged items.
Search evaluation: examine the balance between retrieving relevant documents and keeping the retrieved set useful.
Review queues: compare ranking quality across thresholds before separately evaluating performance at the team's actual review capacity.

Think of it this way

Think of widening a net: a small opening may catch a selective set, while a wider opening can recover more desired items along with unwanted ones. The analogy illustrates the tradeoff. Its limit is that a model's ranking can produce irregular changes in precision, and the area summary does not tell you which net width fits the real costs or capacity.

Worked example

Consider four examples ranked from highest to lowest score with labels [positive, negative, positive, negative]. There are two positives. After selecting the first example, precision is 1 and recall is 0.5. After two, precision is 0.5 and recall remains 0.5. After three, precision is 2/3 and recall is 1. After four, precision is 0.5 and recall stays 1.
For non-interpolated average precision, recall increases by 0.5 at each positive encounter, so AP = 0.5 times 1 plus 0.5 times 2/3 = 5/6, approximately 0.8333. For a trapezoidal calculation using the ordered points (recall, precision) = (0, 1), (0.5, 1), (0.5, 0.5), (1, 2/3), and (1, 0.5), the area is 0.5 times 1 plus 0.5 times (0.5 + 2/3)/2 = 19/24, approximately 0.7917. Vertical segments contribute zero horizontal area. The same ranking therefore gives different summaries under the two stated conventions. This small example has no tied scores and uses an explicit endpoint convention; it does not imply that one numerical difference applies to every dataset.

Illustrative case study

A fictional inspection team compares two model reports and sees different “PR AUC” values for what appears to be the same predictions. It checks the evaluation code and finds that one report uses average precision and the other uses trapezoidal integration. The team reruns both models under the same convention, then examines precision at the recall level needed for its workflow. It also records the positive-event rate so later reports are not compared blindly after the sampled population changes.

Common misconceptions and mistakes

PR AUC is not ROC AUC, accuracy, or precision at a single threshold. Average precision and trapezoidal area should not be treated as interchangeable names. Precision depends on the proportion of positives in the evaluated population, so changes in sampling can change the curve even when class-conditional score behavior is similar. A common population-level random-ranking reference is the positive prevalence, but finite-sample estimates and specific conventions can differ. A high summary can still hide poor performance in the operating region that matters.

Business use

Use a PR summary to compare rankings on the same representative evaluation population and with the same positive-class definition. Report the calculation method, sample size, number of positives, and any averaging or weighting. Pair the summary with threshold-specific precision, recall, and expected review volume. If the evaluation set deliberately changes prevalence through sampling, explain how that affects interpretation. For deployment decisions, assess whether the system can meet the required precision and recall at feasible cost rather than relying on area alone.

Check your understanding

Why do 0.8333 and 0.7917 both arise from the worked ranking?

Answer: They use different summary rules. The first weights precision by recall gains without linear interpolation; the second integrates straight segments between the listed PR points. The evaluation must name its convention before the numbers can be compared.

Related terms

Precision
Recall
Threshold (for decision trees)
False positive
average precision
ROC AUC
threshold
class prevalence
ranking metric
interpolation
positive-class definition

Learn next

Calculate precision and recall at each distinct score threshold for a small labeled ranking. Then compute both average precision and trapezoidal PR area and inspect their difference. Study how prevalence and the operational threshold affect interpretation before using a single area value to choose a model.

Sources

Scikit-learn, average_precision_score and its distinction from trapezoidal PR area: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.average_precision_score.html Scikit-learn, precision_recall_curve: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_curve.html Scikit-learn, auc trapezoidal integration: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.auc.html

Previous: Area sampling frame Next: Argument mining