AI Glossary / Causal inference and experimentation / Beginner

Precision at k (precision@k)

Definition

Precision at k, written precision@k or P@k, is the fraction of the first k items in a ranked result list that are relevant. If three of the first five results satisfy the relevance criteria, precision@5 is 3/5, or 60%. It evaluates the quality of a specified top portion of the list. The cutoff k is part of the metric and must be stated when reporting the score.

What it means

Search and recommendation systems often produce many results, but users may inspect only the first few. Precision@k focuses evaluation on that visible or operationally important portion.

The denominator is k, not the full list length and not the total number of relevant items in the collection. The metric therefore asks how clean the top results are, rather than how much of all relevant material the system found.

Relevance depends on the task. A product might be relevant if it meets a user's stated requirements, while a document might be relevant if it answers a particular information need. Define those criteria before judging results, especially when preferences or ambiguous queries can produce disagreement.

How it works

Choose a cutoff that matches the evaluation purpose, such as the five results displayed on the first screen. Obtain a ranked list and assess the relevance of its first k items using a declared labeling process. Count the relevant items and divide by k.

For multiple queries or users, calculate the score for each and state how the scores are combined. A simple average gives each query equal weight; traffic-weighted reporting answers a different question. Use the same cutoff and relevance protocol when comparing systems.

Specify how ties, duplicate results, unjudged items, and lists shorter than k are handled. The basic definition assumes at least k returned items. If a system can return fewer, a benchmark needs an explicit convention rather than silently changing the denominator from one query to another.

Three practical examples

1. Workplace search: a team measures precision@5 because employees usually begin by scanning five suggested documents.

2. Product recommendation: reviewers assess precision@10 for a ten-item recommendation panel, using predefined requirements for whether each product is suitable.

3. Review prioritization: analysts measure precision@20 for a ranked queue when a reviewer can investigate only the first twenty cases during a work session.

Think of it this way

Think of judging a bookstore display by the first five books placed on its front shelf. Precision@5 asks how many of those five suit a customer's request. It does not assess the entire shop or distinguish where within the five each suitable book appears. The analogy is limited because relevance can differ among users and because a digital ranking's labels may be incomplete or uncertain.

Worked example

Suppose a fictional search list has binary relevance labels at ranks one through eight of: relevant, irrelevant, relevant, relevant, irrelevant, irrelevant, relevant, relevant.

The first three contain two relevant items, so precision@3 = 2/3, approximately 66.7%. The first five contain three, so precision@5 = 3/5 = 60%. The first eight contain five, so precision@8 = 5/8 = 62.5%. Precision@k need not steadily increase or decrease as k changes because each added result can be relevant or irrelevant.

Now move the relevant item at rank four to rank two while keeping the same three relevant items inside the top five. Precision@5 remains 60%, even though the early ordering improved. This reveals a limitation: P@5 counts relevance within the cutoff but does not reward one top-five ordering over another when the relevant count is unchanged. A more position-sensitive measure can complement it.

Illustrative case study

Fictional case: Birch Learning evaluates a course recommender using precision@6, matching its six-card interface. Its reviewers mark broad introductory courses relevant for nearly every query, so the score looks strong even when users ask for a specific advanced topic.

The team refines its relevance guidance to include topic, level, and prerequisites, then relabels a representative query sample. It also checks whether the six recommendations are repetitive. The case illustrates that a well-defined arithmetic metric still needs meaningful judgments and complementary checks. It does not claim a real recommender's measured performance.

Common misconceptions and mistakes

Precision@k is not average precision at k. Average-precision variants incorporate precision at multiple relevant positions and require their own normalization convention. Reporting one under the other's name can change the interpretation substantially.

Do not assume a higher cutoff means a better or worse system. Changing k changes the evaluated task. A system with strong precision@3 can have weak precision@20, and another can show the reverse pattern.

The metric also does not measure recall, novelty, diversity, or factual accuracy beyond the declared relevance label. Repeated near-duplicate results can occupy useful slots while still appearing relevant under a loose protocol. Decide how such cases should be judged and inspect them explicitly.

Business use

Precision@k is useful when a business has a limited display area or review capacity. It translates ranking performance into a question people understand: how many of the items we actually inspect are useful?

Choose k based on that workflow, not only because it produces an attractive score. Report the query population, number of judgments, and label coverage. Pair the result with measures of coverage and user task success where those matter. A top-list score should support a decision about a particular experience, rather than stand in for every aspect of the system.

People also ask

Does P@k care about ordering inside the top k?

Only through which items cross the cutoff. Rearranging the same top-k items leaves the score unchanged.

Can P@k be calculated without knowing every relevant item in the collection?

Yes. It needs relevance judgments for the evaluated top-k items, unlike a recall measure that needs a reference total of relevant items.

What if fewer than k items are returned?

State a benchmark convention or use a valid cutoff. Do not silently substitute a different denominator and present the result as directly comparable.

Check your understanding

A list contains four relevant items among its first ten results and twelve relevant items overall. What is precision@10?

Answer: It is 4/10 = 40%. The twelve relevant items in the full list are not the denominator for precision@10.

Related terms

• Precision
• Recall at k (recall@k)
• Ranking
• Mean average precision
• Normalized Discounted Cumulative Gain
• recall at k
• relevance judgment
• average precision
• retrieval evaluation

Learn next

Create two ten-item relevance lists with identical precision@10 but different early ordering. Compare precision@3 for the lists, then explain which cutoff better matches a user who examines only three results.

• Precision

More in Causal inference and experimentation

• Policy evaluation - Policy evaluation is the task of estimating how much future reward an agent can expect when it follows a specified policy.
• Positive class - The positive class is the category a classification task designates as the condition or event being detected.
• Post-sample evaluation - Post-sample evaluation assesses a forecasting model using observations from after the period on which it was estimated.
• Precision-recall curve - A precision-recall curve shows how a classifier's precision and recall change as its decision threshold varies.
• Preferential Bayesian optimization - Preferential Bayesian optimization is a sequential search method that uses comparisons between alternatives to seek a preferred solution.
• Probabilistic Programming - Probabilistic programming is an approach to expressing statistical models as programs that represent uncertainty through probability distributions.

Sources

Google, Machine Learning Glossary, precision at k: https://developers.google.com/machine-learning/glossary#precision-at-k

The primary glossary defines the top-k relevance fraction and cutoff requirement. Ranked lists, arithmetic, fictional case, and exercise are original teaching examples. Short-list and duplicate policies must be specified by the evaluation protocol.

Previous: Precision Next: Precision-recall curve