The ideal value provides a query-specific reference for the maximum achievable gain under the chosen setup.

Is NDCG@10 always higher than NDCG@5?

No. Changing the cutoff changes both numerator and denominator, so there is no general monotonicity guarantee.

Does changing all predicted scores by a strictly increasing transformation change NDCG?

Not if it preserves their ordering and tie structure.

What if every item is irrelevant?

The mathematical ratio is undefined when ideal gain is zero; software and reporting conventions must state how such queries are handled.

AI Glossary / Statistics and probability / Intermediate

Normalized Discounted Cumulative Gain

Definition

Normalized Discounted Cumulative Gain, usually abbreviated NDCG, is a ranking metric that compares the relevance achieved by a predicted ordering with the relevance achievable by an ideal ordering of the same judged items. Relevant items contribute more when placed nearer the top. Dividing discounted cumulative gain by its ideal value allows comparison on a relative scale. The gain formula, rank discount, cutoff, and treatment of ties must be specified for reproducible results.

What it means

A search or recommendation system should place useful items where users are likely to encounter them. NDCG accommodates graded relevance, such as irrelevant, useful, and highly useful, rather than requiring every item to be simply right or wrong.

The metric evaluates an ordering against supplied judgments. It does not establish that the judgments are correct, that every relevant item was included, or that the ranking satisfies every product objective. An ideal score means ideal ordering under the chosen evaluation setup, not universal perfection of the user experience.

How it works

For a common linear-gain formulation, discounted cumulative gain at cutoff K is the sum, over ranks r from 1 through K, of relevance at rank r divided by log2(r + 1). The logarithmic denominator discounts contributions at lower positions.

Sort the same candidate items by their judged relevance to compute ideal DCG at the same cutoff. Divide actual DCG by ideal DCG. With nonnegative relevance and positive ideal DCG, the score lies between zero and one. A perfect ordering achieves one. When every relevant gain is zero, the denominator is zero and an explicit convention is needed.

Other implementations use transformed gain, such as 2 raised to relevance minus one, which emphasizes large grades differently. Predicted-score ties also require a rule. Aggregate scores across queries using a declared weighting scheme, and compare systems on the same queries, judgments, candidate scope, and conventions.

Three practical examples

A search team compares how well two rankers place highly relevant documents in their first ten results.
A recommendation team evaluates ordered suggestions using graded usefulness judgments rather than only binary relevance.
An AI retrieval pipeline measures the ordering quality of candidate passages before examining whether the final generated answer uses them correctly.

Think of it this way

Think of arranging a shelf so that the most useful items are easiest to reach. NDCG rewards placing valuable items near the front and compares the arrangement with the best possible arrangement of those same items. It does not tell you whether the shelf contains all the items someone needs.

Worked example

Consider an original query with three documents whose relevance grades are 3, 2, and 1. Suppose the predicted order places grades 1, 3, and 2 at ranks one through three. Use linear gain and discount log2(r + 1), with no score ties.

DCG@3 = 1/log2(2) + 3/log2(3) + 2/log2(4), approximately 1 + 1.8928 + 1 = 3.8928. The ideal order is 3, 2, 1, giving ideal DCG@3 = 3 + 2/log2(3) + 1/2, approximately 4.7619. NDCG@3 is therefore 3.8928/4.7619, approximately 0.8175.

At cutoff two, actual DCG is approximately 2.8928 and ideal DCG is approximately 4.2619, giving NDCG@2 approximately 0.6788. The different cutoff changes the question by excluding the third rank from both sums. Using exponential rather than linear gains would produce another score, so a report must state that choice. The calculation evaluates one invented ranking; it does not estimate a deployed system's average performance.

Illustrative case study

Fictional case study: Willow Answers tests a new passage ranker for its AI assistant. The offline NDCG@5 score improves, but users still report missing important sources. Investigation shows that the candidate-retrieval stage often fails to include those sources at all.

The team separates candidate coverage from ranking quality and evaluates both. It also assesses the generated answers because a well-ordered passage list does not guarantee faithful synthesis. Willow retains NDCG as a useful component of evaluation while preventing it from becoming a substitute for the full user task. The fictional case demonstrates the importance of the metric's evaluation boundary.

Common misconceptions and mistakes

“NDCG measures factual accuracy of an answer.” It evaluates a ranking against relevance judgments, not the truthfulness of generated prose.

“Every NDCG implementation gives the same number.” Gain transforms, cutoffs, tie handling, and aggregation can differ.

“A score of one proves complete retrieval.” It can mean ideal ordering within the evaluated candidate set while important items remain absent.

“The score is always between zero and one regardless of labels.” That familiar range relies on suitable nonnegative gains and a meaningful denominator; negative relevance values require particular care.

Business use

NDCG helps businesses evaluate products where early positions matter and relevance has degrees. It can support development decisions for search, recommendations, and retrieval components of AI assistants.

Maintain stable evaluation queries and clear relevance guidelines, and examine results by meaningful query types. A higher average can conceal regressions for important users or difficult questions. Record uncertainty in system comparisons and avoid repeatedly tuning on the final test judgments. Pair the metric with checks appropriate to the product, such as candidate coverage, latency, source quality, and successful completion of the intended task.

Check your understanding

Why must the ideal ranking use the same cutoff as the predicted ranking?

Answer: They must compare gain over the same number of positions. Using a different cutoff changes the denominator's task and breaks the intended normalization.

Related terms

Precision at k (precision@k)
Recall
Information retrieval
Candidate generation
Discounted cumulative gain
relevance judgment
ranking metric
precision at K
tie handling

Learn next

Calculate DCG and NDCG for a small list using both linear and exponential gain. Then examine tied scores and all-zero judgments. Finally, compare ranking quality with candidate coverage and user-task success to understand what each evaluation measures.

Sources

scikit-learn User Guide, Normalized Discounted Cumulative Gain and linear-gain formula: https://scikit-learn.org/stable/modules/model_evaluation.html#normalized-discounted-cumulative-gain

scikit-learn, ndcg_score, implementation assumptions and ties: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.ndcg_score.html

The ranking arithmetic and Willow Answers case are original. The example explicitly uses nonnegative linear gains, base-two discount, and no ties.

Previous: Normalized cuts Next: Normalized utilities