Does J depend directly on the positive-class proportion?

Its formula uses conditional rates rather than that proportion, although population changes can alter those rates.

Is the highest-J threshold always best?

No. It is best for that particular criterion on the data considered, not automatically for costs or capacity.

Can several thresholds have the same J?

Yes. A tie requires a stated selection rule or additional decision considerations.

AI Glossary / Statistics and probability / Intermediate

Youden's J statistic

Definition

Youden's J statistic summarizes a binary decision rule as sensitivity plus specificity minus one. Equivalently, it is the true-positive rate minus the false-positive rate. At a specified threshold it measures separation between the two classes; the maximum J across thresholds is also commonly called the Youden index.

What it means

A classifier can improve sensitivity by flagging more cases, but this may also increase false alarms. J combines the two class-specific success rates into one number, giving sensitivity and specificity equal weight. It is related to balanced accuracy by J = 2 × balanced accuracy - 1.

For a fixed decision direction, J can range from -1 to 1. A value of 1 represents perfect separation, while zero means equal true-positive and false-positive rates at that operating point. A negative value indicates worse-than-diagonal performance for that direction. A zero at one threshold does not prove that the underlying score contains no useful information elsewhere.

How it works

Start with independently assessed reference labels and predictions at a defined threshold. Calculate sensitivity as true positives divided by all actual positives, and specificity as true negatives divided by all actual negatives. Both denominators must be meaningful and nonzero. Add the two proportions and subtract one.

To select a threshold using J, evaluate candidate thresholds and identify the largest value. On an ROC plot this maximizes vertical distance above the diagonal. Multiple thresholds can tie. Threshold selection should occur within an appropriate development procedure; evaluation on separate data then estimates performance without reusing the same observations to select and endorse the rule. Uncertainty in both rates carries into J.

Three practical examples

1. Document review: A team evaluates a classifier that identifies records needing specialist review. J summarizes its sensitivity and specificity while a separate workload analysis counts the expected referrals.

2. Equipment monitoring: Engineers compare thresholds for detecting known fault episodes. They examine J alongside the practical consequences of missed faults and unnecessary inspections.

3. Model comparison: Analysts report J for two classifiers on the same labeled evaluation set. They also show the confusion matrices so readers can see which class-specific trade-offs produced the scores.

Think of it this way

Think of two equally weighted report cards: one for finding positives and another for correctly clearing negatives. J combines them and shifts the scale so that matching the ROC diagonal scores zero. Equal weighting is a deliberate scoring choice; it does not tell an organization that every missed positive costs the same as every false alarm.

Worked example

In an original example, a test set contains 100 positive and 200 negative documents. At a chosen threshold the classifier finds 80 positives and correctly clears 180 negatives. Therefore sensitivity is 80 / 100 = 0.80 and specificity is 180 / 200 = 0.90.

J = 0.80 + 0.90 - 1 = 0.70. Balanced accuracy is 0.85, and ordinary accuracy is 260 / 300, approximately 86.67%. These are distinct quantities.

Now construct another illustrative set with 100 positives and 1,000 negatives, keeping those conditional success rates unchanged: 80 true positives and 900 true negatives. J remains 0.70, but ordinary accuracy becomes 980 / 1,100, approximately 89.09%. Precision changes from 80 / 100 = 80% in the first set to 80 / 180, approximately 44.44%, in the second. The same J can therefore accompany a very different fraction of useful alerts.

Illustrative case study

Fictional case: Harbor Records develops an AI system that flags potentially incomplete supplier files. Its analysts initially choose the validation threshold with the highest J. The operations manager then calculates how many flagged files reviewers would receive each week and how many incomplete files might remain undiscovered.

The preferred operational threshold differs because review capacity is limited and the consequences of the two errors are unequal. The team retains J as a model-evaluation summary but documents the decision rule separately. It checks performance on a held-out sample and repeats the workload calculation using the expected deployment mix rather than the enriched development sample.

Common misconceptions and mistakes

J is not ordinary accuracy, precision, probability calibration or the area under the ROC curve. Each answers a different question. A high J does not imply that most positive alerts are correct when the positive condition is rare.

Equal weighting of sensitivity and specificity is not generally equivalent to equal monetary cost per error. Although J has no explicit prevalence term, its estimated rates can change when the evaluated population or case difficulty changes. Maximizing J on a small dataset can overfit the threshold, and reporting only the maximum conceals that selection process and its uncertainty.

Business use

J provides a compact way to compare binary classifiers when both class-specific rates deserve attention. It can be particularly informative alongside ordinary accuracy when the class counts are very unequal.

A business should still define the actual action taken after a positive result, the cost of review, the consequence of missing a case and the expected frequency of positives. Those quantities determine whether a threshold is useful in practice. Report sensitivity and specificity beside J, identify the evaluation population and distinguish measured performance from assumptions about deployment.

Check your understanding

What is J when sensitivity is 0.80 and specificity is 0.90?

Answer: It is 0.70.

What balanced accuracy corresponds to that J?

Answer: It is (0.70 + 1) / 2 = 0.85.

Why did precision change in the example?

Answer: The number of negative cases increased while the conditional error rates stayed fixed.

Related terms

Confusion matrix
ROC (receiver operating characteristic) Curve
Precision
sensitivity
specificity
ROC curve
balanced accuracy
decision threshold
class imbalance

Learn next

Learn next how deployment prevalence changes predictive values and how explicit error costs influence threshold choice. Then study confidence intervals and nested validation for thresholds selected from data.

Sources

pROC authors, coords documentation: https://search.r-project.org/CRAN/refmans/pROC/html/coords.html . Smits, A note on Youden's J and its cost ratio (2010): https://link.springer.com/article/10.1186/1471-2288-10-89 . The documentation's unshifted optimization score has the same maximizing threshold as conventional J. The calculations here are original teaching examples. Smits was consulted through the indexed author abstract; direct full-text retrieval failed.

Previous: You Only Look Once Next: Z-score