Can equalized odds hold when selection rates differ?

Yes. Different proportions of actual positive cases can produce different overall selection rates despite equal conditional rates.

Does it imply equality of opportunity?

In the binary setting, yes, because its true-positive-rate requirement includes that criterion.

Can it coexist with unequal precision?

Yes, as the worked example shows. Different fairness measures can conflict, so the choice requires context and cannot be settled by naming one metric.

AI Glossary / Governance, ethics and safety / Intermediate

Equalized odds

Definition

Equalized odds is a statistical fairness criterion requiring a model's predictions to be independent of group membership once the actual target outcome is held fixed. For a binary classifier, this means matching both true-positive rates and false-positive rates across the groups being compared. It describes a particular relationship among predictions, labels, and groups; it does not by itself establish that a whole decision system is fair.

What it means

The criterion separates actual positive cases from actual negative cases. Among positives, groups should be recognized at equal rates. Among negatives, groups should be incorrectly labeled positive at equal rates. This differs from demographic parity, which compares overall selection rates without conditioning on the target label. Equalized odds therefore relies heavily on the meaning and quality of that label. If labels capture unequal past treatment or are observed only for selected people, the comparison can inherit those limitations even when the arithmetic is correct.

How it works

Construct a confusion matrix for each group. Divide true positives by all actual positives to obtain the true-positive rate. Divide false positives by all actual negatives to obtain the false-positive rate. Compare both sets of rates, reporting sample counts and uncertainty. An exact definition applies to population probabilities, while practical assessments estimate them and should state an acceptable tolerance. Training constraints or post-processing can seek closer rates, sometimes with tradeoffs in other measures. The suitability of a mitigation depends on the decision context, available information, and applicable rules, not merely on reaching a numerical target.

Three practical examples

A support-triage team compares both missed urgent requests and unnecessary urgent flags across relevant groups, using independently reviewed urgency labels.
A content-classification evaluation checks correct detections and incorrect flags separately across language groups rather than reporting only average accuracy.
A product-inspection model is assessed for matched detection and false-alarm rates across manufacturing sites. This applies the statistical pattern to operational groups without assuming that every fairness question has the same social meaning.

Think of it this way

Think of two alarm systems serving separate areas. Matching how often they detect real incidents is only half the comparison; matching how often they sound when no incident exists is the other half. The analogy is limited because real AI decisions may involve contested labels and unequal consequences. It helps explain why equalized odds places two conditions on binary predictions rather than just one.

Worked example

Suppose Group A has eighty actual positive cases and twenty actual negative cases. The model identifies sixty-four positives and incorrectly flags four negatives. Its true-positive rate is 64/80 = 80 percent, and its false-positive rate is 4/20 = 20 percent. Group B has twenty actual positives and eighty negatives. The model identifies sixteen positives and incorrectly flags sixteen negatives, producing the same rates: 16/20 = 80 percent and 16/80 = 20 percent. Equalized odds holds on these sample rates. Yet the overall positive-prediction rates are 68/100 = 68 percent for A and 32/100 = 32 percent for B. Precision also differs: 64/68 is about 94.1 percent for A, while 16/32 is 50 percent for B. Different underlying positive-label proportions explain why matching conditional rates does not force matching selection rates or precision. These are hypothetical counts, not a fairness verdict.

Illustrative case study

Fictional case: Cedar Moderation evaluates an AI flagging tool. An initial report shows similar recall across two language groups, but a fuller review finds unequal false-positive rates. The team recognizes that equality of opportunity alone did not establish equalized odds. It investigates labeling consistency and evaluates candidate changes on fresh reviewed examples, including the impact of unnecessary flags. The case illustrates the value of separating error types without asserting that a particular threshold adjustment is legally or operationally appropriate.

Common misconceptions and mistakes

Equalized odds is not equal accuracy, equal precision, or equal selection rates. Those quantities use different denominators and can behave differently when target-label prevalence varies. Equality of opportunity is a weaker binary criterion because it constrains only true-positive rates. Another mistake is viewing label-based parity as a solution to biased labels: the criterion takes the target as given and does not validate it. Matching rates by making a useful model worse is not automatically an acceptable intervention. The effects on people and the purpose of the decision remain central.

Business use

Explain why both error types matter for the workflow and whether the chosen labels support the comparison. Report the two conditional rates separately alongside group counts, uncertainty, and operational consequences. Assess relevant intersections where data supports meaningful analysis, without hiding uncertainty in small groups. Evaluate potential changes using held-out evidence and compare them with other relevant requirements such as calibration and accessibility. Record the rationale for choosing this criterion rather than presenting it as the unique mathematical definition of fairness. Monitor deployed decisions as well as model predictions, since human actions can change the final distribution of outcomes.

Check your understanding

Two groups have equal true-positive rates but false-positive rates of 5 percent and 20 percent. Does the classifier satisfy binary equalized odds?

Answer: No. Both true-positive and false-positive rates must match. The first equality alone satisfies only the corresponding equality-of-opportunity condition.

Related terms

Learn next

Use two group-specific confusion matrices to calculate true-positive rate, false-positive rate, precision, and selection rate. Identify which equalities hold and which do not. Then explain which differences matter for the actual decision and what the labels may fail to capture.

Sources

Hardt, Price, and Srebro, Equality of Opportunity in Supervised Learning, original equalized-odds definition: https://www.cs.utexas.edu/~ecprice/papers/fairness.pdf Fairlearn, Common fairness metrics: https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html Fairlearn, Performing a Fairness Assessment: https://fairlearn.org/main/user_guide/assessment/perform_fairness_assessment.html

Previous: Equality symbol Next: Equally weighted means