AI Glossary / Governance, ethics and safety / Intermediate
Incompatibility of fairness metrics
Definition
Incompatibility of fairness metrics means that some mathematical fairness conditions cannot all be satisfied simultaneously under particular data and prediction conditions. Different metrics capture different concerns, such as equal selection rates, equal error rates, or comparable meaning of a prediction across groups. Meeting one condition can therefore leave another unmet, even with correct calculations and implementation.
What it means
Fairness metrics answer different questions. Demographic parity compares selection rates across groups. Equalized odds compares true-positive and false-positive rates conditional on the actual outcome. Predictive parity compares the proportion of correct positive predictions among those selected.
Those denominators differ. When groups have different outcome prevalence, the same error rates can produce different selection rates and different predictive values. A dashboard may therefore show conflicting results without containing a bug.
This does not mean every pair of metrics always conflicts or that fairness work is pointless. Mathematical results depend on their assumptions, and special cases exist. The practical lesson is to choose and explain criteria in relation to the actual concern rather than promise that a single model can make every possible fairness measure equal.
How it works
Start with precise definitions of the prediction, reference outcome, groups, and metrics. Examine the mathematical relationships between the rates and the assumptions under which proposed requirements would conflict.
For a binary classifier with equal true-positive rate t and false-positive rate f across groups, a group's selection rate is t × p + f × (1 − p), where p is its positive-outcome prevalence. If p differs and t differs from f, selection rates must differ. Equalized odds and demographic parity then cannot both hold under those conditions.
Real assessments also involve imperfect labels, finite samples, and uncertainty. A small observed difference might reflect sampling variation; a structural conflict is a different issue. Separate those questions before changing thresholds or training objectives, and evaluate the consequences of any chosen compromise.
Three practical examples
1. A service achieves equal error rates across two groups but finds that the proportion selected differs because the reference outcomes have different prevalence.
2. A team adjusts thresholds to bring selection rates closer together, then observes changes in the false-positive and false-negative rates that matter to users.
3. A project specification asks for several parity conditions at once. Reviewers check whether the conditions can coexist under the data assumptions before promising that optimization will satisfy them all.
Think of it this way
Think of dividing two differently composed baskets while trying to match both the fraction of each fruit type taken and the total number of items taken. Matching one set of proportions can prevent another total from matching.
The analogy is limited because people and social consequences cannot be reduced to baskets of fruit. It explains the arithmetic of competing constraints, not which constraint is ethically preferable. That choice requires understanding what the labels represent, what decisions follow, and who experiences the consequences.
Worked example
Consider two invented groups of 100 cases each. Group A contains 20 actual positives and 80 negatives; group B contains 60 positives and 40 negatives. The classifier has an 80% true-positive rate and a 10% false-positive rate in both groups.
For A, it selects 16 true positives and eight false positives: 24 cases. For B, it selects 48 true positives and four false positives: 52 cases. Equalized odds holds in these counts because both conditional rates match.
Demographic parity does not hold: selection rates are 24% and 52%. Predictive parity also differs: 16/24 is about 66.7%, while 48/52 is about 92.3%.
The example demonstrates a conflict for these particular conditions. It does not justify the difference in underlying prevalence or establish that the labels are valid. Those are separate questions. Nor does it prove that every fairness criterion is incompatible in every possible dataset.
Illustrative case study
Fictional case: Ridgeway Services evaluates a model used to identify requests needing additional assistance. Its governance team initially asks for identical selection rates, identical error rates, and identical positive predictive values across groups.
The analysts explain the different denominators using a transparent count table and investigate what each criterion would mean for missed needs and unnecessary review. They also examine whether the reference labels adequately capture actual need.
The team revises the specification to document its priorities and remaining tradeoffs rather than call one dashboard score a complete fairness verdict. This case illustrates a clearer decision process, not a recommended universal metric or a finding about any real organization.
Common misconceptions and mistakes
“The metrics conflict, so nothing can improve” is an overreaction. Better data, a better task definition, or a changed workflow can still reduce harmful effects.
“One failed parity test proves the whole system is unfair” also skips the contextual analysis. The metric's meaning and the consequences need examination.
Do not invoke an impossibility theorem without checking its assumptions. A result concerning calibrated risk scores is not automatically a result about every binary decision rule. Likewise, approximate equality in a small sample does not establish exact population-level compatibility.
Business use
Businesses should require explicit fairness objectives and explanations of tradeoffs before treating a model as ready for a consequential workflow. Ask which harm each measure addresses, which groups and outcomes are included, and what evidence supports the labels.
Present relevant metrics together with counts and uncertainty. A single favorable number can conceal a deterioration elsewhere. When objectives conflict, the decision belongs within accountable governance, informed by technical and affected-party perspectives. Optimization software can explore feasible choices, but it cannot decide the organization's responsibilities by itself.
People also ask
Are all fairness metrics mutually incompatible?
No. Conflicts apply to particular combinations under specified conditions; special cases can permit several conditions together.
Does improving accuracy always remove the conflict?
No. Better prediction can help, but it does not automatically resolve every relationship between metrics or every social concern.
Which metric should a project choose?
One or more criteria justified by the task, relevant harms, affected people, and applicable requirements, with limitations documented.
Check your understanding
Two groups have matching true-positive and false-positive rates but different selection rates. Is that necessarily a calculation error?
Answer: No. Different outcome prevalence can produce that pattern. Check the count table and definitions before concluding the implementation is wrong.
Related terms
- Demographic parity
- Equalized odds
- Predictive parity
- Calibration
- Base rate
- Fairness constraint
- Confusion matrix
- Harmful bias
Learn next
Study confusion matrices and calibration next. Construct two group-specific count tables, calculate several metrics using their correct denominators, and explain which practical question each result answers.
More in Governance, ethics and safety
- In-processing - In-processing is a family of fairness-mitigation approaches that change how a machine-learning model is trained.
- Incident - An incident is an event or connected set of events that meets an organization's or framework's criteria for requiring investigation and response because...
- Incident response - Incident response is the coordinated work of investigating an incident, limiting its effects, addressing the problem, and restoring an acceptable operating...
- Independence - Independence, in probability, means that knowing one event or variable does not change the probability information about another.
- Individual Bias - Individual Bias is a persistent personal viewpoint, or a narrow set of viewpoints, that influences how someone considers information and makes judgments.
- Individual fairness - Individual fairness is the principle that people who are similar in ways relevant to a particular task should receive similar treatment from a decision system.
Sources
Google, Machine Learning Glossary, incompatibility of fairness metrics: https://developers.google.com/machine-learning/glossary#incompatibility-of-fairness-metrics
Fairlearn, Common fairness metrics: https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html
Kleinberg, Mullainathan, and Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores: https://arxiv.org/abs/1609.05807
Definitions and the original paper's abstract were read. The binary-classifier arithmetic is an original derivation, not a reproduction of the paper's distinct risk-score theorem.
Previous: Inclusion-exclusion principle Next: Incomplete join