Is balance the same as demographic parity?

No. Demographic parity compares positive prediction rates, while this criterion compares errors conditional on the reference outcome.

Does it guarantee equal precision?

No. Precision can differ when outcome prevalence differs, even with matching false-positive and false-negative rates.

Can a model satisfy all fairness criteria simultaneously?

Not generally. Some criteria conflict under common conditions, so the relevant goals and assumptions must be discussed explicitly.

AI Glossary / Graphs and symbolic AI / Intermediate

Balance

Definition

Balance, in the fairness sense used by the source glossary, requires a classifier to have approximately equal false-positive rates and false-negative rates across the groups being compared. It is an error-rate parity criterion, rather than a general guarantee that an AI system is fair.

What it means

A classifier can make two broad kinds of mistake. It can predict a positive outcome when the reference label is negative, or miss a positive case. Balance asks whether each type of mistake occurs at comparable rates across groups, using the appropriate denominators.

Equal false-negative rates are equivalent to equal true-positive rates for an ordinary binary classifier. Together with equal false-positive rates, this corresponds to the familiar equalized-odds criterion. It does not require the false-positive rate to equal the false-negative rate within each group.

Terminology varies. Research on risk scores also uses “balance for the positive class” and “balance for the negative class” for conditions on average scores. Those should not be silently substituted for the thresholded error-rate meaning used here. Neither meaning refers to simply balancing training-set class counts.

How it works

Start by defining the prediction, reference outcome, groups and decision threshold. For each group, count true positives, false positives, true negatives and false negatives. The false-positive rate divides false positives by all actual negatives; the false-negative rate divides false negatives by all actual positives.

Compare each rate across groups, reporting the size of the differences and the supporting sample counts. Exact equality in a finite sample is not usually a realistic universal target, so any tolerance should be stated and justified in context. Small samples require particular care about uncertainty.

The assessment also depends on label quality and who appears in the data. A reference label may itself reflect unequal observation or an unsuitable proxy. Matching error rates against such labels cannot settle every fairness concern. Changes to thresholds or training may alter several metrics at once, requiring explicit consideration of the resulting tradeoffs.

Three practical examples

A document-screening service compares missed relevant documents and incorrectly flagged irrelevant documents across supported languages.
An accessibility team examines both missed voice commands and unintended activations across evaluated user groups.
A service-quality review compares a binary assistance-request classifier across channels while checking whether the reference labels mean the same thing in each channel.

Think of it this way

Think of comparing two inspection desks by looking separately at items wrongly rejected and problems wrongly missed. A single total-error count can hide an imbalance in either direction.

The analogy has limits. The reference judgment may be imperfect, and equal error rates do not make every consequence equal. A missed request can have different effects depending on the person's circumstances and the available alternatives.

Worked example

Equal rates with unequal error counts

In an invented evaluation, Group A has 100 actual positive cases and 100 actual negative cases. The classifier misses 20 positives and incorrectly flags 10 negatives. Its false-negative rate is 20/100 = 20%, and its false-positive rate is 10/100 = 10%.

Group B has 50 actual positives and 200 actual negatives. It misses 10 positives and incorrectly flags 20 negatives. Its false-negative rate is 10/50 = 20%, and its false-positive rate is 20/200 = 10%.

The groups satisfy the stated rate parity in this sample even though their error counts differ. Their positive prediction rates also differ: A receives 90 positive predictions out of 200 cases, or 45%, while B receives 60 out of 250, or 24%. Error-rate parity is therefore not the same as equal selection rates.

Illustrative case study

Fictional case: Birch Assistance evaluates a classifier that identifies messages needing a human reply. Its overall accuracy looks similar across two language groups, but one group experiences many more missed requests and fewer unnecessary referrals.

The team reports the two error rates separately and reviews ambiguous labels with fluent reviewers. It tests changes using fresh examples and measures the resulting workload as well as missed assistance. It avoids declaring the system universally fair merely because one pair of rates becomes closer.

The lesson is that a selected fairness criterion should clarify a specific concern. It should not hide the evidence quality, practical consequences or other affected groups behind a single reassuring label.

Common misconceptions and mistakes

Balance means equal numbers of mistakes. Rates use different denominators and can match despite different counts.
It requires false positives and false negatives to have the same rate. Each rate is compared across groups, not necessarily against the other rate.
Equal accuracy establishes balance. Opposing error patterns can produce similar overall accuracy.
Meeting the criterion resolves every fairness question. Labels, access, individual circumstances and downstream effects still matter.
Every paper uses “balance” identically. Score-based and binary-decision definitions must be distinguished.

Business use

This criterion helps teams examine whether different groups face different patterns of classification error. It is particularly useful when both unnecessary interventions and missed assistance have meaningful consequences.

Document why the criterion fits the application, how groups and outcomes were defined and which uncertainties remain. Review the operational effects of proposed changes instead of optimizing a metric in isolation. Keep an accessible record of both rates and sample sizes so that a small or poorly measured subgroup does not disappear inside an aggregate result.

Check your understanding

Two groups each have 15 false positives. One has 50 actual negatives and the other has 150. Are their false-positive rates balanced?

Answer: No. The rates are 30% and 10%. Equal counts do not establish equal rates.

Related terms

Learn next

Build two small confusion matrices and calculate both error rates and the positive prediction rates. Adjust the group sizes while keeping error rates constant to see which other metrics change. Study equalized odds and calibration next, including their assumptions about the reference outcome.

Sources

AIMA glossary, balance, supplies the source-list error-rate meaning: https://github.com/aimacode/aima-glossary/blob/master/glossary.md?plain=1

Fairlearn, Common fairness metrics, explains equalized odds and assessment assumptions: https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html

Kleinberg, Mullainathan and Raghavan, Inherent Trade-Offs in the Fair Determination of Risk Scores, provides the distinct score-based terminology: https://arxiv.org/abs/1609.05807

Previous: Bagging Next: Balanced accuracy score