Is group fairness identical to demographic parity?

Some sources use the terms that way. In broader usage, group fairness includes several criteria, so the exact definition must be stated.

Can group and individual fairness differ?

Yes. Equal group-level rates do not ensure that similar individuals receive similar treatment, and an individual-focused criterion may not equalize group outcomes.

Should every group have identical outcomes?

That is a substantive question about the task and fairness criterion. The analysis must justify what is being compared and why, rather than impose equality on every possible statistic.

AI Glossary / Governance, ethics and safety / Intermediate

Group fairness

Definition

Group fairness assesses how an AI system's decisions, errors, or benefits compare across defined groups of people. It is commonly expressed through statistical criteria, such as comparable selection rates or comparable error rates. The term can be used broadly for this family of assessments, while some sources use it more narrowly as a synonym for demographic parity. A report should state the exact criterion instead of relying on the label alone.

What it means

A system can perform well overall while serving groups differently. Group-fairness analysis makes differences visible by calculating and comparing relevant quantities for each group.

The comparison might concern access to an opportunity, the rate of missed eligible cases, or the burden of incorrect alerts. Each concern leads to a different denominator and potentially a different assessment of the same decisions.

Groups can be defined using attributes relevant to the fairness question, including intersections of attributes. These categories do not imply that all members have identical needs or experiences. Group-level analysis complements rather than replaces attention to individuals, context, and the wider decision process.

How it works

Identify the decision being assessed, the population affected, and the harm or benefit of concern. Define the groups and outcome measures, checking that the data support the intended comparison and that missing records are understood.

Calculate group-specific rates and compare them using a stated difference, ratio, or other criterion. Demographic parity compares selection rates. Equalized odds compares true-positive and false-positive rates conditional on reference labels. Those reference labels must themselves be appropriate for the task.

Interpret the comparison alongside group sample sizes, uncertainty, and the decision workflow. Investigate disparities and inspect relevant intersections where feasible. Meeting one statistical condition does not establish that all fairness concerns have been addressed or that a model's effects remain unchanged after deployment.

Three practical examples

1. A service team compares successful completion rates across supported language groups to identify whether some users encounter more difficulty.

2. An evaluation team compares incorrect-alert rates across relevant groups because unnecessary alerts impose a review burden on affected users.

3. An allocation review examines both overall group outcomes and intersections, such as language and accessibility needs, when the available data allow meaningful analysis.

Think of it this way

Think of comparing scorecards for teams in a service process instead of looking at the combined total. A strong overall result can conceal a weaker result for one group.

The analogy has limits. People are not competing teams, and fairness is not simply equal scores on any chosen measure. The scorecard must reflect the actual concern, and equal averages can still conceal poor treatment of particular individuals or differences within each group.

Worked example

Suppose a fictional system selects 40 of 100 people in group A and 20 of 100 in group B for an optional service. The selection rates are 40% and 20%, giving a difference of 20 percentage points and a lower-to-higher ratio of 0.5.

Now divide group A into two equally sized subgroups: A1 has 30 selections out of 50, and A2 has 10 out of 50. Their rates are 60% and 20%, which average to the group's 40% because the subgroup sizes are equal.

The combined group figure does not describe every subgroup's experience. These invented counts illustrate aggregation and rate calculations, not a complete fairness judgment. Whether selection is beneficial, how eligibility is defined, why differences occur, and how uncertain the estimates are all remain relevant. No ratio here is offered as a legal compliance threshold.

Illustrative case study

Fictional case: Elmbridge Learning evaluates an AI help service. Overall task success appears satisfactory, but success rates differ across the languages the product officially supports.

The team checks the evaluation examples, user-interface translations, and the model's responses for each language. It also reviews whether the tasks have comparable difficulty and whether important failures were excluded from the logs. Findings are used to prioritize investigation and improvement.

This fictional case shows group analysis as a diagnostic step. It does not attribute every gap to the model or assume that translating the interface alone will produce equivalent service quality.

Common misconceptions and mistakes

Do not treat group fairness as one universally agreed equation. State the specific condition and why it fits the concern.

Do not assume matching group rates guarantees fair treatment of every individual. A system can satisfy an aggregate condition while making arbitrary or harmful decisions within groups.

Avoid assuming that removing sensitive attributes from model inputs eliminates disparities. Other features can carry related information, and the surrounding process can still produce unequal outcomes. Also avoid overinterpreting tiny subgroup samples or presenting noisy point estimates without enough context to understand their limitations.

Business use

Group-fairness assessment helps organizations identify uneven service and evaluate whether proposed changes address the intended concern. It can inform product design, monitoring, and review discussions when used with clear definitions.

Document the group categories, outcome meaning, data sources, and reasons for choosing each measure. Review changes over time and after workflow updates. Include evidence from affected people and domain experts, especially where a statistical label fails to capture the practical consequences of a decision. An improving metric should be connected to an improving experience, not assumed to imply it.

Check your understanding

Two broad groups have the same average success rate. Does that rule out a serious disparity in a smaller intersectional subgroup?

Answer: No. Aggregation can conceal differences within groups. Where the data support it, examine relevant intersections and report uncertainty rather than relying solely on broad averages.

Related terms

Learn next

Learn demographic parity and equalized odds next, then calculate subgroup and overall rates from one dataset. Follow with individual fairness to understand why assessing groups and assessing similarly situated people answer related but distinct questions.

Sources

Fairlearn primary documentation, common fairness metrics and group comparisons: https://fairlearn.org/main/user_guide/assessment/common_fairness_metrics.html

Google Machine Learning Glossary, fairness metrics and individual fairness: https://developers.google.com/machine-learning/glossary

Primary definitions were read, including the narrower use of group fairness as demographic parity. NIST's supplied glossary spreadsheet did not expose its definition text through the browser. The counts, workflow guidance, and Elmbridge Learning case are original teaching illustrations.

Previous: Group concept mapping Next: Group method of data handling