Does Simpson's paradox mean the data must contain an error?

No. Correctly measured and calculated data can exhibit the reversal.

Should we always split data into more groups?

No. Grouping can clarify composition, but inappropriate conditioning can distort a causal analysis.

Can it affect AI model comparisons?

Yes, when models or systems are evaluated on different mixtures of cases and aggregate scores conceal that difference.

AI Glossary / Statistics and probability / Intermediate

Simpson's paradox

Definition

Simpson's paradox is a statistical phenomenon in which an association observed within separate groups reverses when the groups are combined. For example, one system can have a higher success rate in each difficulty category but a lower overall rate because it handles a different mixture of cases. The reversal is mathematically consistent; the apparent contradiction comes from comparing differently weighted populations.

What it means

An overall percentage is calculated from total successes divided by total opportunities. It is a weighted average of the group-specific percentages, with weights determined by how many observations fall into each group.
When the compared systems have different group proportions, their overall rates use different weights. A system serving mainly difficult cases can look worse overall despite outperforming another system within every difficulty group. Both descriptions can be correct summaries of the observed data. Deciding what they imply for a policy or causal claim requires additional information about how the data arose.

How it works

Identify the outcome, comparison variable, and grouping variable. Calculate group-specific counts and rates, then aggregate the counts and recompute the overall rates. Inspect whether differences in group composition explain the reversal.
For a comparison at a common case mix, apply the same chosen group weights to both systems. That produces a standardized descriptive comparison. It does not automatically identify a causal effect. Pearl's analysis emphasizes that whether to condition on a variable depends on the causal structure and the question; more detailed grouping is not always the right adjustment.

Three practical examples

An AI evaluation team can investigate why a model wins within each tested difficulty category but loses on an overall benchmark assembled with unequal case mixes.
A support manager can distinguish changes in resolution performance from changes in the proportion of complex tickets handled by each team.
A learning-platform analyst can examine whether apparent completion-rate differences reflect which courses or learner populations enter each comparison.

Think of it this way

Think of comparing two delivery teams when one receives mostly short routes and the other mostly demanding routes. The overall on-time score mixes route difficulty with team performance. The analogy helps explain weighting, but it does not establish that route categories capture every relevant difference or that assigning a route to a different team would cause the same observed result.

Worked example

Consider an original fictional evaluation of systems A and B on easy and hard cases. On easy cases, A succeeds in 9 of 10 attempts, giving 90 percent. B succeeds in 72 of 90 attempts, giving 80 percent. On hard cases, A succeeds in 18 of 90 attempts, giving 20 percent. B succeeds in 1 of 10 attempts, giving 10 percent. A has the higher observed success rate in both categories.
Combine the counts. A has 9 + 18 = 27 successes out of 100 attempts, or 27 percent. B has 72 + 1 = 73 successes out of 100 attempts, or 73 percent. B now appears better overall. The reversal occurs because 90 percent of A's cases are hard, while 90 percent of B's are easy.
The arithmetic can be written directly as weighted averages. A's aggregate rate is 0.1 times 0.9 plus 0.9 times 0.2, which equals 0.27. B's is 0.9 times 0.8 plus 0.1 times 0.1, which equals 0.73. These are different mixtures, so there is no rule requiring their order to match the within-category comparisons.
For an illustrative common mixture of half easy and half hard cases, A's standardized rate is (0.9 + 0.2)/2 = 0.55. B's is (0.8 + 0.1)/2 = 0.45. At those shared weights, A leads by ten percentage points. Other shared mixtures also preserve A's ten-point advantage here because the within-category differences are identical.
This standardization answers a specified descriptive comparison. The small invented samples do not establish precise population rates or a causal advantage. A real evaluation would need uncertainty estimates, comparable cases, and a justified design. It should also investigate why the original case assignments differed.

Illustrative case study

Fictional case: a service organization initially ranks two support assistants using their overall resolution rates. One assistant receives most of the escalated cases, while the other receives mostly routine questions. A breakdown reverses the apparent ordering.
The team publishes both the observed workload totals and performance on a common evaluation set. It records the routing policy and considers a controlled comparison before changing assignments. This preserves the operational fact that the assistants handled different workloads while avoiding an unsupported conclusion about which assistant causes better outcomes.

Common misconceptions and mistakes

Do not assume the aggregate result is always wrong or that the subgroup result is always right. Conditioning on an unsuitable variable can itself introduce bias.
The phenomenon is not simply averaging percentages incorrectly. It can occur when all calculations are correct. Nor does every difference between aggregate and subgroup statistics qualify as a reversal. Report the actual pattern, including denominators, rather than using the phrase Simpson's paradox as a general label for any surprising result.

Business use

For business reporting, separate the question of actual total outcomes from the question of comparable performance. A dashboard can show workload composition, within-group rates, and an explicitly standardized measure together.
Choose grouping variables and reference weights for a reason connected to the decision. For causal decisions, document assignment mechanisms and relevant assumptions rather than selecting whichever table favors a preferred conclusion. A reversal is a signal to investigate composition and causality, not a license to discard inconvenient observations.

Check your understanding

In the worked example, why does B have the higher overall rate despite losing in both categories?

Answer: B handles mostly easy cases. Its aggregate gives much more weight to the category with high success rates, while A's aggregate is dominated by hard cases.

Related terms

Learn next

Recalculate both standardized rates for a common workload containing 80 percent easy cases and 20 percent hard cases. Then explain why the original observed totals remain valid descriptions of the original workloads even though they answer a different question from the standardized comparison.

Sources

Primary research source: Judea Pearl, Understanding Simpson's Paradox, technical report R-414: https://ftp.cs.ucla.edu/pub/stat_ser/r414.pdf . The two-system example, common-mixture calculations, and fictional support case are original. Source discussion of causal conditioning informs the distinction between descriptive reversal and intervention claims.

Previous: Simplex algorithm Next: Simulated annealing