Should every group have identical counts?

Not necessarily; the appropriate design depends on population, objectives, and precision requirements.

Is oversampling always enough?

No; repeated or synthetic cases may not add missing real-world variation.

Can disparity occur in evaluation even when training is balanced?

Yes; the datasets serve different purposes and may have different compositions.

AI Glossary / Statistics and probability / Intermediate

Sample size disparity

Definition

Sample size disparity is a difference in the number of available observations across groups. In AI fairness discussions, it commonly refers to having substantially fewer training or evaluation examples for some demographic or other relevant subpopulations than for others. This can affect what a model learns and how reliably its performance can be assessed, without being the sole possible cause of unequal outcomes.

What it means

A model trained mostly on one group may receive limited evidence about another group's variation. An overall performance score can also be dominated by the larger group, obscuring difficulties experienced by a smaller one.
Unequal counts are not automatically proof of discrimination or a flawed sampling design. They may reflect population proportions, deliberate sampling, access limitations, or selection bias. The important questions are whether the available evidence supports the intended use and whether performance and uncertainty are acceptable across relevant groups.

How it works

Define the groups and the purpose of the analysis. Groups might concern language, region, device type, or demographic characteristics relevant to evaluating impacts. Use appropriate governance for sensitive data and avoid treating a group label as a complete explanation of individual variation.
Count examples separately in training, validation, and test data. A large training group can still have an inadequately sized evaluation group, and the reverse can occur. Distinguish distinct people or entities from repeated records.
Look beyond marginal counts. A subgroup may be large overall but have very few positive labels, rare cases, or examples at an intersection of characteristics. Those conditional counts can matter more for a particular error rate than the headline total.
Assess learning consequences empirically. Sparse examples can make it harder to learn relevant patterns, especially when group distributions differ. However, shared structure, model choice, data quality, and task difficulty also influence performance; smaller size does not mathematically guarantee worse accuracy in every setting.
Report disaggregated performance with uncertainty. Small groups usually produce less precise estimates under comparable conditions. A noisy estimate should not be silently suppressed or presented as evidence of equality merely because a difference fails to reach significance.
Inspect aggregate weighting. Micro-averaged performance weights groups according to their case counts; a macro average gives each specified group equal influence. They answer different questions. Neither aggregation alone resolves the normative question of which errors or harms matter.
Consider targeted data collection, improved coverage, careful weighting, or resampling when justified. Duplicating examples changes their training influence but does not create new independent information. Synthetic data require validation for quality and coverage rather than an assumption that equal row counts establish fairness.
Evaluate remedies on independently collected or properly separated data. Equalizing counts can leave label errors, measurement differences, distribution shifts, or unsuitable decision thresholds unresolved. Track the intended outcome and possible tradeoffs rather than using numerical balance as the only success criterion.

Three practical examples

Speech recognition: a team checks whether some language varieties have too few examples for reliable evaluation.
Document processing: analysts find that rare document types contribute little to the overall accuracy metric.
Customer-facing AI: researchers examine performance by relevant user groups and by intersections where data may be particularly sparse.

Think of it this way

Think of learning a city's roads from trips concentrated in one district. Many trips provide detailed knowledge of that district but less information about another. Repeating the same route more often does not necessarily teach the missing routes. Likewise, a large total dataset can coexist with sparse evidence for important groups.

Worked example

A fictional test set contains 900 cases from group A and 100 from group B. The model makes 60 errors in A and 20 in B. The group error rates are 60/900, approximately 6.67%, and 20/100 = 20%.
The pooled error rate is 80/1,000 = 8%, which is close to the larger group's result. An equal-group average is (6.67%+20%)/2, approximately 13.33%. Neither calculation changes the underlying predictions; they weight the groups differently.
For a separate precision illustration, suppose both groups had a true independent Bernoulli error probability of 10%. The standard errors of their estimated rates would be sqrt(0.1×0.9/900) = 1 percentage point and sqrt(0.1×0.9/100) = 3 percentage points.
The smaller group's estimate has three times the standard error, or nine times the variance, under that model. These examples demonstrate aggregation and uncertainty, not a causal diagnosis of the observed performance gap.

Illustrative case study

Fictional case: Birch Voice evaluates an assistant using a dataset dominated by recordings from one market. Overall transcription quality appears strong, but a smaller language group shows worse results and wide uncertainty.
The team commissions additional representative evaluation recordings and reviews the corresponding training data and labels. It tests targeted improvements rather than claiming that duplicating the existing small-group recordings has solved the problem. Reports retain both group-level outcomes and the limitations of the evidence.

Common misconceptions and mistakes

Unequal sample sizes and unequal treatment are not identical concepts.
Equal group counts do not guarantee fair predictions or equal data quality.
A large overall test set does not ensure reliable results for every subgroup.
A non-significant difference can reflect limited information rather than equivalent performance.
Reweighting changes influence, not the number of independently observed cases. More data can help a specific information gap while leaving other sources of harmful error unchanged.

Business use

Use sample-size audits to identify where AI performance claims are well supported and where additional evidence is needed. Budget for relevant subgroups, rare outcomes, and data quality. Explain the aggregation used in executive reports so strong results for a large group do not conceal unresolved performance or uncertainty elsewhere.

Check your understanding

Why is the pooled error 8% while the equal-group average is about 13.33%?

Answer: The pooled rate gives nine times as much weight to group A.

Does the worked example prove that fewer training cases caused group B's errors?

Answer: No; it only describes evaluation counts and outcomes.

Related terms

Representation Bias
Effective sample size
Fairness metric
class imbalance
subgroup evaluation
intersectional analysis
weighting
fairness metrics

Learn next

Create a count-and-performance table for relevant groups, including uncertainty and label counts. Identify which gaps call for new observations, better labels, model changes, or a more suitable evaluation design.

Sources

University of Wisconsin–Madison, CS540 Ethics and Trust in AI: https://pages.cs.wisc.edu/~gkotse/cs540s25/documents/lec27.pdf Ding, Xi, and Akoglu, Outlier Detection Bias Busted, primary abstract: https://arxiv.org/abs/2408.13667 All numerical examples and the business case are original fictional illustrations.

Previous: Sample size determination Next: Sample space