Can continuous values define strata?

Yes, for example through predefined ranges, but boundaries and sparse groups need care.

Should every classification split be stratified?

No. The design must match the data and evaluation objective; grouping or temporal separation may be more important.

Does stratification change the model itself?

Not necessarily. It changes how data are organized, sampled or assessed, although those choices can affect training outcomes.

AI Glossary / Mathematics and optimization / Intermediate

Stratification

Definition

Stratification divides a population or dataset into defined subgroups, called strata, using selected characteristics. Sampling, splitting or analysis can then be organized within those groups so that their representation and results are examined deliberately.

What it means

A dataset may contain many ordinary cases and very few rare ones. A purely random small validation split can accidentally contain too few rare cases to support a useful assessment. Stratifying the split can preserve the selected class proportions more consistently.

The grouping characteristic can be a class label, location, age band or another relevant attribute. It must have a clear definition. In a conventional stratified sampling design, strata are mutually exclusive and together cover the target population.

Stratification is broader than class balancing. Proportional allocation preserves existing group shares; equal allocation gives each group the same sample size. These choices support different purposes. Neither automatically repairs bias in the original data or guarantees that the resulting model performs well in every group.

How it works

First define the strata and count their members. Next decide how many observations to allocate from each stratum to each sample or data split. Sample within the strata using an appropriate procedure, and keep the allocation rule with the resulting dataset.

For a proportional validation split, the same approximate fraction of each group enters validation. Integer counts can prevent exact proportions, especially for very small groups. A class with only a few observations cannot support arbitrary numbers of meaningful folds.

If groups are sampled at different rates, population summaries generally need weights that reflect the design and intended population. A simple unweighted average can otherwise describe the sample mixture rather than the population mixture.

Preserving strata does not override independence requirements. Repeated records from one person, related documents or future observations may need grouped or time-based splitting. Leakage can remain even when the class ratios look perfect.

Three practical examples

A classification experiment creates training and validation sets with similar proportions of its rare target class.
A customer survey samples separately by service region, retaining selection probabilities for later population estimates.
A quality review reports prediction errors separately for predefined equipment types to reveal differences hidden by the overall average.

Think of it this way

Think of tasting a mixed fruit delivery by taking a planned sample from each fruit type. The plan prevents a large pile of apples from crowding every pear out of the inspection.

The analogy concerns representation. It does not prove that the sampled fruit represents every supplier or that equal numbers from each type reflect the proportions in the whole delivery.

Worked example

Proportional splitting and weighted reporting

A dataset contains 900 ordinary cases and 100 rare cases. A proportional 20% validation split takes 180 ordinary and 20 rare cases, leaving 720 and 80 for training. Both sets preserve the original 90%/10% mixture.

Suppose an alternative evaluation deliberately includes 100 cases from each group to examine the rare group more closely. On that evaluation, a model has error rates of 5% for ordinary cases and 30% for rare cases. The unweighted error is (5 + 30)/200 = 17.5% because there are five and thirty errors.

If these within-group rates validly estimate the target population rates and the target population is 90% ordinary and 10% rare, the corresponding mixture estimate is 0.9 × 0.05 + 0.1 × 0.30 = 0.075, or 7.5%.

Both numbers have meanings: 17.5% describes the equal-group sample mixture, while 7.5% uses the stated population mixture. Neither should hide the rare group's 30% error rate. Sampling uncertainty and representativeness within each group still need assessment.

Illustrative case study

A fictional document-routing team celebrates high overall accuracy. When it stratifies evaluation by document type, it discovers that an uncommon category has too few validation examples for a convincing estimate.

The team collects a larger evaluation sample from that category and preserves the sampling design. Its next report includes group-specific results and a separately weighted overall estimate. It also keeps documents from the same source bundle together to avoid near-duplicate leakage across splits.

The lesson is that planned representation can improve the questions an evaluation can answer. It does not make poor group performance disappear, and it cannot recover evidence that was never collected.

Common misconceptions and mistakes

Stratification means equal group sizes. Proportional and unequal allocations are also possible.
Similar class proportions guarantee fair performance. Performance must still be measured within relevant groups.
Stratification fixes an unrepresentative source dataset. It cannot create missing populations or correct every selection mechanism.
More strata are always better. Tiny cells can make sampling and evaluation unreliable.
Matching labels prevents leakage. Shared entities and time relationships require additional controls.

Business use

Stratification helps make evaluations and sampling plans deliberate. Define which groups matter to the decision before inspecting favorable or unfavorable results. Record how groups were formed, how observations were selected and which population any aggregate claim describes.

Use sufficient group sample sizes and report uncertainty where appropriate. In cross-validation, stratification can make folds more homogeneous, so variation across those folds may understate other sources of real-world uncertainty. Combine the split design with checks for time dependence, repeated entities and coverage gaps.

Check your understanding

Why is the equal-group evaluation's unweighted error not the same as the population-weighted estimate?

Answer: The evaluation assigns half its weight to each group, while the stated population assigns 90% and 10%. The within-group error rates are combined using different mixtures.

Related terms

Stratified sampling
Sample weighting
Stratum
train-test split
class imbalance
grouped cross-validation

Learn next

Read sampling bias and cross-validation. Design a split for data containing rare classes and repeated customers. Explain how you would preserve useful representation while preventing the same customer's related records from leaking across evaluation boundaries.

Sources

Aalto Dictionary of Machine Learning, stratification: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Dataset splitting context.

Scikit-learn, Cross-validation guide: https://scikit-learn.org/stable/modules/cross_validation.html. Stratified splitting and its limitations.

NIST/SEMATECH glossary, strata and stratification: https://www.itl.nist.gov/div898/handbook/glossary.htm. Broader statistical usage. Allocation arithmetic and fictional review are original examples.

Previous: Strategy-proof Next: Stratified randomization