Does the dataset need an extreme overall class imbalance?

Not necessarily. Conditional imbalance can make the method useful even when overall class counts are closer to balanced.

What is the pilot model for?

It estimates which labeled observations are surprising and supplies information needed for the correction.

Is the retained sample size fixed?

In the basic accept-reject procedure it is random, with an expected size determined by acceptance probabilities.

AI Glossary / AI and machine learning / Advanced

Local case-control sampling

Definition

Local case-control sampling is a subsampling method for fitting logistic regression to large datasets. It uses a preliminary model to retain observations according to how surprising their labels are given their features, then corrects for the selective sampling when estimating the final model.

What it means

A large binary-classification dataset can contain many easy, repetitive examples and a smaller number that carry more information about the decision relationship. Uniformly sampling rows may spend much of the reduced dataset on those easy cases.

Ordinary case-control sampling changes sampling rates by class. Local case-control sampling also considers the features through a pilot probability model. It can therefore exploit conditional imbalance even when the overall class counts are not severely imbalanced.

The retained sample is deliberately unrepresentative in a specific way. That is acceptable only because the estimation procedure accounts for the selection mechanism. Taking the selected rows and treating them as an ordinary random sample would generally distort the fitted probabilities and undermine the method's purpose.

How it works

First obtain a pilot estimate p-tilde(x) of the probability that the label is one. In the baseline procedure, an observation with label y is accepted with probability equal to the absolute residual |y - p-tilde(x)|.

An unexpected positive receives high acceptance probability when the pilot predicted a low probability, while an expected negative receives low acceptance probability in the same region. The method then fits a logistic model to the accepted observations with the appropriate correction involving the pilot model.

Fithian and Hastie analyze statistical efficiency under specified assumptions. Their baseline variance result depends on correct model specification and an independent, consistent pilot; it is not an unconditional promise for every dataset. Practical use must account for pilot quality, selection randomness and the exact correction. The selected sample size is random unless an additional procedure controls it.

Three practical examples

1. An analyst fits logistic regression on a very large event dataset, retaining a more informative subset under a pilot-guided scheme.

2. A research team compares local case-control sampling with uniform and class-only sampling when labels are predictable within feature regions.

3. A modeling experiment measures how pilot quality affects both the selected sample and the final corrected estimates.

Think of it this way

Think of a reviewer spending more time on cases that disagree with an initial assessment. The unusual cases can be informative, but the final summary must remember that they were deliberately overrepresented. The limit is that local case-control sampling uses a precise probability and correction, not an informal rule to keep only mistakes.

Worked example

Suppose a pilot predicts p-tilde(x) = 0.1. For an observed positive y = 1, baseline acceptance probability is |1 - 0.1| = 0.9. For an observed negative y = 0, it is |0 - 0.1| = 0.1.

Thus a surprising positive is nine times as likely to be retained as a negative at that same pilot probability. In a group of 100 such positives and 100 such negatives, expected accepted counts are 90 and 10, although actual counts vary randomly.

Those accepted counts do not estimate the original class prevalence. They reflect the selection design, which is why the final estimation step must correct for sampling.

Illustrative case study

A fictional platform-analytics team uses a pilot model to reduce a large labeled dataset before fitting logistic regression. It initially fits the selected data without the required correction and finds that predicted event rates are too high.

The team implements the full procedure, verifies the acceptance probabilities and evaluates estimates against a manageable full-data benchmark. It also reports the effort needed to obtain the pilot, rather than counting only the final fit.

The case illustrates that smart subsampling is an estimation method, not merely a data-cleaning shortcut. Selection and correction are two parts of the same design.

Common misconceptions and mistakes

The method keeps only misclassified examples. It samples probabilistically using the pilot residual.
It is ordinary class balancing. Selection depends on features as well as labels.
The selected sample can be analyzed as if randomly drawn from the original population. Its deliberate selection requires the appropriate correction.

Business use

Local case-control sampling can help when logistic-regression fitting is constrained by very large data and the pilot can identify substantial conditional imbalance. It is a specialized statistical method that should be implemented and reviewed carefully.

Compare total runtime and estimation quality with full-data fitting where feasible and with simpler sampling baselines. Record the pilot, acceptance rule, randomization and correction. Check calibration on an independent representative dataset. Do not generalize the original method's theoretical guarantees to arbitrary classifiers or sampling modifications without supporting analysis.

Check your understanding

Why would the accepted sample's positive fraction be a poor direct estimate of the population event rate?

Answer: Acceptance depends on the label and pilot prediction, so the sampling process deliberately changes the class composition.

Related terms

Logistic regression
Selection bias
subsampling
case-control sampling
class imbalance
pilot estimate
statistical efficiency

Learn next

Read logistic regression and sampling bias. Calculate acceptance probabilities for several pilot predictions, then explain why a corrected estimator is needed after selective retention.

Sources

Fithian and Hastie, Local Case-Control Sampling: Efficient Subsampling in Imbalanced Data Sets (2014): https://pmc.ncbi.nlm.nih.gov/articles/PMC4258397/. Author preprint: https://arxiv.org/abs/1306.3706. The acceptance-count example and platform-team scenario are invented; theoretical results apply only under their stated assumptions.

Previous: Local beam search Next: Local Community Detection