Is the sample necessarily random?

Selection among included units may be random, but the complete target population is not covered by a positive-probability design when some units are deliberately excluded.

Can cut-off sampling still be useful?

Yes, for a suitable objective with transparent coverage and defensible estimation methods.

Does an 80% activity-coverage figure mean 80% accuracy?

No. Coverage and estimation accuracy are different quantities, and the excluded portion can affect different statistics in different ways.

AI Glossary / Statistics and probability / Intermediate

Cut-off sampling

Definition

Cut-off sampling is a sampling procedure that deliberately excludes part of a target population from possible selection using a threshold or related eligibility rule. In business surveys, a common example excludes enterprises below a specified size. The excluded units have zero selection probability, which has important consequences when estimating quantities for the full population.

What it means

The attraction is practical: many small units may contribute little to an aggregate total while creating substantial collection cost or response burden. Concentrating collection on larger units can capture much of the measured activity efficiently.

However, capturing most activity is not the same as representing most organizations, people, or behaviors. A design useful for estimating aggregate turnover may be poor for estimating the percentage of firms adopting AI. The impact of the cutoff depends on the quantity being estimated, how the excluded units differ, and what auxiliary information is available about them.

This concerns survey selection, not truncation or classification thresholds.

How it works

Define the target population and the variable used for the cutoff, such as employment or turnover measured at a stated reference date. Specify which units can be selected and which cannot. Above the cutoff, the design may include every unit or use a probability sample; cut-off sampling does not require a census of all eligible units.

Separate the contribution of the included population from that of the excluded population. Estimation for the excluded portion may use administrative records, explicit models, or other evidence. Ordinary inverse-probability weights cannot represent units whose inclusion probability is zero.

Document the assumptions supporting a full-population estimate and evaluate their sensitivity. If the target is genuinely restricted to above-cutoff units, label it accordingly. Do not silently redefine the target after collection to conceal missing population coverage.

Three practical examples

A business survey concentrates collection on enterprises above an employment threshold and uses suitable administrative information to assess smaller enterprises' contribution.
An industry study estimates total technology spending from large firms while explicitly modeling the omitted small-firm component and reporting the resulting limitations.
An AI adoption report checks whether an existing cutoff-based survey can support a firm-level adoption percentage, rather than assuming high turnover coverage implies representative firm counts.

Think of it this way

Think of estimating the water in a collection of containers by measuring only the large tanks. They may hold most of the water, but the small containers do not disappear. Estimating the total requires accounting for their contents. Estimating the proportion of containers that are blue is a different problem, for which ignoring many small containers could be especially misleading.

Worked example

Consider an original population of 100 firms. Ten large firms each have turnover of 8 million currency units, and 90 small firms each have turnover of 0.2 million. A cutoff admits only the large firms, all of which are measured.

Observed turnover is 10 times 8 million = 80 million. The excluded contribution is 90 times 0.2 million = 18 million, so the true total in this constructed population is 98 million. Reporting 80 million as the full total understates it by 18/98, approximately 18.37%.

Now suppose eight large firms and nine small firms use an AI tool. Adoption among included firms is 8/10 = 80%, but adoption across all firms is 17/100 = 17%. Turnover coverage is 80/98, approximately 81.63%, yet this does not establish representativeness for adoption.

If an external model assumes average small-firm turnover of 0.15 million, the estimated total becomes 80 + 90 times 0.15 = 93.5 million. Its error is -4.5 million. The adjustment helps relative to ignoring small firms, but its accuracy depends on the unobserved component's model.

Illustrative case study

In this fictional case, Market Lantern surveys large service firms to estimate annual spending on AI systems. Its communications team proposes reusing the results for a headline about “all businesses adopting AI.”

Analysts explain that the cutoff was designed around spending coverage, not organization-level adoption. They publish the large-firm result with its proper population label and develop an additional sampling strategy for smaller firms before making broader claims.

For spending, they describe the auxiliary evidence and sensitivity analysis for the excluded population. This invented case illustrates how a dataset can support one purpose better than another; it reports no actual adoption statistic.

Common misconceptions and mistakes

Cut-off sampling is not the same as probability-proportional-to-size sampling. In a probability design, even small units can have positive selection probabilities, whereas excluded cutoff units have none.

Increasing the sample size above the cutoff does not reveal the excluded population's outcomes. A highly precise estimate for covered units can still miss the full-population total substantially.

Do not assume a cutoff remains suitable indefinitely. Businesses grow, shrink, enter, and exit, while the relationship between the size measure and the study variable can change. Model-based adjustments and calibration require supporting assumptions; they are not automatic repairs for absent coverage.

Business use

Businesses and statistical teams can weigh collection cost against the consequences of excluding units. The relevant criterion is total error for the intended estimate, not simply the percentage of units or turnover covered.

Keep a record of the cutoff rule, reference date, excluded population, estimation method, and sources of auxiliary information. Examine results separately for small domains, where omitted units may form a large share even when national coverage looks strong. When the decision depends on smaller firms' experiences or needs, collect evidence that directly represents those firms.

Check your understanding

Check your understanding: Why is adoption 17% in the worked population despite 80% among surveyed firms?

Answer: Most firms are below the cutoff and have a different adoption proportion.

Can surveying the ten large firms more carefully recover small firms' unknown outcomes?

Answer: No.

What makes the 93.5 million estimate conditional?

Answer: It relies on an assumed average for excluded firms.

Related terms

Coverage error
Calibration
Sampling frame
inclusion probability
survey weighting
model-based estimation
probability-proportional-to-size sampling
small area estimation

Learn next

Compare a cutoff design with a stratified probability design that retains some small units. Examine how total error changes when the objective shifts from estimating aggregate turnover to estimating the proportion of businesses with a characteristic.

Sources

Sources: Statistics Canada, Small area estimation methods under cut-off sampling, Section 2, https://www150.statcan.gc.ca/n1/pub/12-001-x/2020001/article/00004/02-eng.htm. Elisson and Elvers, Cut-off sampling and estimation, official publication summary, https://www150.statcan.gc.ca/n1/en/catalogue/11-522-X20010016259. The population, calculations, and fictional case are original.

Previous: Cut detection Next: Cutoff test