Can a census have undercoverage?

Yes. Attempting to enumerate everyone does not guarantee everyone is included.

Will weighting always fix it?

No. Its effectiveness depends on available information and assumptions about covered and omitted units.

Is every small subgroup undercovered?

No. A small group may be properly covered but have few sampled observations; that is a different issue.

AI Glossary / Data science and analytics / Advanced

Undercoverage Bias

Definition

Undercoverage bias is systematic distortion in an estimate caused by failing to cover part of the intended population, when the omitted part differs in ways relevant to that estimate. It commonly arises when a survey's sampling frame leaves out eligible people or organizations. Missing coverage and the resulting bias are related but distinct.

What it means

Suppose a business wants to understand all its learners but surveys only people with active online accounts. Learners attending solely through partner classrooms may have no opportunity to be selected. If their experiences differ, the survey can misrepresent the full learner population.
The problem is not simply that the sample contains fewer people than the population. A probability sample can legitimately select only a fraction of eligible people while giving everyone a known chance of selection. Undercoverage concerns eligible units the collection arrangement fails to reach or include.
Its effect depends on the question. An omitted group might have similar satisfaction but very different access requirements. The same coverage gap can therefore cause little distortion in one estimate and substantial distortion in another.

How it works

Define the target population, including the period and eligibility rules. Identify the frame or collection mechanism used to access it, then compare the two. Outdated lists, missing locations, restricted contact channels and omitted categories can create gaps.
Assess which units are absent and how their characteristics relate to the quantities being estimated. A coverage percentage alone cannot establish the size or direction of bias.
Improve the frame where feasible, use complementary sources carefully and document remaining exclusions. Combining lists requires attention to overlapping records so the correction does not introduce duplicate inclusion.
Weighting or modeling may reduce bias when suitable auxiliary information and defensible assumptions are available. It cannot automatically reveal the answers of an entirely unobserved group. Report the dependence on those assumptions rather than presenting adjustment as a guaranteed repair.

Three practical examples

Customer research: A retailer surveys only loyalty-card members while describing the results as representing all customers. Non-members may have different purchasing habits or service experiences.
AI evaluation: A speech tool is tested only on recordings from one supported device family, although deployment includes other devices. The evaluation fails to cover part of the intended operating population and may misstate overall performance.
Community planning: An online-only questionnaire is used to describe everyone in a neighborhood. Residents unable to access the questionnaire may be absent from the collection mechanism, separately from those who could respond but choose not to.

Think of it this way

Think of measuring a garden by looking only through one window. Taking more photographs from that window can describe the visible area more precisely. It does not reveal the part behind the wall. Whether the estimate of the whole garden is misleading depends on how the hidden part differs from the visible part.

Worked example

Imagine a training provider with 1,000 learners: 800 online learners and 200 classroom-only learners. For this illustration, the actual satisfaction rates are known to be 80% and 50% respectively.
The full-population satisfaction rate is (800 × 0.80 + 200 × 0.50) / 1,000 = 740 / 1,000 = 74%.
A frame containing only online learners covers 80% of the population. Even if every person on that frame responds accurately, the reported satisfaction rate is 80%, six percentage points above the full-population rate.
The missing 20% does not imply a 20-percentage-point bias. Here the discrepancy is the omitted share multiplied by the difference between group rates: 0.20 × (0.80 − 0.50) = 0.06.
If classroom-only learners also had 80% satisfaction, this particular rate would have no coverage-induced discrepancy, despite the incomplete frame. In a real survey, their satisfaction would not normally be known; obtaining evidence about it is part of the challenge.

Illustrative case study

Fictional publisher Birch Learning plans new resources using feedback from registered website users. A partner points out that many workshop participants read printed materials without creating accounts, although the project intends to serve them too.
The team compares the website list with partner participation records and discovers an omitted route into the service. It designs a supplementary collection process, explains the intended population and checks for people appearing in both sources.
Birch reports website-user findings separately while the broader study develops. It does not simply relabel the first survey as representative of everyone. This fictional case illustrates a coverage investigation, not a claim that combining two lists necessarily achieves complete or unbiased coverage.

Common misconceptions and mistakes

A large sample does not eliminate undercoverage. Additional observations from the same incomplete frame can leave the gap unchanged.
Undercoverage and nonresponse are different mechanisms. A person missing from the frame is different from a selected eligible person who does not answer, although both can affect representativeness.
An incomplete frame does not impose the same bias on every estimate. Outcome differences matter as well as the extent of omission.
A conventional sampling margin of error does not automatically account for coverage bias. Narrow intervals can accompany a systematically distorted estimate.
Undercoverage does not necessarily push a reported percentage downward. In the worked example it pushes satisfaction upward.

Business use

Coverage assessment helps businesses avoid designing services around only their easiest-to-observe customers. State who the evidence represents before using it for product priorities, staffing or AI performance claims. Track changes in the collection frame over time: an apparent trend may reflect newly included groups. Where coverage remains limited, narrow the claim honestly and explain which decisions need additional evidence.

Check your understanding

Why is the example's overall satisfaction 74%?

Answer: It combines both groups using their population sizes.

Why would surveying all 800 online learners not solve the problem?

Answer: The 200 classroom-only learners remain absent.

Does 20% undercoverage establish a 20% bias?

Answer: No. Bias also depends on relevant differences between groups.

Related terms

Coverage error
Selection bias
Sampling error
Sampling frame
target population
nonresponse bias
survey weighting
representativeness

Learn next

Compare a proposed survey's target population with its actual contact list. Identify who has no route into the sample and what evidence would be needed to understand the effect of those omissions.

Sources

Statistics Canada, Quality Guidelines, Coverage and Frames, archived guidance: target populations, frames and coverage assessment. https://www150.statcan.gc.ca/n1/pub/12-539-x/2009001/coverage-couverture-eng.htm U.S. Census Bureau, SIPP Nonsampling Error: undercoverage, weighting adjustments and limitations of bias correction. https://www.census.gov/programs-surveys/sipp/methodology/nonsampling-error.html The learner calculation, practical examples and Birch Learning case are original illustrations, not findings from either agency.

Previous: Unconstrained minimization Next: Underdamped Langevin dynamics