Can nonresponse error affect a census?

Yes. Attempting to contact everyone does not ensure that everyone supplies the intended information.

Is it always caused by refusal?

No. Noncontact, incomplete answers, and other mechanisms can create missing information with consequences for estimates.

Can it be measured exactly?

Usually not without knowing or credibly recovering the missing values. Follow-up evidence and models can help assess it.

Does adding more respondents fix it?

Only if the added information and analysis address the relevant differences; increasing counts alone is insufficient.

AI Glossary / Statistics and probability / Intermediate

Nonresponse error

Definition

Nonresponse error is the error in a statistical result attributable to information not being obtained from units or items intended to be observed. It is especially concerning when responders and nonresponders differ on characteristics relevant to the estimate. Nonresponse is the missing-information event; nonresponse error is its consequence for the result. The size of that consequence is not determined by the response rate alone.

What it means

If the observed answers systematically differ from the missing answers, an estimate based on respondents may misrepresent the intended population. A large number of responses can still leave this problem unresolved.

The term covers more than the simple percentage of missing records. Its effect depends on the outcome, the pattern of missingness, and the estimation method, including any weighting or imputation. The same survey can have different nonresponse effects for different questions because willingness or ability to answer may relate differently to each outcome.

How it works

Define the complete-information quantity that the study intends to estimate. Identify how the actual estimator uses the available observations and how missing information is handled. Investigate relevant differences between responding and nonresponding units using auxiliary information, follow-up data, or other justified evidence.

For a simple finite group, let r be the responding fraction, yR the respondent mean, and yN the nonrespondent mean. The complete-group mean is r × yR + (1 − r) × yN. The difference between the respondent mean and complete-group mean is therefore (1 − r) × (yR − yN).

This identity shows two ingredients: how much information is missing and how different its average would be. In real studies, yN is often unknown, which makes direct evaluation difficult. Weighting or imputation can reduce error under appropriate assumptions, but cannot guarantee that the unobserved information has been recovered correctly.

Three practical examples

Satisfied customers respond more frequently to a feedback request, inflating the respondent-only satisfaction percentage.
People with unusually long task times are less likely to complete a follow-up questionnaire, distorting the reported duration distribution.
A survey's income question has selective item nonresponse, affecting income estimates even when other questions have high completion rates.

Think of it this way

Think of judging a meeting by the comments from people who stayed until the end. If those who left had similar views, the missing comments may change little. If they left because the meeting failed their needs, the remaining comments can paint a very different picture.

Worked example

Consider an original finite group of 1,000 customers. Six hundred respond to a survey, and 480 of those respondents are satisfied, so respondent satisfaction is 80%. Suppose, for this teaching example only, that the 400 nonrespondents contain 160 satisfied customers, giving nonrespondent satisfaction of 40%.

The full-group satisfaction proportion is (480 + 160)/1,000 = 64%. Reporting the respondent percentage as the full-group value overstates satisfaction by 16 percentage points. The identity gives the same result: (1 − 0.6) × (0.8 − 0.4) = 0.16.

Now consider a separate scenario with only 300 respondents out of 1,000, but both responders and nonresponders have satisfaction of 64%. The respondent-only percentage then equals the full-group percentage despite the lower response rate. This does not make low response desirable: precision and other vulnerabilities may remain. It demonstrates that response rate alone does not identify nonresponse error. The missing outcomes are deliberately supplied in these teaching scenarios; an actual analyst would generally need additional evidence or sensitivity assumptions to know them.

Illustrative case study

Fictional case study: Maple Support evaluates an AI service through a voluntary post-session survey. Its favorable-response percentage rises after a redesign, but the proportion reaching the survey falls because more sessions terminate early.

An analyst investigates the collection path and compares available session characteristics across response groups. The team conducts a targeted follow-up and reports how alternative assumptions about missing feedback affect the conclusion. Maple avoids describing the respondent percentage as a complete measure of customer experience until the evidence supports that scope. The fictional case highlights how an apparent improvement can coincide with a change in who supplies the measurements.

Common misconceptions and mistakes

“A 90% response rate proves negligible bias.” The missing 10% could differ substantially on the outcome that matters.

“A low response rate proves the estimate is biased.” Missingness creates concern, but the actual effect depends on differences and the estimator.

“Weighting on age and region removes all nonresponse error.” Unmeasured differences can remain within those adjustment groups.

“Sampling error includes the entire problem.” Conventional sampling uncertainty may not capture distortion from missing answers.

Business use

Nonresponse error matters whenever business decisions depend on feedback, follow-up measurements, or observed outcomes that are selectively unavailable. For AI products, failures and abandoned sessions may be particularly relevant because those experiences can be less likely to generate a completed rating.

Maintain a stable definition of the target population and track collection changes alongside reported performance. Allocate follow-up effort where missing information could change the decision. Report adjusted and unadjusted results where useful, and explain the assumptions behind adjustment. A sensitivity range can be more informative than a precise-looking percentage that silently treats missing answers as comparable to observed ones.

Check your understanding

If 20% of a group is missing and respondents average ten points higher than nonrespondents, how far above the complete-group mean is the respondent mean?

Answer: Two points, because 0.20 × 10 = 2. This uses the simple complete-group decomposition and assumes the two group means are known.

Related terms

Nonresponse
Non-sampling error
Selection bias
Missing data
Sensitivity analysis
response propensity
weighting adjustment
imputation

Learn next

Separate response-rate reporting from bias assessment. Work through the respondent - nonrespondent decomposition, then examine what auxiliary information can and cannot establish. Next, compare follow-up designs and sensitivity analyses that make assumptions about missing outcomes visible.

Sources

NIST/SEMATECH e-Handbook glossary, nonresponse error: https://www.itl.nist.gov/div898/handbook/glossary.htm

Statistics Canada, Non-sampling error, consequences of nonresponse and imputation: https://www150.statcan.gc.ca/n1/edu/power-pouvoir/ch6/nse-endae/5214806-eng.htm

The finite-group identity, numerical scenarios, and Maple Support case are original teaching explanations. The calculations use explicitly supplied missing-group outcomes, not estimated facts about an actual survey.

Previous: Nonresponse Next: Nonserial dynamic programming