AI Glossary / Statistics and probability / Intermediate
Unmatched count
Definition
Unmatched count refers to an indirect survey technique, also called the item count technique or list experiment, for estimating the prevalence of a sensitive characteristic. Respondents report how many statements apply to them rather than identifying each applicable statement. A randomly assigned treatment group receives a list containing an additional sensitive item, while a control group receives only the nonsensitive items. Under the design's assumptions, the difference in average reported counts estimates the sensitive item's prevalence.
What it means
The method aims to reduce pressure to reveal a sensitive answer directly. It estimates an aggregate rate rather than classifying every respondent. Random assignment makes the groups comparable in expectation, so their average counts for the common items should match. Adding one sensitive item then shifts the treatment group's expected count by its prevalence. This reasoning depends on respondents understanding and reporting counts as intended and on the extra item not changing responses to the original list.
How it works
Construct a set of nonsensitive items and randomly assign respondents to receive either that list or the same list plus the sensitive item. Ask only for the number of applicable statements. Estimate the difference between the treatment and control mean counts and calculate uncertainty using the actual sampling and assignment design. Important assumptions include no design effect on the common-item responses and appropriate truthful reporting. Floor and ceiling counts can weaken perceived privacy because some totals reveal all or none of the items, so list construction and piloting matter.
Three practical examples
Example 1: A workplace survey estimates the proportion who used an unapproved AI tool without asking each employee to identify that behavior directly. Example 2: A research study estimates a sensitive attitude through an added list item. Example 3: An organization compares aggregate reports of rule violations across populations using an appropriately designed list experiment. These applications require careful privacy and measurement design; the technique is not a guarantee that every respondent's answer remains undiscoverable.
Think of it this way
Think of asking how many lights are on behind a screen rather than asking which lights are on. Adding one special light to one group's setup changes the average total if that light is often on. The difference can reveal its overall frequency without normally identifying each person's setting. Extreme totals, however, can reveal more than intended.
Worked example
Suppose 400 respondents are randomly assigned to each group. Using rounded sample summaries, the control list has mean count 2.00 and sample variance 0.80; the treatment list has mean 2.35 and sample variance 0.90. The difference-in-means prevalence estimate is 2.35 minus 2.00, equal to 0.35, or 35%. Under an independent-groups approximation, the estimated standard error is the square root of 0.80/400 plus 0.90/400, approximately 0.0652. An illustrative normal 95% interval is 0.35 plus or minus 1.96 times 0.0652, approximately 0.222 to 0.478. This interval calculation assumes the stated design and a suitable large-sample approximation. It does not account for violations of truthful counting or changes to common-item responses caused by the sensitive item.
Illustrative case study
In this fictional case study, Fir Research evaluates unapproved AI-tool use through a list experiment. The team pilots the nonsensitive items to reduce revealing extremes and checks that employees understand the count-only instruction. Its report presents an aggregate estimate with uncertainty and explains the assumptions. Managers do not attempt to infer individual employees' behavior from their totals. The study is treated as one measurement approach whose credibility depends on implementation, rather than an automatic solution to sensitive-question bias.
Common misconceptions and mistakes
Common mistakes include interpreting the difference as a count of identified individuals, assuming random assignment eliminates all reporting bias, and ignoring the no-design-effect assumption. Another mistake is treating a negative or above-one raw estimate as meaningful prevalence; sampling variability or assumption failures can produce such estimates. The design can be statistically inefficient compared with direct questioning when direct answers are reliable. Privacy protection is also not absolute, especially for extreme totals or when auxiliary information is available.
Business use
Businesses considering this method should work with survey expertise to choose items, wording, allocation and sample size. Decide whether the desired aggregate estimate justifies the additional complexity and uncertainty. Keep individual inference outside the purpose of the survey and explain the reporting format clearly. Analyze nonresponse and subgroup precision, and avoid overstating what a single list experiment establishes. When comparing with direct questions, differences may reflect several measurement mechanisms rather than one method revealing a universally correct answer.
People also ask
Is unmatched count a matching procedure?
No; it is a list-based indirect questioning method.
Does it identify who has the sensitive characteristic?
That is not its purpose.
Why randomize the lists?
To make common-item counts comparable in expectation.
Can adding the sensitive item change other responses?
Yes, which would violate a key assumption.
Does the method always yield a more precise estimate?
No; indirect measurement often increases variance.
Check your understanding
What prevalence does the worked example estimate?
Answer: 35%.
What is the approximate standard error?
Answer: 0.0652, or 6.52 percentage points.
What assumption connects the mean difference to prevalence?
Answer: The added item does not alter the common-item responses, alongside valid assignment and reporting assumptions.
Does the interval quantify every possible measurement bias?
Answer: No.
Learn next
Learn next how researchers test for some violations of list-experiment assumptions and why those checks cannot prove all assumptions. Compare indirect questioning with randomized response and direct questioning. Then study how list composition, sample size and floor or ceiling responses affect precision and perceived privacy.
More in Statistics and probability
- Unit Weibull distribution - The unit Weibull distribution is a continuous distribution on the open interval from zero to one, obtained by transforming a positive Weibull variable with...
- Unit-weighted regression - Unit-weighted regression is a simple linear prediction approach that combines predictors using equal-magnitude weights, commonly after standardizing them...
- Univariate design - A univariate design is an experimental design with one dependent variable or outcome.
- Unobserved heterogeneity in duration models - Unobserved heterogeneity in duration models means that units differ in their tendency to experience an event for reasons not captured by the observed...
- Up-and-down design - An up-and-down design is a sequential experimental design in which the level used for the next observation depends on the preceding response.
- Uplift modeling - Uplift modeling estimates how an intervention changes an outcome, often as a function of a person's or item's characteristics.
Sources
Sources: Kosuke Imai, Multivariate Regression Analysis for the Item Count Technique, https://imai.fas.harvard.edu/research/list/; Graeme Blair and Kosuke Imai, Statistical Analysis of List Experiments, https://imai.fas.harvard.edu/research/listp/. These primary research sources define the technique and examine its assumptions. The prevalence and uncertainty calculation is original and independently checked; the workplace case is fictional.
Previous: Unlikelihood training Next: Unnormalized form