Must list sampling be systematic?

No. Systematic selection is a common form; random and stratified designs can also use lists.

Is a customer database always a suitable frame?

Only if its units and coverage match the intended question sufficiently well.

Does using the entire list remove every error?

No. It removes sampling from that list, but coverage, nonresponse, and measurement problems can remain.

AI Glossary / Statistics and probability / Beginner

List sample

Definition

A list sample is a sample selected from an explicit list of units in the population being studied. The list acts as the sampling frame. It might contain businesses, accounts, addresses, or equipment identifiers. The supplied OECD definition highlights systematic selection at equal intervals after a random start, although list-based sampling can also use other designs. A list sample is not automatically representative: both the list's coverage and the method of selecting and observing units matter.

What it means

The list defines which units have a chance to be selected. Missing eligible units have no chance under a design that uses only that list. Duplicate entries can give some units extra selection opportunities unless handled appropriately.

The target population, frame population, selected sample, and responding sample are different objects. A list of current paying customers, for example, cannot by itself represent everyone who has ever tried a service. Even a carefully selected sample can become unrepresentative if response patterns are ignored.

How it works

Define the population and the unit of analysis, then build or obtain a dated list with stable identifiers. Check eligibility, duplicates, missing units, and information used for stratification or contact. Preserve a reproducible version of the frame.

Choose the selection design. For a simple systematic example with N = nk entries, select a random start from 1 through k and then every kth entry. Other list designs include simple random or stratified selection, with probabilities recorded as appropriate.

Collect data using the selected identifiers and document nonresponse or ineligibility. Estimate results with weights and uncertainty methods matched to the design. A convenient list provides operational access, but it does not eliminate coverage or measurement errors.

Three practical examples

A company samples accounts from its current customer register to estimate service satisfaction among those accounts.
A maintenance team selects equipment identifiers from an asset list for inspection, checking that retired and newly installed units are treated consistently.
An AI quality team samples records from a frozen production log to audit outputs, defining whether the population consists of messages, sessions, users, or completed tasks before selecting entries.

Think of it this way

Think of drawing names from an address book. The drawing can be perfectly random while still excluding everyone absent from the book. If someone appears several times, they may have more chances to be drawn. Good selection and a good book solve different parts of the problem, so both require attention.

Worked example

Consider a fictional list of 12 unique units, ordered from 1 through 12. For a transparent calculation, suppose each unit's measured value equals its position. We want a systematic sample of four, so the interval is 12/4 = 3.

Choose a random start of 1, 2, or 3 with equal probability. Start 1 selects positions 1, 4, 7, and 10, with mean 5.5. Start 2 selects 2, 5, 8, and 11, with mean 6.5. Start 3 selects 3, 6, 9, and 12, with mean 7.5.

Each unit has inclusion probability one-third. Across the three possible samples, the expected sample mean is (5.5 + 6.5 + 7.5)/3 = 6.5, matching the full-list mean of 78/12. Multiplying each sample sum by the inverse inclusion probability, 3, gives total estimates 66, 78, and 90, averaging to the true total 78.

This verifies the finite-list calculation, not coverage of a larger population. The three possible systematic samples are also different from the many possible simple random samples of four; their uncertainty cannot automatically be treated as identical.

Illustrative case study

Fictional case: Elmbridge Software plans to audit its AI assistant by sampling from a list of successfully completed sessions. A reviewer notices that sessions ending in errors are absent from the list.

The team rebuilds the frame from all eligible sessions in a specified period and keeps completion status as a recorded attribute. It checks duplicate session identifiers and selects an auditable probability sample.

Reviewers then report error estimates for that defined period and population. They document missing transcripts separately instead of silently replacing them with easy-to-review cases. The sampling improvement comes from clarifying both the frame and follow-up process, not merely increasing the number of reviews.

Common misconceptions and mistakes

Do not treat a list as proof of complete coverage. Administrative systems may omit inactive, newly added, offline, or otherwise hard-to-record units. Do not equate selected units with responding units, or replace nonrespondents casually without recording the change.

Systematic selection can interact poorly with repeating patterns in list order. Sorting can also be useful when deliberately designed, but its consequences must be understood. Equal probabilities for list entries do not necessarily imply equal probabilities for people when one person can have multiple accounts or records.

Business use

List sampling supports customer research, operational audits, asset inspection, and AI evaluation when an explicit frame is available. Stable identifiers and documented selection make the exercise easier to reproduce and review.

Store the frame date, eligibility rules, unit definition, selection procedure, and response outcomes. Separate sampling uncertainty from coverage limitations. A narrow confidence interval around a sample of an incomplete list does not account for people or events that were never eligible to be selected from that frame.

Check your understanding

A company randomly samples from successful AI sessions only. Can the result estimate the error rate across all sessions without further assumptions?

Answer: No. Failed sessions are missing from the frame and have zero selection probability. Random selection within the available list does not repair that coverage problem.

Related terms

Random start
Coverage error
Nonresponse
sampling frame
systematic sampling
inclusion probability
sampling weight

Learn next

Next, compare list frames with area frames and review how duplicates change inclusion probabilities. Then learn how stratification and documented nonresponse adjustments can support a clearly defined survey estimand.

Nonresponse

Sources

Sources: OECD Glossary of Statistical Terms (2008), List Sample, printed page 309: https://www.oecd.org/content/dam/oecd/en/publications/reports/2008/09/oecd-glossary-of-statistical-terms_g1gh9ad7/9789264055087-en.pdf . Statistics Canada, Coverage and Frames: https://www150.statcan.gc.ca/n1/pub/12-539-x/2009001/coverage-couverture-eng.htm . Primary definitions and frame-quality guidance were consulted. The 12-unit example and fictional case are original; all three possible samples and weighted totals were independently checked.

Previous: List Next: Literal