Is sampling risk the same as sampling error?

They are related, but this entry focuses on the risk of an incorrect conclusion; sampling error can describe the difference between an estimate and the population value.

Does a census remove sampling risk?

For the defined population and procedure, yes. Other sources of error remain.

Can judgmental selection have sampling risk?

Yes. Not using a probability sample does not make uncertainty disappear, although a design-based numerical calculation may be unavailable.

AI Glossary / Statistics and probability / Advanced

Sampling risk

Definition

Sampling risk is the possibility that a conclusion based on a sample differs from the conclusion obtained by applying the same examination procedure to the whole population. In auditing, only some records or control operations may be examined. It concerns uncertainty introduced by sampling, rather than every possible reason an assessment could be wrong.

What it means

A well-selected sample can still miss exceptions that exist elsewhere in the population. It can also contain an unusually high proportion of exceptions and make a population look worse than a full examination would suggest.

Random selection makes those possibilities analysable under the design; it does not guarantee a representative result in every realized sample. To describe sampling risk meaningfully, specify the population, selection method, examination procedure and decision rule.

How it works

In audit testing, sampling can lead to incorrect acceptance or incorrect rejection of a conclusion about a balance. For tests of controls, it can produce too much or too little reliance on a control's effectiveness.

Separate these possibilities from nonsampling problems such as using an inappropriate examination procedure or failing to recognize an error in an item that was inspected. Examining more items addresses uncertainty from partial coverage only if the examination itself remains suitable. Full-population inspection removes the sampling component for that population, but does not guarantee correct measurement or interpretation.

Three practical examples

1. Document checks: a random sample contains no incorrectly extracted dates, although some exist among the unexamined documents. The absence of observed errors is evidence from that sample, not proof that every extraction is correct.

2. Workflow review: a sample happens to contain several approval exceptions from an otherwise mostly compliant set of transactions. The reviewer considers sampling uncertainty before extrapolating the observed fraction.

3. Automated inspection: an AI checker processes every record but systematically overlooks one error type. This is an examination-method problem, not an error caused by selecting only a sample.

Think of it this way

Think of tasting spoonfuls from a batch containing unusually salty portions. You can miss those portions even when the spoonfuls are chosen fairly. But if your taste test cannot detect salt, tasting more will not fix that separate problem.

Worked example

Consider a fictional twenty-record population, four of which contain a specified exception. An examiner can identify that exception perfectly whenever an affected record is inspected. Select five distinct records uniformly without replacement.

For this teaching exercise, use the decision rule “pass if no sampled exception is found.” Suppose a full-population pass would require no more than one exception. Since the population contains four, a sample pass is an incorrect acceptance under these stipulated rules. This mathematical example is not a prescribed audit plan.

There are C(20,5) = 15,504 equally likely five-record samples. A sample misses all four exceptions only when every selected record comes from the sixteen clean records. There are C(16,5) = 4,368 such samples.

Therefore, the probability of an incorrect pass is 4,368/15,504 = 91/323, approximately 28.17 percent. The probability of finding at least one exception is the complement, 232/323, approximately 71.83 percent. These probabilities assume the specified population and procedure.

Increase the sample to ten records while retaining the same zero-observed-exception rule. Now there are C(20,10) = 184,756 possible samples, of which C(16,10) = 8,008 miss all exceptions. The incorrect-pass probability becomes 14/323, approximately 4.33 percent. For this population and rule, the larger sample reduces the risk of missing all exceptions.

Do not reverse the conditioning. The calculation does not say that a population has a 28.17-percent probability of being unacceptable after five clean records are observed. That inference would require additional assumptions about possible populations. Here, the population's four exceptions were stipulated before the sampling probability was calculated.

Illustrative case study

Fictional Cedar Records uses the twenty-record example to examine its validation policy. Its report treats five clean observations as confirmation that the population meets the requirement. The team replaces that statement with the actual sample result and the conditional risk calculation.

It also considers a different population containing exactly one exception, which is acceptable under the same population criterion. Under the five-record rule, the sample fails whenever it includes that exception. Its probability of inclusion is 5/20 = 25 percent, so this rule can also reject an acceptable population.

Cedar therefore evaluates both kinds of decision error rather than only increasing sample size. This fictional example shows why population criteria and sample rules must be designed together.

Common misconceptions and mistakes

“Random selection eliminates sampling risk.” It does not. The five-record example uses uniform random selection and still misses every exception in 4,368 possible samples.

“A larger sample improves every decision-error probability under an unchanged rule.” Not necessarily. In the one-exception acceptable population, increasing the sample from five to ten raises the chance of failing the zero-observed-exception rule from 25 to 50 percent.

“A clean sample proves a clean population.” The worked example supplies a counterexample.

“Testing everything eliminates all assessment risk.” Incorrect labels, incomplete population records and an ineffective checker can remain even without sampling.

Business use

Define the decision before choosing the sample. Record which population the conclusion concerns, what constitutes an exception and how the sample result will affect the decision. A fixed sample count by itself does not specify the risk being controlled.

For AI-assisted review, assess the selection procedure and the checker separately. Keep the full population frame and sampled identifiers traceable, and document any excluded records. If the checker misses known exceptions in validation, increasing sample size alone does not justify treating its results as perfect examinations. Use the actual selection design when calculating uncertainty.

Check your understanding

Why is C(16,5) used in the numerator?

Answer: All five selected records must come from the sixteen clean records to miss every exception.

Why is perfect detection stated?

Answer: It isolates sampling risk from examination errors.

Why can the one-exception population fail the sample rule?

Answer: The sample rule permits no observed exception even though the population criterion permits one.

Related terms

Sampling design
Hypergeometric distribution
Audit sampling
incorrect acceptance
incorrect rejection
nonsampling risk
detection risk

Learn next

Allow one observed exception before failing the sample and recompute the two error probabilities. Compare how both sample size and the acceptance rule influence the decision.

Sample size

Sources

Public Company Accounting Oversight Board, AS 2315, Audit Sampling, particularly paragraphs .09–.14 and .17, version consulted October 2026: https://pcaobus.org/oversight/standards/auditing-standards/details/AS2315

Previous: Sampling rate Next: Sampling with replacement