AI Glossary / Statistics and probability / Intermediate
Group testing
Definition
Group testing is a method for identifying items with a particular property by testing collections of items together. In the basic noiseless binary model, a group test is positive if at least one included item is positive and negative if none are positive. Carefully chosen groups can reduce the number of tests needed when positives are uncommon. The method includes adaptive procedures, which use earlier results to choose later tests, and nonadaptive procedures, which fix all groups in advance.
What it means
Testing one item at a time can be inefficient when almost every result is negative. A negative result for a group can clear several items at once, provided the test reliably detects a positive item anywhere in that group.
A positive group result usually does not identify which member is positive. Further tests or an informative pattern of overlapping groups are needed. Group testing trades the design and interpretation of pooled tests against the cost of individual tests.
Processing independent tests simultaneously is not group testing unless information is combined into group-level observations. The test must support the assumed pooling behavior.
How it works
In the two-stage Dorfman procedure, partition items into nonoverlapping groups. Test each group once. A negative group needs no further tests under the ideal model; for a positive group, test every member individually.
If a group contains k items, each independently positive with probability p, its negative probability is (1 - p)^k. The expected number of tests for that group is 1 + k*[1 - (1 - p)^k], for k greater than one. Dividing by k gives expected tests per item.
Other designs split positive groups repeatedly or use overlapping tests and decoding algorithms. Practical design must consider test errors, pool-size limits, dependence among item statuses, turnaround time, and handling costs. Physical pooling can weaken a detectable signal, while logical tests may fail the ideal rule because of interactions. Such effects require an appropriate model and validation.
Three practical examples
- Quality inspection: screen groups of items using a validated test that detects the target defect if any group member has it, then resolve positive groups.
- Fault localization: design subset checks to identify a small number of faulty components when a check's outcome reliably reflects whether its subset contains a fault.
- Data-science research: study pooled observations as a sparse-inference problem, comparing test designs and decoding methods under explicit error and resource constraints.
Think of it this way
Think of checking boxes for at least one item bearing a detectable tag. A reliable negative scan can clear the entire box. A positive scan says that a tag is somewhere inside, so the items need further separation or testing. Bigger boxes save scans only when the extra ambiguity and detection limitations remain manageable.
Worked example
Consider an original idealized scenario with 100 items, each independently positive with probability 0.02. Use twenty groups of five and the two-stage Dorfman procedure, assuming perfect group and individual tests.
A group is negative with probability 0.98^5 = 0.9039208, and positive with probability 0.0960792. It uses one initial test, plus five individual tests if positive. Expected tests per group are 1 + 5*0.0960792 = 1.480396. Across twenty groups, the expectation is 29.6079 tests, compared with 100 individual tests.
Independent enumeration of all 32 possible positive/negative configurations in one five-item group, weighted by their Bernoulli probabilities, gives the same expectation. The expected reduction is about 70.4%, but the actual total varies. If two groups test positive, the procedure uses 20 + 2*5 = 30 tests. If all twenty groups test positive, it uses 120 tests, more than individual testing.
The calculation is conditional on its assumptions. If p rises to 0.20, expected tests become 20*[1 + 5*(1 - 0.8^5)] = 87.232. A fixed pooling plan can therefore lose much of its advantage as positives become more common. These figures demonstrate arithmetic, not a recommended operating pool size or a guarantee of detection accuracy.
Illustrative case study
In a fictional electronics workshop, engineers explore a group-level test for a rare component defect. They first verify that one faulty component remains detectable when mixed with the permitted number of good components. They also investigate whether groups of components interact in ways that change the test outcome.
A pilot compares testing effort, handling time, missed defects, and identification delay. The team retains individual testing where grouping provides little benefit. This fictional case balances expected savings with operational constraints; it does not claim that any batch inspection satisfies the group-testing assumptions.
Common misconceptions and mistakes
Common mistakes include declaring every member of a positive group positive, assuming negative pools are infallible in a noisy setting, and equating fewer tests with lower overall cost.
Large groups can increase retesting or reduce detection sensitivity. Dependence among item statuses changes the negative-pool probability, so the independent formula should not be used blindly. Different item risks also require a more suitable calculation than one common p.
Group testing is not automatically effective for arbitrary software failures: a failure may require an interaction among several components, violating the basic any-positive rule. Likewise, overlapping pooled tests need a decoding method appropriate to their design. A visually clever grouping pattern alone does not establish recoverability.
Business use
Use group testing when the target property is relatively sparse, pooling is technically valid, and test savings outweigh added handling and delay. Record the design, status assumptions, test accuracy, decoding procedure, and escalation rules. Business evaluation should measure total effort and error consequences as well as the expected number of tests.
People also ask
Does a positive pool identify the positive item?
Usually not by itself; follow-up tests or other pooled outcomes are needed.
Is group testing always adaptive?
No. Nonadaptive designs fix all tests in advance.
Can it use more tests than individual testing?
Yes. For example, many positive Dorfman groups require extensive individual retesting.
Check your understanding
Answer: Ten groups of four items are tested, and three groups are positive.
Under the stated two-stage procedure, how many tests are used?
Answer: There are ten initial group tests plus twelve individual follow-ups, giving 22. The positive pools do not imply that all twelve retested items are positive.
Related terms
- Decoding
- False negative
- Combinatorial search
- Pooled testing
- Dorfman procedure
- sparse inference
- adaptive testing
- nonadaptive design
Learn next
Learn the ideal Boolean group-test model, then compare adaptive splitting and overlapping nonadaptive designs. Study how noise, dependence, and physical pool limits alter both recoverability and expected savings.
More in Statistics and probability
- Graphical data editing - Graphical data editing is the use of graphs to identify potentially erroneous, inconsistent, or otherwise unusual data for investigation and possible...
- Grid sampling - Grid sampling, in the OECD glossary sense used here, is a form of cluster sampling whose clusters are grid areas composed of basic cells arranged in a...
- GRIM test - The GRIM test, short for granularity-related inconsistency of means, checks whether a reported mean is mathematically compatible with its stated sample...
- Growth curve (statistics) - A growth curve in statistics describes how an outcome changes over time, often using repeated measurements from the same individuals or other units.
- Growth function - The growth function of a binary hypothesis class gives the largest number of distinct label patterns that its classifiers can produce on a set of a given size.
- Grubbs's test - Grubbs's test is a statistical test for a single unusually extreme observation in a univariate sample whose ordinary observations are assumed to come from...
Sources
Landolfi and Lall, Optimal Dorfman Group Testing for Symmetric Distributions: https://arxiv.org/html/2308.11050v2 — primary formulation of the two-stage procedure and dependence-aware costs. Aldridge, Johnson, and Scarlett, Group Testing: An Information Theory Perspective: https://arxiv.org/pdf/1902.06002v2 — author monograph on group-testing models, algorithms, and noise. The numerical scenario and workshop case above are original teaching material.
Previous: Group method of data handling Next: Grouped Dirichlet distribution