Is adaptive sampling the same as active learning?

Active learning commonly uses adaptive selection to request informative labels, but adaptive sampling also covers other measurement and search settings.

Must it select one sample at a time?

No. It can select batches and update between them, trading responsiveness against parallel efficiency and feedback cost.

Does it always concentrate on uncertain cases?

No. Some methods seek high predicted value, rare events or other objectives. The adaptation rule follows the task rather than a universal uncertainty rule.

AI Glossary / Generative AI and language / Intermediate

Adaptive sampling

Definition

Adaptive sampling is an approach in which information from earlier samples influences which samples are selected next or how sampling effort is allocated. It aims to use limited measurement, labeling or computation resources more effectively. The term describes a family of strategies, not one universal algorithm.

What it means

A fixed sampling plan chooses its allocation without reacting to newly observed results. An adaptive plan contains a feedback loop: observe, update what is known, and choose the next sampling action accordingly.

The objective can vary. A team may want to reduce uncertainty, find rare conditions, improve a predictive model or search for promising designs. A strategy that excels at finding interesting cases may be unsuitable for estimating their population frequency without an appropriate analysis.

In generative-model research, adaptation can change the distribution from which candidate designs are drawn based on feedback about their properties. This is one application, not the definition of every adaptive sampler. It should not be confused with ADASYN, a particular synthetic oversampling algorithm.

How it works

Specify the objective, initial sampling plan, available actions and resource budget. Collect an initial set of observations and use them to update a model, estimate or other summary of current knowledge.

A selection rule then chooses where to spend the next unit or batch of effort. It might favor high uncertainty, promising predicted outcomes, poorly covered regions or a mixture of these. After observing the selected samples, update again and repeat until the budget or stopping criterion is reached.

The rule must account for exploration. If it repeatedly follows only what already looks promising, it may overlook useful regions or reinforce errors in its current model. Predicted uncertainty and predicted value are themselves fallible.

Keep a record of selection decisions and, where relevant, selection probabilities. Adaptive observations are not automatically a simple random sample. Population estimates, uncertainty calculations and stopping decisions must respect the sampling design and assumptions. Independent evaluation can help determine whether the adaptive strategy actually improves the target task.

Three practical examples

An expensive simulation study allocates additional runs where a surrogate model predicts poorly or where decisions are especially sensitive.
A labeling project selects informative documents for annotation as its classifier changes, while retaining separate evaluation data.
A generative design experiment updates its candidate-generation distribution using feedback from a property evaluator, then checks promising candidates independently.

Think of it this way

Think of searching a large room with a limited torch battery. Early observations can guide where you look next, rather than illuminating every area for the same time.

That search may efficiently locate a dropped object, but it would not automatically provide an unbiased estimate of how much dust covers the whole room. The analogy separates targeted discovery from representative measurement.

Worked example

An adaptive allocation changes the observed mixture

Suppose an invented inspection project has 60 checks available across three equal-sized regions. It starts with ten checks per region. The first results suggest that region B deserves closer investigation, so the remaining 30 checks are allocated as five to A, twenty to B and five to C.

Final sample counts are 15, 30 and 15. Region B accounts for half the observations even though it contains one-third of the population.

If the team simply pools the observed defect indicators and calls their average the overall population defect rate, the deliberate extra attention to B can distort that interpretation. The appropriate estimator depends on the design, including how the adaptive allocation was decided. The example explains the issue; it does not prescribe a complete survey estimator.

Illustrative case study

A fictional materials studio, Alder Designs, uses a generative model to propose candidate surface patterns. A predictive evaluator gives each candidate a score, and the sampler gradually concentrates on highly rated patterns.

Independent checks reveal that some extreme proposals exploit weaknesses in the evaluator rather than meeting the intended property. The team retains broader exploration, limits unsupported extrapolation and sends selected candidates through a more reliable assessment.

It also measures useful discoveries per unit of total effort, including evaluator calls and failed candidates. The adaptation remains a promising search technique, but its success depends on the quality of the feedback and the evaluation protocol. The fictional case avoids treating a rising model score as proof of improved real designs.

Common misconceptions and mistakes

Adaptive does not mean optimal. A poorly chosen feedback rule can waste effort faster than a simple baseline.

More samples near an interesting observation do not automatically provide representative population coverage. Nor does a small reported uncertainty guarantee that the model's uncertainty estimate is reliable.

Do not evaluate the strategy only on the cases it chose because they looked favorable. Keep the distinction between selection, observation and final assessment clear. Repeatedly checking results and stopping at a convenient point also affects which statistical conclusions are justified.

Business use

Consider adaptive sampling when observations are expensive and their information value varies. Define the benefit in operational terms, such as useful labeled cases or reliable design discoveries per unit of time and cost.

Compare the full adaptive procedure with a straightforward allocation under the same budget. Include feedback delays and coordination overhead, which can reduce the advantage of sequential decisions. Document the selection rule and its changes so stakeholders can understand why particular cases were observed and which claims the resulting data support.

Check your understanding

An adaptive inspection scheme deliberately samples a suspicious area more heavily. Can its raw overall defect fraction automatically be treated as the population defect rate?

Answer: No. The selection design changes which observations enter the sample. Population estimation requires an analysis that accounts for that design and its assumptions.

Related terms

Active learning
Sampling bias
Surrogate model
Importance sampling
sequential design
exploration and exploitation
uncertainty sampling
generative design

Learn next

Write an objective and a budget for a sampling problem. Compare a fixed allocation with a feedback-driven rule, specifying what observations trigger a change. Then explain separately how you would assess search success and estimate a population quantity from the collected data.

Sources

Sources: US EPA, sampling-design guidance and adaptive cluster sampling: https://www.epa.gov/sites/default/files/2015-06/documents/g5s-final.pdf Brookes and Listgarten, Design by adaptive sampling: https://arxiv.org/abs/1810.03714 Original ASAMS study, adaptive sequential sampling for surrogate modelling: https://pmc.ncbi.nlm.nih.gov/articles/PMC7571090/

Previous: Adaptive response rate Next: Adaptive sharpening