Must the model be Bayesian?

No. Batch active learning includes Bayesian and non-Bayesian strategies.

Can annotators work in parallel?

Yes. Grouped requests can support that arrangement, although label quality and adjudication still require attention.

Does the model create the true labels itself?

Usually it requests them from an annotator or another information source. Automatically generated labels introduce separate reliability assumptions.

AI Glossary / Probabilistic machine learning / Advanced

Batch active learning

Definition

Batch active learning is an approach in which a learning system selects a group of examples for labeling before receiving those labels and updating its model. The aim is to make useful labeling requests while accommodating parallel annotation, costly retraining or other practical constraints.

What it means

An active learner chooses which missing labels to request. In a sequential workflow, it can learn from one returned label before choosing the next example. In a batch workflow, several choices are committed before that feedback arrives.
This creates a trade-off. Batches can make annotation easier to schedule, but their members may overlap in the information they provide. A good batch is therefore not necessarily the collection of individually highest-scoring examples. Batch active learning describes a workflow and selection problem, rather than one universal algorithm.

How it works

Begin with labeled training data, an unlabeled candidate pool and a labeling budget. Fit a model, then use an acquisition strategy to select a batch. Obtain labels, check their quality, add them to the training data and update the model before the next round.
Selection may combine uncertainty, diversity, representativeness or estimated information gain. Joint scoring considers candidates together; simpler approaches rank them individually. BatchBALD is one particular Bayesian approach that scores information about model parameters from a group of labels. It is not a synonym for all batch active learning. Evaluation should include labeling effort and downstream performance.

Three practical examples

A customer-support team could send a group of messages to annotators each morning. Selecting varied unresolved message types could be more useful than repeatedly selecting near-identical complaints.
A manufacturer could arrange inspection of several selected images during an expert's scheduled review session. The acquisition process would need to consider both likely learning value and the expert's available time.
A document-processing team could request labels for a batch of unusual layouts. It would still need a separate evaluation sample that reflects the documents expected in deployment, rather than testing only on the actively selected cases.

Think of it this way

Imagine preparing a list of questions before an interview. If you could ask one question and immediately revise the next, you would adapt to each answer. Preparing the whole list saves coordination, but several questions might uncover the same fact. A thoughtful list covers useful uncertainties without assuming every additional question provides fresh information.

Worked example

Use an original toy model with four equally plausible hypotheses, H1 through H4. There are three candidate examples, A, B and C, each requiring a binary label. Assume the labels are deterministic under each hypothesis and annotators return them correctly.
The predicted labels for (A,B,C) are H1: (0,0,0), H2: (0,0,1), H3: (1,1,0) and H4: (1,1,1). Before labeling, uncertainty over the hypotheses is log2(4)=2 bits. Each individual candidate has equally likely labels 0 and 1, so each label supplies one bit of information about the hypothesis in this toy setup.
Suppose the labeling budget permits two examples. A purely individual score treats every pair as tied. However, A and B always have identical labels under the hypotheses. Receiving both leaves two possible hypotheses, giving only one bit of total information. If their labels are both zero, H1 and H2 remain equally possible.
By contrast, the label pair (A,C) has four equally likely outcomes, and each identifies a different hypothesis. That batch supplies two bits. Thus two individually informative requests can be jointly redundant. The conclusion depends on the stipulated hypothesis distribution and noiseless labels; it is not a promise that practical selection will identify a real model after two annotations.

Illustrative case study

Fictional software provider Alder Desk wants better routing of support messages. It starts with a small labeled set and reserves an independently sampled evaluation set. Its annotators can review messages together once per day, making batches more practical than a request after every label.
The first proposed batch contains many slight variations of the same billing question. The team adds a diversity check and compares its selection procedure with random batches at the same annotation budget. It records annotation time, disagreement and routing performance across rounds. This fictional case shows how to evaluate the workflow; it does not claim that diversity checks always improve accuracy or that any labeling savings were measured.

Common misconceptions and mistakes

“A training mini-batch is an active-learning batch.” A training mini-batch is used for an optimization update; an acquisition batch identifies examples whose labels are being requested.
“The most uncertain examples always form the best group.” Their information can overlap, as the worked example shows.
“Diversity alone guarantees useful labels.” A highly unusual example may be irrelevant to the intended population or difficult to label consistently.
“Large batches are always more efficient.” They reduce opportunities to adapt within a round and can commit substantial labeling effort before feedback arrives.
“Selected labels provide an unbiased performance test.” The selection procedure shapes the labeled sample; evaluation needs an appropriate independent design.

Business use

Use this approach when labeling is expensive and the organization can update a model over multiple rounds. Set a budget in actual annotation time or cost where examples differ substantially in difficulty. Track acquisition computation, retraining delays and expert availability as well as label counts. Compare with a straightforward random-selection baseline, and decide in advance what evidence would justify continuing, changing the strategy or stopping the labeling exercise.

Check your understanding

Why does selecting A and B yield only one bit in the example?

Answer: their labels convey the same split between hypotheses.

What advantage does the pair A and C have?

Answer: its four possible label pairs distinguish all four hypotheses in the stipulated model.

What feedback opportunity is lost within a batch?

Answer: the system cannot use the actual returned labels from earlier selections to revise the remaining committed requests.

Related terms

Active learning
Query by committee
BatchBALD
Information gain
Sampling bias
acquisition function
uncertainty sampling
annotation budget

Learn next

Compare sequential and batch selection on a small labeled dataset treated as an unlabeled pool. Keep evaluation separate, record annotation costs and examine whether selected examples repeat the same information.

Sources

Settles, B. Active Learning Literature Survey, Technical Report 1648, updated 2010, especially Section 6.1: https://burrsettles.com/pub/settles.activelearning.pdf Kirsch, A., van Amersfoort, J., and Gal, Y. BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning, NeurIPS 2019: https://papers.neurips.cc/paper/8925-batchbald-efficient-and-diverse-batch-acquisition-for-deep-bayesian-active-learning.pdf The hypothesis example and fictional case study are original illustrations.

Previous: Batch Next: Batch gradient descent