Must the missing material be one word?

No. Related tasks can hide multiple tokens or spans.

Is cloze the same as next-token prediction?

No. A cloze input can reveal context on both sides of a gap.

Can two different answers both be acceptable?

Yes, depending on the text and scoring rubric.

AI Glossary / Natural language and speech / Advanced

Cloze task

Definition

A cloze task asks a person or model to supply missing material in a text using the available context. In a familiar version, one word is removed from a sentence and the respondent fills the blank. In natural language processing, related tasks hide tokens or spans and use their recovery for training or evaluation.

What it means

Consider “Nadia put the letter inside the ___ before posting it.” The surrounding words constrain the missing item, but they do not necessarily identify one uniquely correct string. Envelope is a natural completion, while the precise original wording depends on the source text.
A cloze task therefore combines two design decisions: what information the respondent sees and what counts as a correct answer. Recovering the original word, selecting from a list, and producing any acceptable completion are different objectives. A score is interpretable only when these rules are stated.
The task can involve context before and after the blank. This differs from ordinary next-token prediction, which does not expose later words in the sequence.

How it works

A task designer chooses text, selects positions or spans to hide, and constructs the input presented to the respondent. The missing material becomes a target, an accepted-answer set, or the basis for a judgment rubric. The choice of blanks controls what the task tests: grammar, vocabulary, factual associations, or broader contextual interpretation.
In masked language modeling, a training procedure derives targets from the original text itself. A model predicts vocabulary items at selected positions, and a loss rewards probability assigned to the original targets. BERT's masked-language objective is a prominent example inspired by cloze tasks. It is one implementation, not a definition that all cloze exercises must follow.
Tokenization matters. One written word may correspond to several model tokens, and a missing span may require a different output format from a single missing token. Evaluation should also specify whether it scores each blank separately or requires an entire set of completions to be correct.

Three practical examples

A language-learning exercise removes selected function words and asks learners to restore forms that fit the sentence's grammar.
A representation-learning project hides parts of existing text and trains a model to predict the withheld items from their context.
An evaluation team constructs paired fill-in-the-blank prompts to examine whether a model responds to a changed contextual clue, while checking for memorized examples and ambiguous answers.

Think of it this way

Think of reconstructing a missing piece of a message from the pieces still visible. The surrounding pieces help, but several replacements may fit. Whether the task rewards the original piece or any compatible replacement must be decided before judging the answer.

Worked example

Consider an original exercise with four single-blank items. Suppose the source-text targets, in order, are parcel, quickly, gate, and Tuesday. A respondent supplies package, quickly, door, and Tuesday.
Under an exact-target rule with no synonym allowance, only the second and fourth responses match. The score is 2/4=50%. The fact that package might fit one sentence does not change that rule; it simply shows why exact recovery and contextual acceptability should not be confused.
Now suppose the exercise designer had established these accepted-answer sets in advance: {parcel, package}, {quickly, rapidly}, {gate}, and {Tuesday}. Against that separate rubric, the first, second, and fourth responses pass. The score becomes 3/4=75%. This change measures a different acceptance policy, not an improvement in the respondent's answers. These stipulated sets are not universal synonym claims: acceptability depends on the sentence and intended meaning.
A probability-based training measure asks another question. Suppose a model assigns probabilities 0.4, 0.8, 0.25, and 0.5 to the four original targets. Their mean negative log probability is [−ln(0.4)−ln(0.8)−ln(0.25)−ln(0.5)]/4, approximately 0.8047 nats.
This loss uses probabilities of the original targets even where the displayed top answer differs. It cannot be recovered from the 50% exact-match score alone. Two models can select the same answers while assigning different target probabilities and therefore receiving different losses.
Finally, if all four blanks formed one exercise scored only when every answer matched the original, this respondent would receive zero for that exercise. Per-blank accuracy and all-blanks-correct scoring use different units. Reports should identify which unit they count.

Illustrative case study

Fictional example: ReadBench evaluates an assistant using missing words in internal training passages. Reviewers initially disagree about several responses because one person expects the original wording and another accepts plausible alternatives.
The team documents two separate evaluations: exact recovery and contextual acceptability under a reviewed rubric. It also checks that the passages used for evaluation were not copied into the model's task-specific training set. The scenario illustrates clearer measurement, not a claim that either score establishes general comprehension or guarantees freedom from memorization.

Common misconceptions and mistakes

A plausible completion is not necessarily the source's original word, and the original word is not necessarily the only linguistically acceptable completion. Both facts matter when interpreting errors.
Cloze performance also does not prove broad understanding. A model may exploit local syntax, frequent phrases, or prior exposure. Removing a word that is repeated elsewhere in the visible passage can make an item easier than intended. Conversely, hiding too much context may make it impossible to identify a unique target fairly.
Not every blank corresponds to one model token, and not every cloze implementation uses the same masking policy. Training settings should be described explicitly rather than inferred from the task's name.

Business use

Use cloze tasks for targeted probes, training exercises, or model development when their scoring matches the intended question. Document blank selection, available context, accepted answers, tokenization, and scoring units. Inspect ambiguous items and compare performance on new material. A strong result on missing-word recovery should be supplemented with evaluation of the actual workflow the organization wants to support.

Check your understanding

What is the exact-target score in the example?

Answer: Fifty percent.

Why does the accepted-set score rise to 75%?

Answer: Package is accepted by the second rubric.

Does identical accuracy imply identical training loss?

Answer: No. Target probabilities can differ.

Related terms

Masked language modeling
Self-supervised learning
Cross-entropy
Exact match
Data leakage
token prediction
contextual representation
span corruption

Learn next

Create a short cloze exercise and define its answer policy before testing it. Compare per-blank accuracy, whole-exercise scoring, and a probability-based measure, noting what each result does and does not establish.

Sources

Devlin and colleagues, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019, cloze-inspired masked language modeling: https://aclanthology.org/N19-1423.pdf Jurafsky and Martin, Speech and Language Processing, archived chapter on contextual embeddings, cloze prediction: https://web.stanford.edu/~jurafsky/slp3/old_dec20/11_jan122022.pdf

Previous: Cloudlet Next: Cluster assumption