Is contrastive search random sampling?

The standard version chooses the best score deterministically; stochastic extensions are separate choices.

Does it require contrastive training?

The search rule can be applied without that training, but its effectiveness depends on the model's representation geometry and should be evaluated.

What does top-k mean here?

It defines the candidate shortlist for rescoring, rather than randomly sampling a token from that shortlist.

AI Glossary / Natural language and speech / Advanced

Contrastive search

Definition

Contrastive search is a text-decoding method that selects a next token by balancing its model probability against its hidden-state similarity to previous context tokens. It evaluates a shortlist of likely candidates to discourage repetitive generation while retaining a preference for plausible continuations.

What it means

A language model supplies probabilities, but a decoding method decides how to turn them into text. Greedily choosing the highest-probability token can sometimes contribute to repetitive output. Contrastive search adds another consideration: whether a candidate's contextual representation closely resembles earlier representations.

The method described by Su and colleagues first restricts attention to the top-k candidates. It then chooses the candidate with the best combined probability-and-similarity score. The standard procedure is deterministic, subject to implementation and tie handling.

This is distinct from contrastive decoding that compares outputs from an expert and an amateur model. It is also distinct from contrastive training: a model can be trained with a contrastive objective, while search is the procedure used when producing text.

How it works

At each step, obtain the model's next-token probabilities and keep the k most probable candidates. For each candidate, evaluate the hidden state that would result if it were appended to the current context.

Compare that candidate state with the stored states of earlier context tokens using cosine similarity. Take the largest similarity as the candidate's degeneration penalty.

The original selection score is (1 − alpha) × probability − alpha × penalty. The probability here is the model's probability, not its logarithm. Choose the highest-scoring candidate and repeat from the extended context.

Alpha controls the tradeoff, while k controls which candidates are considered. A token outside the shortlist cannot win regardless of its potential diversity. Candidate evaluation requires additional computation, and the usefulness of the similarity signal depends on the model's representations.

Three practical examples

A text-generation experiment compares greedy output with contrastive search on prompts that tend to produce loops.
A researcher varies k and alpha while assessing coherence, necessary repetition and runtime separately.
A model developer distinguishes hidden-state-based search from an unrelated method that contrasts two models' token scores.

Think of it this way

Think of choosing the next discussion point from a shortlist of plausible options while checking whether each mostly repeats something already said. A sensible choice balances staying on topic with adding something useful.

The analogy explains the tradeoff, but hidden-state cosine similarity is not a direct measure of conversational usefulness or factual novelty. A different representation can still lead to an unhelpful or incorrect statement.

Worked example

Suppose three shortlisted tokens A, B and C have model probabilities 0.50, 0.30 and 0.15. Other vocabulary items share the remaining 0.05 probability. Their maximum contextual cosine similarities are 0.90, 0.40 and 0.20.

With alpha = 0.4, A scores 0.6 × 0.50 − 0.4 × 0.90 = −0.06. B scores 0.18 − 0.16 = 0.02. C scores 0.09 − 0.08 = 0.01. Contrastive search selects B, while greedy decoding would select A.

These scores need not be probabilities, and negative scores are allowed. The highest score determines the choice.

If alpha becomes zero, the score reduces to model probability and A wins. If k is one, only the highest-probability candidate is available, so the penalty cannot make the procedure select a different token. The example illustrates the rule, not a real model's behavior.

Illustrative case study

The fictional PineDraft team tests a writing assistant that sometimes repeats the same sentence structure. It compares decoding methods while keeping the underlying model and prompts fixed.

A configuration with a strong similarity penalty reduces some repeated phrases in its test outputs but also changes wording that needs to remain consistent, including names and technical labels. Reviewers therefore score those cases separately from open-ended prose.

The team records k, alpha, model version and generation length alongside runtime and human judgments. This invented case illustrates a controlled evaluation. It does not report a measured improvement or imply that a decoding change makes generated claims factually reliable.

Common misconceptions and mistakes

Contrastive search must use two language models. The method here compares candidate hidden states with context states from one model.
The penalty simply bans repeated words. It operates on contextual representations and still allows repetition when the combined score favors it.
A more diverse continuation is necessarily true. The scoring rule does not check external evidence or factual correctness.

Business use

Contrastive search is relevant when investigating repetitive output in a model whose hidden states are accessible. It can be evaluated as an inference-time choice, with attention to the additional work of candidate evaluation.

Compare it with existing decoding settings on realistic tasks, not only on prompts chosen to expose repetition. Include exact terminology, quotations and structured text where repetition may be desirable. Track latency and memory as well as output quality. Keep factual verification separate: a fluent, varied continuation can still contain unsupported information. An API that returns only final text may not expose what this method needs.

Check your understanding

A candidate has the lowest similarity penalty but is outside the top-k shortlist. Can it be selected by the standard procedure at that step?

Answer: No. The method only rescores and selects among the shortlisted candidates.

Related terms

Decoding
Greedy search
Top-k sampling
Cosine similarity
Contrastive Learning
hidden state
degeneration
contrastive decoding

Learn next

Read greedy decoding and cosine similarity. Recalculate the worked scores for several alpha values, identifying the point where the preferred candidate changes and explaining why that is not a factual-quality threshold.

Sources

Yixuan Su and colleagues, A Contrastive Framework for Neural Text Generation, 2022, Section 3.2: https://arxiv.org/html/2202.06417v2

The original search equation, top-k restriction and deterministic-method discussion were reviewed. The numerical comparison is original. Historical experiment claims are not extrapolated to every model, language or current deployment.

Previous: Contrastive Learning Next: Control as inference