Does decoder-only exclude every external encoder?

No. A multimodal application may attach other encoders to a decoder-only language backbone.

Can training process several positions together?

Yes, with masks and shifted targets preserving the prediction constraint.

Does the model always generate the same answer?

That depends on generation settings and implementation, not just the architecture label.

AI Glossary / Statistics and probability / Intermediate

Decoder-only model

Definition

A decoder-only model is a neural sequence model built around a decoder stack without a separate encoder stack for the text input. In its common transformer language-model form, it processes a prompt and generates a continuation using causal self-attention and next-token prediction. The prompt and generated answer occupy one evolving token sequence. The architecture describes information flow, not a guarantee of intelligence or accuracy.

What it means

The word only distinguishes this arrangement from an encoder-decoder architecture. It does not mean that the model performs only one task. A decoder-only language model can express many tasks as text continuation, depending on its training and the surrounding application.

For example, a prompt can contain an instruction, a passage, and a request for a summary. The model continues that sequence with answer tokens. It does not require a separate text encoder to read the passage before a different decoder begins producing the answer.

Modern systems may combine a decoder-only language backbone with image encoders, audio components, retrieval, or tools. The label for the language backbone should not be mistaken for a complete inventory of the application.

How it works

Input token identifiers are mapped into vectors, with position information supplied through the model's chosen mechanism. A stack of transformer blocks updates those representations using attention and other transformations. A language-model output layer converts the resulting hidden state into scores for possible next tokens.

In the standard causal arrangement, position i may attend to positions up to and including i, but not later positions. Its output is used to predict the following token. During training, shifted targets ensure the model is not simply shown the answer it must predict.

At generation time, the prompt is processed and a next token is selected. That token is appended, and the process continues until a stopping condition is met. Cached attention information can avoid repeating some calculations for earlier tokens, but it uses memory. Specific attention patterns, cache designs, and training objectives vary across implementations.

Three practical examples

A writing application places a style instruction and an outline into the prompt, then generates a proposed continuation.
A document assistant includes retrieved passages in the context and asks the model to produce an answer with evidence that must still be checked.
A code assistant supplies preceding code and requests a completion, followed by tests that evaluate behavior rather than fluency.

Think of it this way

Think of a growing notebook containing the question and answer in progress. Each new line uses the preceding pages. The causal mask specifies which earlier positions can inform each token representation.

Worked example

Consider an original four-position teaching sequence with input tokens A, B, C, and D. A standard causal attention mask allows position one to attend only to A; position two to A and B; position three to A, B, and C; and position four to all four input tokens.

The number of permitted query-key pairs is 1+2+3+4 = 10 out of 16 possible pairs. The six pairs above the diagonal are blocked. This counts allowed connections in one attention head; it does not measure the total computation or memory used by a complete implementation.

If the training targets are B, C, D, and E respectively, the representation at position three predicts D from A, B, and C. It cannot attend to input position four, which contains D, because that would reveal its target. The final representation predicts E from the complete four-token prefix.

Suppose the assigned probabilities of the correct targets at these positions are 0.5, 0.25, 0.5, and 0.8. Their mean negative log probability is −[ln(0.5)+ln(0.25)+ln(0.5)+ln(0.8)]/4, approximately 0.74893 nats. This invented loss calculation illustrates target alignment; it is not a performance claim about a real model.

Illustrative case study

In this fictional case, Meridian Manuals builds an assistant for internal equipment documentation. Its decoder-only language model receives a question followed by selected approved passages and an instruction to explain the relevant procedure.

The team discovers that some long requests leave insufficient room for the answer or cause relevant passages to be omitted. It revises context selection and measures answer coverage, unsupported claims, and response time on representative cases. Staff review consequential instructions against the source documents.

The invented case shows that having source text in the prompt does not prove the model will use it correctly.

Common misconceptions and mistakes

Decoder-only does not mean the model cannot summarize, classify, or translate. These tasks can be formulated as generation, although another architecture may perform better for a particular need.

Causal attention does not prevent the final prompt position from using earlier prompt tokens. It prevents a position from using later tokens when forming the relevant representation. This distinction is especially important during parallel training.

The architecture does not provide unlimited memory, automatic access to current information, or guaranteed factual grounding. A cache is a computational device, not a verified knowledge store. Also, the term does not specify one parameter count, tokenizer, context length, or instruction-tuning method. Those properties need separate documentation.

Business use

Businesses should evaluate a decoder-only model as one candidate within a complete task workflow. Relevant measures include answer quality, unsupported content, context handling, latency, operating cost, and the ease of validating the output.

Use realistic prompts and documents, including long or contradictory cases. Track model and configuration versions, and test whether generated structured outputs both meet their schema and contain correct values. Architecture labels can help explain system behavior, but product decisions should follow evidence from the intended workload. A strong result on short demonstrations may not transfer to lengthy documents or unfamiliar terminology.

Check your understanding

Check your understanding: Why can position three not attend to D in the example?

Answer: D is its next-token target at position four.

How many attention pairs are allowed?

Answer: Ten.

What does only identify?

Answer: The absence of a separate text encoder stack in the described language architecture.

Does including evidence guarantee an accurate answer?

Answer: No.

Related terms

Decoder models
Transformer
Context window
causal self-attention
token embedding
language-model head
key-value cache
encoder-decoder model
autoregression

Learn next

Trace shifted targets through a small causal model. Then compare prompt processing with token-by-token generation and examine the memory implications of attention caching.

Sources

Sources: Jurafsky and Martin, Speech and Language Processing, chapter 9, architecture comparison and causal masking, https://web.stanford.edu/~jurafsky/slp3/9.pdf. Hugging Face Transformers glossary, https://huggingface.co/docs/transformers/glossary. The mask count, loss calculation, and fictional document-assistant case are original.

Previous: Decoder models Next: Decoding