AI Glossary / AI and machine learning / Intermediate
Decoder
Definition
A decoder is the part of a model that turns an internal numeric representation back into an output people can use, such as words, images or predictions.
What it means
Modern models squeeze an input into a compact internal form: a list of numbers capturing what the input means for the task. Something must then expand that form into the answer, and that something is the decoder.
In a translation model the encoder reads the French sentence into numbers and the decoder writes the English sentence out, one word at a time, each word conditioned on the numbers and on what it has written so far.
In a chat model there is no separate encoder: the whole model is a decoder, reading the conversation and generating the reply token by token. Decoder-only is now the dominant design for language models.
In image and audio work, decoders run the same trick in other directions: turning numbers into a picture, a cleaned-up recording or a compressed file restored to full size.
For a business, the decoder is where quality is won or lost at the point the customer sees output: the same internal understanding can produce a clear, confident answer or a muddled one depending on how decoding is done.
How it works
Receive the internal representation from an encoder, or, in decoder-only models, build that representation from the input text directly.
Generate output one piece at a time: a word, an image patch or a sound fragment, feeding each piece back in so the next piece can depend on it.
At each step, score every candidate next piece and choose among them, greedily, by beam search or by sampling, a choice that trades reliability for variety.
Stop when the decoder emits an end marker or hits a length limit, and deliver the assembled output.
Train the decoder by showing it correct outputs and penalizing the difference between its predictions and the real next piece, over many examples.
Three practical examples
- Machine translation: the decoder writes the target-language sentence after the encoder has read the source.
- Chat assistants: a decoder-only language model generates each reply word by word from the conversation so far.
- Image captioning: a decoder turns the numbers describing a photo into a readable sentence about it.
Think of it this way
Think of a translator's two hands: one hand reads and compresses the meaning, the other writes it out in the new language. The decoder is the writing hand, and it can only write well if it keeps glancing back at what the reading hand understood.
The limit: the writing hand is also where style lives. Two decoders with the same understanding can produce a stiff or a fluent sentence, so decoding choices matter as much as understanding.
Worked example
decoding settings in a support assistant
Suppose a support tool drafts replies with a decoder-only model, and the team must choose how it picks words. The settings are illustrative.
Greedy decoding, always taking the single most likely next word, gives safe but repetitive replies that customers describe as robotic.
Sampling with some randomness gives livelier replies but occasionally a wrong fact stated cheerfully, which is worse than dull.
The team settles on beam search for factual answers, keeping the three best candidate openings, and light sampling for tone, then reviews a sample of drafts weekly.
Illustrative case study
A fictional retailer launched a product-description generator and was puzzled that every description opened with the same flourish.
The cause was greedy decoding: at each step the model took its safest word, and safe words repeat. Switching to constrained sampling varied the openings while a banned-phrase list kept claims factual.
The invented lesson: when generated text feels mechanical, the decoding strategy is the first dial to turn, not the model itself.
Common misconceptions and mistakes
- The decoder is the whole model. Sometimes it is, as in chat models, but in translation and image systems it is only the output half, paired with an encoder.
- Decoding is deterministic. Sampling-based decoding deliberately adds randomness, which is why the same prompt can give different replies.
- Better decoding fixes a weak model. Decoding shapes expression; it cannot supply understanding the encoder or training failed to build.
Business use
Ask vendors how their system decodes, greedy or sampled, because it changes how consistent and how risky the output is.
For customer-facing text, prefer conservative decoding plus human review over adventurous decoding that occasionally fabricates.
When generated output degrades, check decoding settings before assuming the model has changed; a settings tweak is a common silent culprit.
Keep a record of decoding settings alongside model versions; a reply that changed tone overnight is usually a settings change, not a new model.
People also ask
What is a decoder-only model?
A model with no separate encoder that reads input and generates output in one stack, the design behind most current chat systems.
What is beam search?
A decoding strategy that keeps several candidate partial outputs alive and picks the best complete one, trading speed for quality.
Can a decoder produce images?
Yes. Image generators decode from numbers into pixels, often step by step, the same generate-then-continue pattern as text.
Check your understanding
In a translation system, what does the decoder produce, and what does it use to produce it?
Answer: It produces the target-language sentence one word at a time, using the encoder's numeric representation of the source sentence plus the words it has already written.
Related terms
Learn next
Prompt Like a Pro explains how generation settings change the answers you get from language models, and how to control them: https://aiforbusinessandlife.com/academy/prompt-like-a-pro-clear-framework. [C3]
More in AI and machine learning
- Decision forest - A decision forest is a model made of many decision trees whose answers are combined, usually by voting or averaging, so the group predicts better than any...
- Decision tree - A decision tree is a model that predicts an answer through a sequence of yes-or-no style questions about the input, arranged like the branches of a tree...
- Decision tree pruning - Decision tree pruning is the practice of cutting branches off a grown decision tree so it stops memorizing the training data's noise and predicts new cases...
- Deep Q-Network - A Deep Q-Network is a reinforcement learning system that uses a neural network to estimate the value of each available action in a given situation, so the...
- Deep tomographic reconstruction - Deep tomographic reconstruction uses deep-learning methods to recover an internal image or volume from indirect measurements.
- Delta feature - A delta feature is an input built from the change between values, such as this month's sales minus last month's, so the model sees movement rather than only...
Sources
[S1] Russell and Norvig, AIMA glossary; [S2] Google, Machine Learning Glossary; [S8] Jurafsky and Martin, Speech and Language Processing (3rd ed. draft). Full URLs are in the source ledger. Verified 11 October 2026.
Previous: Declarative bias Next: Decoder models