AI Glossary / AI foundations / Intermediate

Encoder-decoder model

Definition

An encoder-decoder model is a neural network design with two connected parts. The encoder reads an input and turns it into an internal representation. The decoder uses that representation to produce an output, one piece at a time. It is not a single fixed product: the same layout is used for translation, summarization and speech-to-text, and some modern language models use only one of the two parts.

What it means

Think of the model as a reader and a writer who share notes. The reader takes in everything that was given, such as a French paragraph, a long contract or a recording. The writer then composes a new piece, such as the English version, a summary or a transcript, guided by what the reader understood.

This split matters when the input and output are different in form or length. A ten-page report can become a three-line summary. A spoken sentence can become written text. The design lets the model handle each side of that change separately, while still linking them closely.

How it works

First, the input is broken into small units called tokens, and each token is converted into numbers. The encoder processes these numbers through several layers. In the common transformer design, each layer lets every token look at every other token, so the representation captures context. The result is a set of vectors that describe the input in a form the decoder can use.

The decoder then generates the output one token at a time. At each step it looks at what it has already written and also consults the encoder's representation, using a mechanism called cross-attention. It picks the next token, adds it to the output and repeats until it produces a stop signal. During training, the model is shown many pairs of inputs and correct outputs and adjusts its internal weights to make the correct output more likely.

Not every modern model uses both halves. Many popular chat-style language models use a decoder only, with the instructions and the answer treated as one continuous text. Encoder-only models also exist and are used mainly to understand or classify text rather than write it. Encoder-decoder models remain a natural fit when there is a clear input to be transformed into a clear output.

Three practical examples

• Finance team: A team receives supplier invoices as scanned images in many layouts. A system with an image encoder and a text decoder reads each scan and writes out the fields as structured text, such as supplier name, date and total. A person reviews low-confidence results before they reach the ledger.
• Retailer: An online store sells in several countries. A translation model with an encoder-decoder design reads each product description in the original language and writes it in the target language. Staff spot-check a sample of each batch for tone and terminology.
• Support desk: A support team stores long ticket threads. A summarization model encodes the whole thread and decodes a short recap, including the issue and the last action taken. Agents read the recap before replying instead of scrolling through the entire history.

Think of it this way

It works like an interpreter at a meeting who listens to a whole statement, holds its meaning in mind, then speaks it again in another language. The analogy stops working because the model holds numbers, not understanding, and it can state something fluent that is wrong with no awareness that it has done so.

Worked example

Suppose a translation model has an input of 20 tokens, and the encoder turns it into 20 vectors. The decoder is asked to write the output. Say the output needs 24 tokens.

Step 1: The encoder runs once on the 20 input tokens and produces 20 vectors.

Step 2: The decoder runs once per output token, so it runs 24 times.

Step 3: At each of those 24 steps, cross-attention compares the current position against all 20 encoder vectors. That is 24 x 20 = 480 comparisons in total for this one layer of cross-attention.

If the input grows to 40 tokens and the output to 48, the comparisons become 48 x 40 = 1,920. Doubling both lengths made the work four times larger. This is why long documents cost noticeably more to process than short ones, and why vendors often set limits on input and output length.

Illustrative case study

Fictional case: Harbor Lane Insurance, a small made-up insurer, wanted to turn long claim notes into one-paragraph summaries for adjusters. The team trialed a summarization model with an encoder-decoder design. Early results read well, but a reviewer noticed that some summaries stated a repair cost that did not appear in the notes. The model had produced a plausible-looking figure instead of copying the real one. The team changed the process in two ways. They added a step that checked every number and date in a summary against the source notes and flagged any mismatch. They also required an adjuster to approve each summary before it was filed. Flagged summaries dropped to a small share of the total, and adjusters said that reading summaries saved them time, as long as the checks stayed in place.

Common misconceptions and mistakes

Encoder-decoder models are the only kind of language model. Many current chat models are decoder-only, and some models are encoder-only. The design is chosen to suit the task.

The model understands the input the way a person does. It builds numerical patterns from training data. It can produce fluent output that is inaccurate, so important outputs need checking.

The encoder and decoder are separate tools that can be bought or swapped freely. They are trained together as one system, so changing one half usually means retraining or adjusting the other.

A longer input costs the same as a shorter one. Cost and speed generally rise with the length of both input and output, so long documents are more expensive to process.

Business use

To start, pick a task with a clear input and a clear output, such as summarizing tickets, translating product text or transcribing calls. Run the model on a few hundred real examples where you already know the right answer. Have the people who do the work today review the results. Compare the output with the source, not just with how fluent it sounds.

Measure accuracy against a checked sample, the share of outputs that need human correction, time saved per item and cost per item. Watch for errors that sound confident, since these are harder to catch than awkward wording. Costs include usage fees or computing, review time and the effort to monitor quality over time. Data privacy also matters when inputs contain personal or confidential information, so check how the provider stores and uses what you send.

People also ask

What is the difference between an encoder and a decoder?

The encoder reads and represents the input. The decoder generates the output from that representation, one token at a time.

Is ChatGPT an encoder-decoder model?

Chat-style assistants are generally built as decoder-only models, though the exact design of any given product varies and may not be published. Encoder-decoder models are more typical of translation and summarization systems.

When should a business use an encoder-decoder model?

It suits tasks that turn one clear input into a different clear output, such as translation, summarization or speech-to-text. Many tools hide the design, so the choice often comes down to testing results on your own data.

Check your understanding

What does the encoder do and what does the decoder do?

Answer: The encoder turns the input into an internal representation. The decoder uses that representation to write the output.

In the worked example, how many cross-attention comparisons occur for a 20-token input and a 24-token output?

Answer: 24 x 20 = 480 comparisons for one layer of cross-attention.

Why did Harbor Lane add a number-checking step?

Answer: The model sometimes wrote figures that were not in the source notes, so each number and date was checked against the original.

Do all modern language models use both an encoder and a decoder?

Answer: No. Many use only a decoder and some use only an encoder.

Related terms

• Transformer
• Attention mechanism
• Tokenization
• Large language model
• Sequence-to-sequence model
• Neural network
• Hallucination
• Fine-tuning

Learn next

Try a small test on a task from your own work, using a tool that translates or summarizes, and check the output against the source. Then read the entries on the transformer and the attention mechanism to see how the encoder and decoder look at different parts of the text. After that, read about hallucination to understand why checking outputs matters.

More in AI foundations

• Deep learning - Deep learning is a branch of machine learning that learns representations through multiple layers of computation.
• Direct preference optimization (DPO) - Direct preference optimization (DPO) is a method for adjusting a language model so its answers match human preferences, using pairs of responses where...
• Embodied AI - Embodied AI is artificial intelligence that has a physical body, such as a robot, a vehicle or a machine with sensors and moving parts, and learns or acts...
• Expert system - An expert system is a computer program designed to apply specialized knowledge to problems within a defined domain.
• F1 score - The F1 score is a single number between 0 and 1 that summarizes how well a classification model performs by combining precision and recall using their...
• Feature - A feature is one measurable input field a model learns from, such as a customer's age, an order's total or the hour a call arrived.

Sources

Vaswani et al., Attention Is All You Need: https://arxiv.org/abs/1706.03762 Google, Machine Learning Glossary: https://developers.google.com/machine-learning/glossary Jurafsky and Martin, Speech and Language Processing: https://web.stanford.edu/~jurafsky/slp3/ The sources were consulted for the definition of the encoder and decoder roles and the cross-attention mechanism; the examples, analogy, worked example and case are original educational illustrations. Checked 5 Nov 2026.

Previous: Encoder-decoder Next: Encoder models