Is BERT bidirectional?

Yes. BERT reads the whole input at once and predicts masked words from both sides.

Can a bidirectional model generate text?

Not in the usual live, word-by-word way. Generation needs a one-directional model.

What is the opposite of bidirectional?

Unidirectional or autoregressive, reading only left to right.

Does bidirectional processing cost more at prediction time?

The whole input is processed at once, so cost per document is higher than streaming, but for tagging and search the difference is small.

AI Glossary / AI and machine learning / Intermediate

Bidirectional

Definition

In AI, bidirectional describes a model that reads a sequence in both directions at once, from start to end and from end to start, so each word or item is understood with context from both sides.

What it means

The meaning of a word often depends on what comes after it. In the sentence the bank closed after the flood, the word bank is only clear once you read the rest. A one-directional reader meets bank too early; a bidirectional one sees the whole sentence.

Bidirectional models are built for understanding, not writing. They power tasks such as tagging, classification, search and question answering, where the full input is available at once.

The best-known bidirectional language models are the BERT family. They are trained by hiding words and predicting them from both sides, which teaches rich representations of meaning.

They cannot be used in the usual way to generate text word by word, because at writing time the future words do not exist yet. Generative chatbots use one-directional models for that reason.

For a business, this explains the quiet split in AI tools: the models behind search, tagging and analysis are often bidirectional encoders, while chatbots and writers are usually one-directional generators.

A simple way to feel the difference: ask each model type to judge the sentence the film was hardly boring. The one-directional reader meets boring late and leans negative; the bidirectional reader sees hardly and boring together and leans positive.

How it works

The input is read by two passes, one left to right and one right to left, and the two views are combined so every position carries information from both sides.

During training, words are masked out and the model predicts each hidden word using everything around it, on both sides. This is called masked language modeling.

The result is a representation, a set of numbers per word, that captures meaning in context. Those representations feed smaller models that tag, classify or match.

Because both directions must be available, the full text is processed at once rather than streamed, which shapes the cost and latency of running such models.

Older bidirectional designs used two recurrent networks running in opposite directions. Modern ones use attention layers, where every position can look at every other position, which achieves the same whole-sentence view more efficiently.

Three practical examples

Sentiment analysis, where the phrase hardly bad needs both sides of hardly to be read correctly.
Named-entity recognition, deciding whether Jordan is a person, a country or a river from the surrounding sentence.
Search relevance, matching a query to documents by meaning rather than shared keywords.

Think of it this way

Imagine reading a sentence through a moving slot that only ever shows what you have already read. Guessing the next word is hard. Now imagine the whole sentence printed in front of you. Understanding each word is easy. The slot reader is one-directional; the full page is bidirectional.

The catch: the full-page reader cannot tell you what comes next, because for it there is no next; the page is complete.

Worked example

tagging tickets with a two-way reader

Suppose a firm tags support tickets by topic. The sentence I want to dispute this charge on my card confuses a one-directional tagger, which leans toward payments before reading the whole line.

A bidirectional encoder reads the ticket as a whole. Words such as dispute, seen alongside charge and card, steer it toward a disputes queue. The examples and results are illustrative.

The firm compares both models on 300 tickets tagged by hand and finds the bidirectional one clearly better on short, ambiguous tickets.

It keeps the bidirectional model for tagging, but still uses a generative model to draft replies, because that is a writing task.

Illustrative case study

A fictional legal-tech company builds a clause extractor for contracts. Its first model reads left to right and keeps misreading clauses whose key words come late.

Switching to a bidirectional encoder cuts those errors, because every clause is read as a whole. The team notes that extraction is now accurate but the model cannot draft clauses, and pairs it with a generative tool for drafting.

The invented lesson: match the direction of the model to the direction of the task, reading for understanding, one-way for writing.

Common misconceptions and mistakes

Bidirectional means the model writes text in both directions. It reads in both directions; writing is still one word after another.
Bidirectional is always better. It is better for understanding whole inputs, and unusable for live word-by-word generation.
It is twice as expensive as one-directional. The cost structure differs, but the figure is not simply double; it depends on the model and task.

Business use

When evaluating tools for search, tagging or analysis, know that bidirectional encoders are the usual engine, and ask vendors what sits underneath.

Do not expect a bidirectional model to write. If a task needs fluent generation, it needs a generative model, whatever the marketing says.

Check your understanding

Why can a bidirectional model not predict the next word of a live stream in the usual way?

Answer: It needs words from both sides of each position, and the words after the current one do not exist yet in a live stream.

Related terms

Learn next

AI for Business Owners helps you match model types to business tasks such as search, tagging and drafting: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Bidiagonal matrix Next: Bidirectional associative memory