A weighting that boosts words that are frequent in a document but rare overall.

Does word order matter?

Not in a plain bag of words. N-grams, counts of short word sequences, add a little order.

Is it the same as an embedding?

No. Embeddings represent meaning in dense vectors; bag of words only counts.

Two neighboring words counted as one feature, which restores some of the word order that plain bag of words loses.

AI Glossary / AI and machine learning / Beginner

Bag of words

Definition

A bag of words is a way of representing text by counting which words appear in it and how often, ignoring grammar and word order.

What it means

To use text in a model, it must be turned into numbers. The bag-of-words approach does this by making a vocabulary list and counting how many times each word occurs in a document.

The name comes from the image of tipping a text into a bag: the words are all there, but their order is lost. The sentences dog bites man and man bites dog look identical.

Despite that loss, it works surprisingly well for tasks where vocabulary matters more than structure, such as sorting emails into topics, spotting spam or measuring how similar two documents are.

Variants adjust the counts. Common words like the can be removed, and schemes such as TF-IDF give more weight to words that are distinctive for a document.

For a business, bag of words is the simple, cheap baseline for text analysis. Modern language models capture meaning and order, but a bag of words is often fast and easy to explain.

Bigrams are a common upgrade: counting pairs of neighboring words, such as not good, as single features restores a little of the lost order and often improves results at modest cost.

How it works

First, split each text into words, a step called tokenizing, and tidy them by lower-casing and removing punctuation. Often very common words are dropped.

Next, build a vocabulary from all the documents, a list of every distinct word. Each document becomes a row of counts with one column per vocabulary word.

A model, such as a simple classifier, learns which words point to which outcome: refund and late to complaints, invoice and receipt to billing.

The vectors are large and mostly zeros, since any one document uses a small part of the vocabulary. The method does not understand synonyms or negation: not good and good share a word.

TF-IDF is the most common weighting. It multiplies how often a word appears in a document by how rare it is across all documents, so distinctive words count more than common ones.

Three practical examples

Spam filters: counting words like winner or free to judge a message.
Support routing: sorting tickets into billing, delivery and technical queues.
Document search: finding documents with similar word counts.

Think of it this way

Think of emptying a pile of Scrabble tiles from a bag and counting the letters. You can learn a lot about the language from the counts, but nothing about the sentence order.

The limit: not a good day and a good, not a day look the same if you only count words, so meaning that depends on order is lost.

Worked example

sorting customer emails by topic

Suppose a small business wants to route emails to billing or delivery. It takes 200 past emails already sorted by staff.

It counts words in each email. Billing emails contain words like invoice, charge and refund; delivery emails contain parcel, tracking and late.

A simple model learns these patterns, and new emails are routed by their counts. The owner checks 50 new emails by hand and notes the share routed wrongly. Figures here are illustrative.

Mistakes cluster on emails like my parcel invoice is wrong, which mix topics, so the owner adds a manual review queue for low-confidence cases.

Illustrative case study

A fictional publisher uses bag of words with TF-IDF to recommend related articles. It is quick to build and cheap to run.

It struggles with articles that discuss the same idea in different words. The team later adds a language-model approach for meaning, but keeps the bag-of-words method as a baseline to prove the new one is better.

The invented lesson: simple methods are valuable baselines, and any fancier approach should beat them to justify its cost.

Common misconceptions and mistakes

Bag of words understands the text. It counts words without grasping meaning or order.
It is obsolete. It remains a strong, cheap baseline and appears inside many systems.
A bigger vocabulary is always better. Very large vocabularies make sparse, noisy data.

Business use

Use bag of words for quick text sorting, topic tagging and similarity checks when you have labeled examples and need something explainable.

Always compare a more advanced method against it. If the gain is small, keep the simpler one.

Keep preprocessing consistent between training and use, including lower-casing and the stop-word list, or the same email will be counted two different ways.

Check your understanding

Why might a bag-of-words model misread not good?

Answer: It counts the words separately and ignores order, so not good looks similar to good and the negation is lost.

Related terms

Learn next

Prompt Like a Pro teaches how language models handle meaning and instructions, which goes beyond word counting: https://aiforbusinessandlife.com/academy/prompt-like-a-pro-clear-framework. [C3]

Sources

[S1] Russell and Norvig, AIMA glossary; [S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: BADIR Next: Bagging