AI Glossary / AI foundations / Foundation

Data annotation

Definition

Data annotation is the process of adding labels, tags or notes to raw data such as text, images, audio, video or records, so that a machine learning system can learn from it or be tested against it. It is not the same as data cleaning, which fixes errors, or data collection, which gathers the raw material. The word is often used interchangeably with data labeling, though some teams use annotation for richer markup than a single label.

What it means

Raw data does not tell a model what it is looking at. A folder of photos is only pixels until someone marks which photos show a damaged parcel and which show an intact one. Annotation is the human (and sometimes automated) step that turns raw material into examples with known answers.

For a business, annotation is often the largest hidden cost of a supervised AI project. The quality of the labels sets a ceiling on how well the system can perform. Unclear rules, rushed work or disagreement between labelers all show up later as errors in the model.

How it works

A project starts by deciding what the model should predict, then writing annotation guidelines. The guidelines define each label, give examples and edge cases, and say what to do when an item is unclear. A sample of the data is then given to annotators, who may be staff, subject experts, contractors or crowd workers, using a tool that shows the item and records the label.

Quality control follows. The same items are often given to two or more annotators, and the team measures how often they agree. Items where people disagree are reviewed, and the guidelines are fixed if the rule was the problem. A trusted subset, sometimes called a gold set, is labeled by experts and used to check annotator accuracy over time.

Many teams now use a model to pre-label data and ask people to correct it, which is faster than labeling from scratch. The risk is that reviewers accept wrong suggestions without looking closely. Practice varies on how much of the work is automated, but a human check on a sample is common.

Three practical examples

• Finance team: An accounts payable group wants a model that reads invoices. Staff mark the supplier name, invoice date, total and tax amount on 2,000 sample invoices. The marked fields become the answers the model learns to find.
• Retailer: An online shop wants to tag product photos by color and style. Annotators assign each photo one color label and one style label from a fixed list. Search results then improve because every item carries consistent tags.
• Support desk: A help desk wants incoming messages sorted into billing, delivery, returns and other. Agents label a few thousand past tickets by topic and urgency. The labeled tickets train a routing model and also serve as a test set for checking it.

Think of it this way

Think of annotation like marking a stack of practice exam papers with the correct answers before handing them to a student. The student can only learn what the answer key teaches. The analogy stops working because a student can question a wrong answer key, while a model simply absorbs whatever errors the labels contain.

Worked example

A small lender wants to label 1,000 customer emails as complaint or not complaint. Two annotators each label all 1,000 emails. They agree on 880 and disagree on 120.

Raw agreement is 880 divided by 1,000, which is 88 percent. That sounds good, but the team reviews the 120 disagreements and finds that 90 of them come from one unclear rule: whether a polite request for a refund counts as a complaint. They add a sentence to the guidelines saying that it does.

The annotators relabel the 90 items and now disagree on only 30 emails. Agreement rises to 970 divided by 1,000, which is 97 percent. The 30 remaining items go to a senior reviewer for a final decision. Cost check: if each label takes 30 seconds, one pass over 1,000 emails is 30,000 seconds, or about 8.3 hours. Two passes cost about 16.7 hours before the review work.

Illustrative case study

Fictional case: Harbor Lane Logistics, a mid-sized freight broker, wanted a model to flag shipping documents with missing signatures. A team of six clerks labeled 5,000 scanned documents over three weeks. The first model tested poorly, catching only about half of the problem documents.

A review showed the clerks had followed different habits. Some marked a stamp as a signature and others did not, and a few labeled blurry scans as missing when they simply could not read them. The labels contradicted each other, so the model had no consistent pattern to learn.

The operations lead wrote a one-page guide with pictures of acceptable signatures, stamps and initials, and added an unreadable category for poor scans. Two clerks relabeled a sample of 800 documents under the new rules, and a senior clerk settled every disagreement. The model was retrained on the cleaned labels and improved noticeably. The team kept a small expert-labeled set to test future versions.

Common misconceptions and mistakes

More labeled data always means a better model. Volume helps only if the labels are consistent and correct, and a smaller, carefully checked set often beats a large noisy one.

Annotation is a one-time task finished before the project starts. Data, products and customer behavior change, so labels and guidelines usually need regular review and refreshing.

Automated labeling removes the need for people. Machine-generated labels are quick, but they repeat the mistakes of the model that produced them unless people check a sample.

Any annotator can label any data. Medical, legal or technical material often needs subject knowledge, and untrained labelers can introduce confident but wrong labels.

Business use

Start small. Pick one clear task, write short guidelines with examples, and label a pilot batch of a few hundred items with two people. Measure agreement between annotators, review the disagreements, and fix the rules before scaling up. Keep a trusted expert-labeled set aside for testing, and do not use it for training.

Track agreement rates, error rates against the expert set, time per item and cost per labeled item. The main risks are inconsistent labels, bias in who or what gets labeled, and privacy: the data may contain personal or confidential information, so check access controls and any contract terms with outside labeling vendors. Costs include staff time, tool licenses, vendor fees and the ongoing effort to keep labels current.

People also ask

What is the difference between data annotation and data labeling?

In most business contexts the terms mean the same thing. Some teams use labeling for assigning a single category and annotation for richer markup, such as drawing boxes around objects or highlighting phrases.

Who does data annotation?

It may be internal staff, subject experts, outside vendors or crowd workers, depending on the task. Specialist material usually needs experts, while simple tasks can use trained general workers.

Do I need data annotation to use a generative AI tool?

Usually not, because the provider already trained the model. You may need it if you want to fine-tune a model on your own data or build a test set to check how well a tool performs on your work.

Check your understanding

In the worked example, what was the raw agreement before the guideline fix?

Answer: 880 of 1,000 emails, or 88 percent.

Why did agreement improve after the team added one sentence to the guidelines?

Answer: Most disagreements came from one unclear rule about refund requests, so clarifying it removed 90 of the 120 conflicts.

Why keep an expert-labeled set separate from the training data?

Answer: It gives an honest test of the model and of annotator accuracy, because the model has not learned from those items.

What risk comes with letting a model pre-label data?

Answer: Reviewers may accept wrong suggestions without checking, so the model's errors become part of the training labels.

Related terms

• Training data
• Supervised learning
• Ground truth
• Data quality
• Fine-tuning
• Evaluation
• Synthetic data
• Human in the loop

Learn next

Try labeling 50 items from your own work with a colleague and compare answers, since the disagreements show where your rules are unclear. Then read about training data and ground truth to see how labels feed model building, and evaluation to see how a labeled test set is used.

More in AI foundations

• Common Crawl - Common Crawl is a nonprofit organization and the large, openly available archive of web pages it collects by crawling the public internet and releases for...
• Context window - A context window is the limited amount of information a language model can work with in a particular generation.
• Dartmouth workshop - The Dartmouth workshop was a summer research project held at Dartmouth College in 1956 that is widely treated as the founding event of artificial...
• Data labeling - Data labeling is the process of attaching tags, categories, scores or other annotations to raw data such as text, images, audio or records, so that a...
• Deep learning - Deep learning is a branch of machine learning that learns representations through multiple layers of computation.
• Direct preference optimization (DPO) - Direct preference optimization (DPO) is a method for adjusting a language model so its answers match human preferences, using pairs of responses where...

Sources

Google, Machine Learning Glossary: https://developers.google.com/machine-learning/glossary ISO, ISO/IEC 22989 Artificial intelligence concepts and terminology: https://www.iso.org/standard/74296.html Goodfellow, Bengio and Courville, Deep Learning: https://www.deeplearningbook.org/ The sources were consulted for standard definitions of labeled data and supervised learning; the examples, analogy, worked example and case study are original educational illustrations. Checked 5 Nov 2026.

Previous: Data analysis Next: Data archaeology