AI Glossary / Data science and analytics / Intermediate

Labeled data

Definition

Labeled data are examples paired with target information for a defined learning or evaluation task. A label may be a class, a number, a text annotation, a region in an image, or another structured target. In supervised learning, the model learns a relationship between the example's input features and its label. A label is a recorded reference, not an automatic guarantee of objective truth.

What it means

For a support-message classifier, the message is input data and a category such as billing is its label. For a duration predictor, the eventual number of minutes can be the label. The same record can have different labels for different tasks. A message's topic and urgency are distinct targets, even if both are useful. Clearly defining the target is therefore as important as collecting many examples.

How it works

Start with a task specification and a labeling policy that covers ordinary cases, ambiguous cases, and missing information. Labels may come from human annotation, measured outcomes, operational records, rules, or other models. Preserve their origin and relevant timing. Check a sample for consistency and validity, investigate disagreements, and correct systematic problems. Separate training, validation, and test data appropriately so that reported performance reflects the intended use. A model's prediction can later become a candidate annotation, but it should not silently acquire the status of an independently verified label.

Three practical examples

• Text classification: support requests are paired with reviewed topic categories to train a routing model.
• Visual inspection: images are paired with annotated defect locations and types to train or evaluate a detector.
• Regression: completed delivery records are paired with measured journey durations to train a numerical prediction model.

Think of it this way

Think of practice questions with an answer key. The key tells a learner what the task considers a correct response. The analogy has limits: a dataset may include ambiguous or subjective judgments, and an operational label may be a noisy proxy for the real objective. A confident answer key can still contain mistakes.

Worked example

Suppose a fictional team has 100 support messages labeled billing or technical. A second reviewer independently labels the same messages using the written policy. The reviewers agree on 88 and disagree on 12, giving raw agreement of 88/100 = 88%. This calculation measures agreement, not accuracy: both reviewers might make the same mistake, and class imbalance can affect how impressive the percentage appears.
The team inspects the 12 disagreements and finds that 7 involve a billing problem caused by a technical failure. Its policy never specified whether to choose the user's immediate request or the underlying cause. It clarifies that routing should follow the team responsible for the requested action, then rechecks affected records and a fresh sample. It does not simply force agreement by copying the first reviewer's answers. This example illustrates how label review can reveal an unclear task definition before model training begins.

Illustrative case study

In a fictional document-search project, clicked results are used as labels for relevance. Early analysis treats every unclicked result as irrelevant. A reviewer notes that some results were never shown prominently enough to receive attention, so absence of a click is not a reliable negative judgment. The team documents the behavioral label's limitations and creates a separately reviewed evaluation set. The case illustrates the difference between an observed action and the concept the model is intended to learn; it does not claim that click data are always unusable.

Common misconceptions and mistakes

Labeled does not mean manually labeled, and manually labeled does not mean error-free. A dataset can have accurate labels for the wrong business question. Missing labels should not automatically become negative labels. Labels available only after an outcome occurs may be appropriate targets but inappropriate input features at prediction time. Duplicate examples across training and evaluation can inflate apparent performance. A single class label may also hide legitimate disagreement or multiple valid answers, so some tasks need richer annotations or uncertainty information.

Business use

Budget for defining and maintaining labels, not only for collecting raw records. Record the policy version, source, review procedure, and relevant timestamps so changes can be traced. Sample difficult and underrepresented cases as well as easy ones. Assess the cost of different labeling errors in the intended workflow. If AI-assisted labeling is used, measure its errors and decide where human review is needed. Keep a dependable evaluation set that reflects the task, population, and conditions under which the model will be used.

People also ask

Can a label be a number?

Yes. Numerical targets are common in regression tasks.

Are unlabeled records useless?

No. They can support other learning methods, exploration, or later annotation.

Does high reviewer agreement prove the labels are correct?

No. Agreement is useful evidence about consistency, but shared bias or an unsuitable policy can still produce misleading labels.

Check your understanding

Why should the team not report 88% label accuracy from the reviewer example?

Answer: It calculated agreement between two reviewers, not correctness against an independently justified reference. Both reviewers could agree on an incorrect or poorly defined label.

Related terms

• Label
• Supervised learning
• Ground truth
• Inter-rater agreement
• Label noise
• Weak supervision
• annotation
• training data
• evaluation dataset

Learn next

Write labeling instructions for a small task and test them on ambiguous examples before scaling up. Compare disagreement cases with ordinary cases to find missing rules. Then study the difference between human judgment, measured outcomes, and behavioral proxies, and how each affects model evaluation.

More in Data science and analytics

• k-nearest neighbors - K-nearest neighbors, often written k-NN, is a method that uses the k closest reference examples to make a prediction or retrieve similar cases.
• K-optimal pattern discovery - K-optimal pattern discovery is an exploratory data-mining approach that seeks the k best patterns according to a specified measure and set of constraints.
• L1-norm principal component analysis - L1-norm principal component analysis refers to PCA-related methods that use absolute-value criteria instead of the squared-value criteria associated with...
• Latent variable - A latent variable is a variable included in a model but not directly observed in the available data.
• Lift - Lift is a ratio that compares an observed or model-selected outcome with a stated baseline.
• Line chart - A line chart displays values as points connected in an ordered sequence, commonly to show change over time.

Sources

Google, labeled examples and supervised-learning terminology: https://developers.google.com/machine-learning/glossary/fundamentals Google, human and machine labeling issues: https://developers.google.com/machine-learning/crash-course/categorical-data/issues?hl=en Google Cloud, labeling guidelines and quality control: https://cloud.google.com/use-cases/data-labeling Google PAIR, data collection and evaluation design: https://pair.withgoogle.com/chapter/data-collection/

Previous: Label vector Next: Labeled data point