Must soft labels sum to one?

For a mutually exclusive multiclass target distribution, yes. Multilabel targets follow a different setup.

Can a soft label contain zeros?

Yes. It need not assign positive mass to every class.

Does using soft labels guarantee better accuracy?

No. Their usefulness depends on target quality, the learning setup and the evaluation objective.

AI Glossary / Mathematics and optimization / Intermediate

Soft label

Definition

A soft label is a target that assigns weights or probabilities across classes instead of selecting only one class. In ordinary single-label classification, it is commonly represented by nonnegative values that sum to one, such as (0.7,0.2,0.1).

What it means

A hard label says that an example belongs to one class, often encoded as a one-hot vector such as (1,0,0). A soft label preserves a distribution across alternatives. That distribution may reflect uncertainty, disagreement among annotators, a teacher model's predictions or a deliberately modified training target.

Those origins are not interchangeable. Annotator vote proportions describe the collected judgments. A teacher distribution describes a model's output. Label smoothing introduces a designed preference against extreme targets. None automatically establishes the true conditional probability of the real-world class.

The source-list sense concerns a finite, mutually exclusive class space. In multilabel problems, several labels can apply simultaneously, and separate probability targets need not sum to one. The task definition determines the appropriate interpretation.

How it works

For a target distribution q and predicted distribution p, a common training loss is cross-entropy, L=−Σcq_c log(p_c). Each target weight controls how much its class contributes. With a one-hot target, this reduces to the negative log probability of the selected class.

Knowledge distillation can use a larger model's class distribution as a target for a smaller model. A temperature parameter may soften distributions so that relative probabilities outside the top class influence training. The temperature and loss-scaling conventions should be documented.

A separate construction is label smoothing. For K classes, one convention uses q=(1−ε)y+εu, where y is one-hot and u is uniform across all K classes. Other conventions distribute smoothing mass only among incorrect classes, producing different numbers.

A target vector should match the class ordering and intended loss. Treating raw logits as probabilities, forgetting normalization or mixing different class maps can create a valid-looking array with the wrong meaning. Softness is a representation choice, not a substitute for label-quality review.

Three practical examples

A student classifier learns from the full class distribution produced by a teacher model during distillation.
An annotation team preserves the distribution of several reviewers' judgments instead of reducing every example to a majority vote.
A researcher uses explicitly defined label smoothing and compares its effect on held-out predictive behavior.

Think of it this way

Think of recording the distribution of votes in a committee instead of writing down only the winning option. The full tally can show a close decision or a strong consensus.

The analogy does not make the vote share an objective probability of truth. Voters may share biases, have different expertise or receive incomplete information. The meaning of a soft label likewise depends on how it was constructed.

Worked example

The same predicted class, different loss

Suppose a three-class target is q=(0.6,0.3,0.1). Compare prediction p=(0.7,0.2,0.1) with r=(0.9,0.05,0.05). Both choose class one if the final decision takes the largest probability.

Using natural logarithms, the soft-target cross-entropy for p is −0.6ln(0.7)−0.3ln(0.2)−0.1ln(0.1), approximately 0.92709. For r it is −0.6ln(0.9)−0.3ln(0.05)−0.1ln(0.05), approximately 1.26151. Prediction p better matches the supplied distribution.

With the hard target (1,0,0), the losses instead become −ln(0.7)≈0.35667 and −ln(0.9)≈0.10536. The hard target prefers r. This difference follows from the target information being optimized, not from an arithmetic inconsistency.

For a separate smoothing example, K=3 and ε=0.15 under uniform-mixture smoothing turn (1,0,0) into (0.9,0.05,0.05). That constructed target should not be confused with an observed 90-5-5 vote distribution, even though the numerical vectors match.

Illustrative case study

A fictional document-routing team asks ten reviewers to classify an ambiguous message. Six choose billing, three choose account support and one chooses another category. The team considers using (0.6,0.3,0.1) as the training target.

Before doing so, it reviews the disagreement. Some reviewers may be interpreting the category definitions differently, so preserving their votes could preserve a labeling problem rather than useful ambiguity.

The team clarifies the guidelines and records whether each final target comes from reviewed votes, a teacher model or smoothing. It evaluates the model against a consistent task definition and avoids presenting all soft targets as equally authoritative.

Common misconceptions and mistakes

A soft label is just a low-confidence hard label. It can preserve relative weights across several alternatives.
Soft probabilities are guaranteed to be calibrated truth. Their source and construction determine what they mean.
Soft labels and softmax outputs are identical concepts. A soft label is a target; a softmax output is a model prediction or transformation that may be used to create one.
All smoothing formulas are equivalent. Their mass-allocation conventions differ.
The largest class is all that matters. The full target distribution affects training losses.

Business use

Soft labels can help retain information that a single class assignment would discard. They are useful when transferring model behavior or representing genuine ambiguity, but they also make provenance especially important.

Store class definitions, class order, target-generation method and any temperature or smoothing settings. Review whether downstream users need a single action, a probability distribution or an option to defer. A model trained with soft labels still needs evaluation for the actual decisions it will support, including cases where uncertainty or disagreement matters operationally.

Check your understanding

Why does the soft target in the example prefer p over r even though r assigns more probability to the most likely class?

Answer: The target also assigns substantial weight to class two. Prediction r nearly excludes that class, so its full-distribution cross-entropy is worse.

Related terms

One-hot encoding
Cross-entropy
Probability calibration
Hard label
knowledge distillation
label smoothing

Learn next

Read cross-entropy and knowledge distillation. Compare a majority-vote label with the full vote distribution for an ambiguous example, then explain what information is lost when only the winning class is retained.

Cross-entropy

Sources

Hinton, Vinyals and Dean, Distilling the Knowledge in a Neural Network: https://arxiv.org/pdf/1503.02531. Primary soft-target and temperature treatment.

Aalto Dictionary of Machine Learning, soft label: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Class-distribution definition. Loss calculations, smoothing illustration and fictional annotation case are original examples.

Previous: Soft independent modeling of class analogies Next: Soft margin