Can Bayes error be zero?

Yes, if the observed features determine the label almost surely under the population distribution.

Can it change over time?

Yes, if the joint distribution or available information changes.

Can a finite test score appear better than the limit?

Sampling variation can produce that appearance; the bound concerns expected population performance, not every realized sample.

AI Glossary / Statistics and probability / Intermediate

Bayes error rate

Definition

The Bayes error rate is the smallest possible expected misclassification rate for a specified joint distribution of observed features and class labels, when all classification mistakes have equal cost. It is achieved by selecting a most probable class for each feature value. The limit depends on what information is available to the classifier.

What it means

Even ideal classifiers face ambiguity when identical observed features occur in different classes.

Bayes error describes that remaining population-level uncertainty. It separates the difficulty of the prediction problem from the additional error caused by limited training data, imperfect estimation, or restrictions on the model family.

The word irreducible is conditional on the task and information set. Adding genuinely informative measurements can lower the limit, while simply trying a more complex algorithm on the same fully specified information cannot beat it in expected population error. The limit is usually unknown in real projects because the true distribution is unknown.

How it works

At a particular input x, let the class probabilities be P(Y = j given X = x). Choosing the most likely class produces a conditional error of 1 minus the largest of those probabilities.

Average this quantity over the population distribution of inputs. The resulting expectation is the Bayes error rate: E[1 - max_j P(Y = j given X)]. For a binary outcome with positive-class probability eta(x), the same expression is E[min(eta(X), 1 - eta(X))].

This is a probability of error under zero-one loss. With unequal mistake costs, the corresponding minimum expected loss is a Bayes risk, but it need not equal an ordinary error rate.

Theoretical calculations can be exact when a data-generating distribution is specified. Empirical work must instead estimate or bound the limit with assumptions and uncertainty. A model's observed test error is not itself the Bayes error, and a low finite-test error does not prove that a theoretical population limit has been exceeded. Sampling variation, data leakage, or a changed task can explain apparent contradictions.

Three practical examples

Sensor design: assess whether ambiguity arises because the current measurements omit information needed to distinguish two operating states.
Benchmark interpretation: separate a classifier's performance from the inherent difficulty of labels that overlap under the benchmark's available inputs.
Data investment: compare improving probability estimation with collecting an additional useful feature, recognizing that these target different sources of error.

Think of it this way

Think of sorting sealed envelopes using only their color. If envelopes of the same color contain different categories, perfect color recognition still cannot reveal every category. Opening an additional permitted window may provide new information; examining the same color more carefully cannot eliminate the underlying ambiguity.

Worked example

Suppose a binary-classification population has three feature groups, A, B, and C, occurring with probabilities 0.50, 0.30, and 0.20. Their true positive-class probabilities are respectively 0.90, 0.40, and 0.20.

The Bayes rule predicts positive for A and negative for B and C. The conditional error rates are therefore 0.10, 0.40, and 0.20. Weighting them by group prevalence gives 0.50 × 0.10 + 0.30 × 0.40 + 0.20 × 0.20 = 0.21. Thus the Bayes error rate is 21%, and the maximum population accuracy using these features is 79%.

Now suppose an additional measurement splits B into two equally common subgroups with positive probabilities 0.80 and 0.00. Their average remains 0.40, consistent with the original group B. With the new measurement, B's average conditional error becomes (0.20 + 0.00) / 2 = 0.10.

The new overall Bayes error is 0.50 × 0.10 + 0.30 × 0.10 + 0.20 × 0.20 = 0.12, or 12%. The improvement comes from additional predictive information, not from defeating the original limit on its own terms.

Illustrative case study

Fictional case: Harbour Metrics predicts whether a service request will need an onsite visit using its initial text and product category. Several strong models make similar errors on vague requests.

The team does not declare that it has reached the Bayes error. It examines labels, sampling, and model limitations, then pilots a short clarification question available before the decision. The answer sometimes reveals an equipment condition absent from the original inputs.

Evaluation compares the expanded workflow with the original one and includes the burden of asking the question. Managers learn that reducing ambiguity may require better information, while recognizing that a practical performance plateau is not a proof of theoretical optimality.

Common misconceptions and mistakes

Do not equate Bayes error with human error. A human may use additional information, apply a different label definition, or make avoidable mistakes.

Do not equate it with the error of naive Bayes, a Bayesian neural network, or whichever fitted model currently performs best. Those are particular procedures whose population errors can exceed the limit.

Do not assume more training examples reduce the true Bayes error for a fixed distribution. They may help a learner approach it more closely.

Do not describe label noise as automatically random or irreducible. Some apparent noise reflects fixable annotation inconsistencies or omitted information. Changing the target definition or measurement process changes the prediction problem and therefore potentially changes its limit.

Business use

The concept helps organizations ask whether additional algorithm development addresses the right bottleneck. When observed information is weak, investment in measurement quality, clearer labels, or a different workflow may be more valuable than another model variation.

However, the limit should not become an unsupported excuse for poor performance. Claims of being close to Bayes error need evidence and stated assumptions. Decision makers should distinguish measured model error, estimated uncertainty, data-quality concerns, and theoretical limits. Costs, fairness, and service objectives may also require evaluating more than overall misclassification rate.

Check your understanding

Why weight the conditional errors in the example?

Answer: The feature groups occur at different rates in the population.

Does an observed model plateau prove the Bayes error has been reached?

Answer: No. Estimation, architecture, optimization, or data problems may still explain the plateau.

Related terms

Bayes classifier
Conditional probability
Label noise
Feature engineering
irreducible error
zero-one loss
excess risk
generalization error

Learn next

Calculate Bayes error for a small known distribution, then simulate training and test samples to compare the theoretical limit with realized scores. Study empirical bounds before making claims about unknown real-world limits.

Sources

Siva Balakrishnan, Carnegie Mellon University, Data Mining Lecture 2, Bayes classifier and error derivation: https://www.stat.cmu.edu/~siva/teaching/462/lec2.pdf Sekeh, Oselio, and Hero, Learning to Bound the Multi-class Bayes Error, author abstract describing the estimation problem and bounds: https://arxiv.org/abs/1811.06419

Previous: Bayes classifier Next: Bayes estimator