Is empirical loss always training loss?

No. It can be measured on any finite evaluation sample; training loss is one important instance.

Is empirical loss the same as accuracy?

Only for a particular relationship, such as zero-one loss equaling one minus accuracy on the same unweighted classification sample.

Does zero training loss mean the problem is solved?

No. It may reflect exact fitting of the observed sample without reliable performance elsewhere.

AI Glossary / Graphs and symbolic AI / Intermediate

Empirical loss

Definition

Empirical loss is the loss measured on a finite set of observed examples, usually calculated as the average of a per-example loss function. It quantifies how a model performs on those examples under a specified penalty rule. It is an observed-sample quantity, not automatically the model's expected loss on future data.

What it means

A model makes predictions, and a loss function assigns a penalty to each prediction relative to its target. Empirical loss aggregates those penalties over a dataset. With squared error, large numerical mistakes receive larger penalties; with zero-one loss, each incorrect classification contributes one and each correct classification contributes zero.
The dataset matters as much as the formula. Training loss describes the examples used to fit the model. Validation or test loss describes other samples used for evaluation. All are empirical measurements, but they serve different purposes and support different conclusions about generalization.

How it works

For n examples, calculate each loss L(prediction_i, target_i), add the values, and divide by n for an unweighted empirical mean. Some systems report a sum or use weights instead, so state the aggregation convention. Empirical risk minimization chooses a model to minimize such a training-sample objective.
The actual optimization objective may also include regularization or other penalties. Those additions should be distinguished from the data loss itself. Mini-batch training estimates the objective from subsets during updates, and a batch's reported loss need not equal the full dataset's mean. Consistent definitions are necessary when comparing runs.

Three practical examples

A demand-forecasting experiment reports mean squared prediction error on a held-out set of observations.
A classifier measures its fraction of incorrect labels using zero-one empirical loss.
A model-training dashboard separates the average training-data loss from a regularization penalty and from validation performance.

Think of it this way

Think of grading a set of completed practice questions using a stated marking scheme. The average penalty tells you how the student did on those questions. The analogy is limited because a model's training process may adapt directly to the same examples, and a good practice score does not guarantee performance on a different exam distribution.

Worked example

Suppose three targets are 2, 4, and 6. Model A predicts 1, 4, and 8. Its errors are -1, 0, and 2, so the squared losses are 1, 0, and 4. Mean squared empirical loss is (1 + 0 + 4)/3 = 5/3, approximately 1.67.
Model B predicts 2, 3, and 6. Its squared losses are 0, 1, and 0, giving a mean of 1/3, approximately 0.33. Under this dataset and squared-error criterion, B has the lower empirical loss.
If we use absolute error instead, A's mean is (1 + 0 + 2)/3 = 1, while B's is 1/3. The ranking happens to agree here, but the numerical values and penalty meanings differ. Neither comparison proves how the models perform on unseen examples. If the targets are measured in units, squared loss has squared units, while absolute loss has the original units. The calculations are illustrative and do not describe a business forecast.

Illustrative case study

At fictional Birch Forecasts, a team celebrates a lower training loss after increasing model complexity. Validation loss rises on examples not used for fitting. The team investigates overfitting and checks whether the training sample reflects the intended forecasting period.
It reports both measurements with their data splits and loss definitions. This fictional case illustrates the distinction between fitting observations and predicting new cases, without claiming that one particular remedy will always improve generalization.

Common misconceptions and mistakes

Low empirical training loss is not proof of low population loss. A model can memorize training peculiarities, exploit leakage, or perform poorly on underrepresented cases. A test estimate also has sampling uncertainty and depends on how representative the test data are.
Do not compare raw loss numbers across different functions or scales as though smaller always means better in a common sense. A value of 0.2 under one loss may be incomparable with 2 under another. Likewise, averaging all examples equally can hide subgroup differences or unequal business consequences. The aggregation should reflect the intended evaluation while retaining useful breakdowns.

Business use

Businesses should define what errors matter before selecting the loss. Record the formula, units, weights, data split, and handling of missing targets. Evaluate on data separated appropriately from model fitting and tuning. Compare operational metrics alongside loss when a mathematical penalty is only a proxy for the intended outcome. If an objective includes regularization, show its contribution separately so reviewers can understand whether a reported improvement comes from fitting data better or from changing the penalty balance.

Check your understanding

A classifier makes four errors on 20 equally weighted examples. What is its mean zero-one empirical loss?

Answer: 4/20 = 0.2. Its accuracy on those same examples is 0.8, but neither number alone establishes performance on future data.

Related terms

Learn next

Add a fourth example with a much larger error to the worked dataset. Compare how mean squared error and mean absolute error change. Then explain which penalty better reflects a hypothetical task where occasional large errors have disproportionately high consequences.

Sources

Goodfellow, Bengio, and Courville, Deep Learning, original optimization chapter, empirical risk: https://www.deeplearningbook.org/contents/optimization.html University of Edinburgh, original machine-learning theory introduction defining empirical loss: https://opencourse.inf.ed.ac.uk/sites/default/files/https/opencourse.inf.ed.ac.uk/mlt/2024/introduction.pdf

Previous: Empirical gradient Next: Empirical risk