Is training error the same as training accuracy?

Not generally. For unweighted classification error defined as the wrong-label fraction, accuracy is one minus that error. Other losses have no such conversion.

Should training error always decrease each step?

No. Stochastic updates and other training choices can produce fluctuations.

Can I use training error to choose the best deployed model?

It is useful diagnostic evidence, but selecting for new-data performance requires an appropriate evaluation beyond the fitted examples.

AI Glossary / Mathematics and optimization / Beginner

Training error

Definition

Training error is a model's measured loss on the examples used to fit it. Usually it is an average over the training set, with a specified loss function such as squared error or classification error. It describes performance on already-used data. A small training error alone does not establish that the model will perform well on new examples.

What it means

Learning algorithms use training examples to adjust model parameters. Evaluating on those same examples tells us how well the resulting model fits the material it has seen.
The word error is broader than the number of wrong answers. In regression, it might mean mean squared error; in classification, it might mean the fraction of incorrect labels or an average probability-based loss. These quantities answer different questions and have different numerical scales.
Some mathematical discussions use training error for the minimum empirical loss achievable within a chosen model class. In a practical report, however, a measured value need not be that minimum. Limited optimization, regularization, or early stopping may leave a model above the smallest possible training loss.

How it works

Choose a dataset of n training examples, a fixed model h, and a per-example loss L. Compute the loss for each prediction and average: training error = (1/n) sum L(h(x[i]), y[i]). Weighted averages require an explicitly stated weighting rule.
Separate the data-fitting loss from the full optimization objective. A training objective may also contain a parameter penalty, and that extra term is not an error on a particular training observation.
For meaningful comparisons, keep the dataset, preprocessing, loss definition, and evaluation procedure consistent. A running average collected during training may involve many different parameter states. It therefore need not equal a fresh evaluation of the final model on the whole training set.

Three practical examples

A demand-forecasting team can compare training and validation mean squared error to investigate whether extra model complexity is improving performance beyond the fitted observations.
A document classifier can monitor training classification error while separately reporting log loss, since confident wrong probabilities and wrong labels measure different aspects of performance.
An engineer can use unexpectedly high training loss to investigate a broken label mapping, unsuitable features, insufficient model capacity, or an optimization problem before attributing everything to generalization.

Think of it this way

Think of practicing a set of examination questions and then being tested on those exact questions. Your score reveals how well you answer the practice set. It may reflect understanding, memorization, or both. A separate set is needed to probe transfer. Unlike a simple examination score, a machine-learning loss can also grade the size or confidence of each error.

Worked example

A regression training set has targets [1, 2, 4, 5]. Model A predicts [1, 3, 3, 5], giving residuals [0, 1, −1, 0]. The squared residuals sum to 2, so training mean squared error is 2/4 = 0.5.
On a separate two-example validation set, targets are [2, 6] and A predicts [4, 4]. Squared errors are 4 and 4, so validation MSE is 4. The observed validation-minus-training difference is 3.5. This finite-sample difference is a diagnostic observation, not the exact population generalization gap.
Suppose model B predicts [2, 3, 5, 6] on the same training set. Every residual is 1, making training MSE 1. On the validation set it predicts [3, 5], giving validation MSE 1.
B fits the training examples less closely but performs better on this validation set. Choosing solely by training MSE would favor A. The tiny sample illustrates the distinction; it is insufficient evidence for a reliable deployment decision.
If A's optimization objective adds a parameter penalty of 0.3, its objective value is 0.8, while its training MSE remains 0.5.

Illustrative case study

In a fictional support-routing project, a team celebrates nearly perfect training accuracy after adding a customer identifier. Its evaluation also looks strong because tickets from the same customers appear on both sides of a random split.
The team clarifies whether deployment involves new customers or future tickets from existing customers and designs an evaluation that matches that question. It also checks whether the identifier is available and appropriate at prediction time.
The lesson is that training error cannot settle how a model behaves in a different operating situation. The fictional example does not establish that identifiers are always inappropriate or that every random split is invalid.

Common misconceptions and mistakes

Do not treat zero training error as proof of correctness. A model may memorize labels, and labels themselves can contain errors. Conversely, nonzero training error does not automatically mean training failed: the chosen model, penalty, and stopping rule may deliberately accept some mismatch.
Validation error is not guaranteed to exceed training error in every observed run. Sampling variation and differences in evaluation conditions can reverse that ordering.
Avoid comparing a sum of losses from one report with an average from another, or a regularized objective with an unregularized error. Also avoid interpreting a changing minibatch curve as if it measured a fixed dataset using one fixed model.

Business use

Training error helps diagnose model development, but business acceptance needs evidence about the intended operating population. Pair it with suitable validation or test results, relevant baselines, and task-specific costs.
When a report says error fell, ask which loss, which observations, and which evaluation conditions changed. Record the distinction between fitting performance and expected service performance. This makes it easier to decide whether to improve data, revise the model, fix optimization, or reconsider the evaluation design.

Check your understanding

A model makes three wrong predictions on 20 equally weighted training examples. What are its training classification error and training accuracy?

Answer: Error is 3/20 = 15%, and accuracy is 17/20 = 85%. These figures alone do not tell us its error on new data.

Related terms

Learn next

Calculate both mean absolute error and mean squared error for model A's training predictions. Then design a validation split for a concrete future use case, documenting which examples or groups must remain separate. Study generalization and leakage before interpreting a small training loss as a business result.

Sources

Aalto Dictionary of Machine Learning, training error and training set entries: average training loss and its distinction from evaluation on other observations. https://aaltodictionaryofml.github.io/ADictML_Main.pdf

Previous: Training curve Next: Training loss