Can average error be zero when predictions are poor?

Yes. Large positive and negative errors can cancel.

Is RMSE always preferable to MAE?

No. They emphasize different losses, and the decision context matters.

Are classification mistakes represented the same way?

Categorical labels need suitable classification losses or probability scores; arbitrary subtraction of category codes is generally not meaningful.

AI Glossary / Statistics and probability / Intermediate

Errors of prediction

Definition

Errors of prediction are the differences between observed outcomes and their corresponding predicted values. Using the convention error = observed minus predicted, a positive error indicates underprediction and a negative error indicates overprediction. Individual errors show how predictions missed particular outcomes; summaries such as mean absolute error describe performance across cases. The evaluation context determines whether these are training residuals or genuine predictions of previously unseen outcomes.

What it means

If a system predicts fifteen minutes and an observed task takes eighteen, its prediction error is +3 minutes under this convention. If it predicts twenty, the error is -2 minutes. Neither value alone says whether the prediction method is useful overall.
The supplied glossary uses Y minus Y-hat. Some software and disciplines reverse the sign, so the convention should always be stated. Absolute and squared error magnitudes do not change when the sign convention reverses, but interpretations of positive bias do.

How it works

Match each prediction to the correct observed outcome, unit, and horizon. For a forecast made at time t for time t + h, record what information was available when the prediction was issued. Later revisions or unavailable future inputs must not silently enter the evaluation.
Calculate each signed error e_i = y_i - y-hat_i. The mean error is the average signed discrepancy and helps reveal a tendency toward underprediction or overprediction. Positive and negative errors can cancel, so a mean near zero does not imply accurate individual predictions.
Mean absolute error averages absolute discrepancies. Mean squared error averages squared discrepancies, giving large misses greater influence. Root mean squared error takes the square root of that average and returns to the outcome's units. The appropriate metric depends on the decision's loss: underprediction and overprediction may have different costs.
Distinguish fitted residuals from out-of-sample errors. A model can fit its training observations closely because parameters were chosen using them. Evaluate generalization with an appropriate held-out design or cross-validation. For time-dependent problems, preserve information order and use forecasting horizons relevant to the deployment.
Inspect patterns as well as averages. Error dependence, systematic differences across groups, changing spread, or extreme misses can reveal weaknesses hidden by a single score. Compare models on comparable cases and against a relevant baseline.
Prediction errors contain several possible contributions, including imperfect estimation, model mismatch, irreducible outcome variation under the available information, and measurement problems in the observed target. A large error does not by itself identify which contribution caused it. Uncertainty intervals or distributional forecasts require their own calibration and scoring checks.

Three practical examples

A staffing forecast is compared with realized call volume at the same forecast horizon to assess underprediction during busy periods.
An AI regression model's held-out errors are examined across product categories rather than only as one overall average.
An engineer compares predicted and measured process outputs while checking whether unusually large discrepancies indicate sensor problems or model inadequacy.

Think of it this way

Think of estimating arrival times. The signed difference tells you whether an arrival was earlier or later than predicted, while the absolute difference tells you how far off the estimate was. Several early and late arrivals can cancel in an average without making the estimates dependable.

Worked example

Use an original set of four observed values: 10, 15, 12, and 20. The corresponding predictions are 12, 13, 12, and 17, produced without using those outcomes.
Observed-minus-predicted errors are -2, +2, 0, and +3. The mean error is (-2 + 2 + 0 + 3)/4 = 0.75, indicating net underprediction in these four cases.
Mean absolute error is (2 + 2 + 0 + 3)/4 = 1.75. Mean squared error is (4 + 4 + 0 + 9)/4 = 4.25. Root mean squared error is the square root of 4.25, approximately 2.062.
The mean error's smaller magnitude does not indicate better performance; it reflects cancellation. Squaring also gives the last error a contribution of nine, larger than the contribution of four from each two-unit error.
These are descriptive evaluation results for four cases, not precise estimates of future performance. If these had instead been fitted training values, the numerical summaries would be identical while their evidential interpretation would differ.

Illustrative case study

Fictional case: Elm Dispatch compares two arrival-time models. One has a small average signed error, so its report initially calls it highly accurate. Drivers nevertheless experience frequent large misses in both directions.
The analyst adds absolute error, tail summaries, and route-specific checks, using predictions recorded before each trip. The team discovers that opposing route biases cancel in the overall mean. It improves the evaluation and investigates route conditions before choosing a replacement. The revised report connects misses to operational consequences rather than celebrating cancellation.

Common misconceptions and mistakes

Mistake: using mean signed error as the sole accuracy measure.
Mistake: comparing errors measured in different units or at different horizons without a justified normalization.
Mistake: evaluating on outcomes already used for model selection and treating the result as an untouched test.
Mistake: applying percentage errors when outcomes are zero or close to zero without addressing instability.
Mistake: assuming every residual is an out-of-sample prediction error.
Mistake: claiming a large observed error proves the underlying process changed; other explanations require investigation.

Business use

Businesses use prediction errors to compare models, set service expectations, detect changing performance, and prioritize costly failure modes. Choose metrics before selecting a winner and connect them to the decision's costs. Retain the prediction timestamp, target definition, horizon, and evaluation population so results remain auditable. Strong average performance should be supported by checks of important subgroups and extreme errors.

Check your understanding

Why is the worked RMSE larger than the MAE?

Answer: Squaring gives more influence to larger error magnitudes before taking the square root.

What makes an error a valid prospective forecast error?

Answer: The prediction must use only information available before the target outcome and be matched to the intended horizon.

Related terms

Learn next

compare signed and unsigned error summaries, evaluate forecasts by horizon, and use a decision-specific loss when overprediction and underprediction have different consequences.

Sources

Sources: Howell, Statistical Methods for Psychology glossary, Errors of prediction: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html . Hyndman and Athanasopoulos, Forecasting: Principles and Practice, The forecaster's toolbox, fitted values and forecast accuracy sections: https://otexts.com/fpp3/toolbox.html . The four-case error calculation is original.

Previous: Error variance Next: Essential matrix