Can the average residual be zero while errors are large?

Yes. Positive and negative values can cancel.

Is a residual the same as an unobserved error term?

No. A residual uses the fitted model; the theoretical error is defined relative to the underlying model relationship.

Are all residuals measured in the original units?

No. Standardized and other specialized residuals use different definitions and scales.

AI Glossary / Governance, ethics and safety / Intermediate

Residual

Definition

A residual is the part of an observed value left unexplained by a fitted statistical model. In ordinary regression, a common convention defines it as the observed value minus the model's fitted value. Residuals help reveal where and how a model misses the data. Their size, direction, and patterns can be more informative than a single overall accuracy measure.

What it means

If a model predicts a delivery time of 30 minutes and the observed time is 36, the residual under this convention is +6 minutes. The positive sign means the model underestimated the observation. A negative residual means it overestimated it. Residuals are calculated using a fitted model, whereas the unobserved statistical error in a theoretical model refers to deviation from the true underlying relationship. The two are related but not identical. This entry concerns statistical residuals, not residual connections in neural networks or residual risk remaining after controls.

How it works

Fit a model, compute its predicted value for each relevant observation, and subtract that prediction from the observed response. Plot residuals against fitted values, input variables, time, or other relevant factors. Curved patterns can suggest a missing relationship, changing spread can suggest unequal variability, and time patterns can suggest dependence or drift. These are diagnostic clues, not automatic diagnoses. Different model families use different residual types, including standardized, Pearson, or deviance residuals. Their definitions and expected patterns must match the model rather than being interpreted as if every residual were an ordinary difference in original units.

Three practical examples

A demand forecaster examines residuals by season to discover that the model systematically underestimates a recurring peak.
A delivery-time model plots residuals against distance and notices that longer trips have a wider spread of errors.
A quality-monitoring system examines residuals over time to detect a sustained departure from the fitted relationship. These examples use residuals to guide investigation rather than immediately attributing a pattern to a known cause.

Think of it this way

Think of laying a fitted curve over plotted observations and measuring the vertical gaps between them. The gaps show what the curve did not capture at those points. The analogy is limited because many models have multiple inputs or nonstandard outcomes, and some residuals are transformed quantities. A visually tidy plot also does not prove that the model will work well on future data.

Worked example

Suppose observed values are 10, 14, and 15, while fitted predictions are 12, 12, and 15. Using observed minus predicted, the residuals are -2, +2, and 0. Their mean is zero, but the model is not perfect: the mean absolute residual is (2 + 2 + 0)/3 = 1.333, and the mean squared residual is (4 + 4 + 0)/3 = 2.667. The root mean squared residual is about 1.633 in the original response units. This example shows how positive and negative deviations can cancel in a mean. It also distinguishes a descriptive mean squared residual from an unbiased regression error-variance estimate, which uses appropriate degrees of freedom and assumptions. The reported dataset and calculation should be stated clearly.

Illustrative case study

Fictional case: Pine Logistics sees a near-zero average residual and concludes that its timing model is satisfactory. A reviewer plots the individual residuals and finds consistent overestimation on short routes and underestimation on long ones. The team revises the model specification and evaluates the change on later data. This invented case illustrates why an aggregate signed average can hide systematic structure. It does not imply that every visible pattern justifies adding more model complexity.

Common misconceptions and mistakes

A residual is not automatically random noise, and a large residual is not automatically a bad data point. It can reflect a real unusual case, measurement error, missing predictors, or model misspecification. Ordinary least-squares training residuals have algebraic properties under particular conditions, such as summing to zero when an intercept is included; those properties do not prove good future predictions. The same properties need not hold for other fitting methods or new data. A residual plot should be interpreted with the study design, model assumptions, and practical consequences in view.

Business use

Use residual analysis to complement task-level metrics. Inspect patterns across meaningful input ranges, time periods, and user groups where appropriate. Preserve the sign convention and units in reports. Investigate unusual cases before deleting them, and record any data corrections or model changes. Keep training diagnostics separate from evaluation on held-out or future observations. When a model is used operationally, ensure the eventual observed outcome is measured consistently before interpreting a residual trend. A changing residual pattern can signal a problem, but further evidence is needed to identify its source.

Check your understanding

A prediction is 45 and the observation is 40. Under the observed-minus-predicted convention, what is the residual and what does its sign mean?

Answer: The residual is -5. The model overestimated the observation by 5 units.

Related terms

Mean Squared Error
Autocorrelation
Outlier
regression
fitted value
prediction error
residual plot
heteroscedasticity
model diagnostics
standardized residual

Learn next

Compute residuals for a small set of predictions and observations. Plot them against the predictions and the order of collection. Explain what a curved pattern, a widening spread, or a sustained run of one sign would suggest investigating, without treating any pattern as proof of a cause.

Sources

NIST Engineering Statistics Handbook, residual definition and graphical model assessment: https://itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm Frank Harrell, Glossary of Statistical Terms, residual entry: https://hbiostat.org/glossary/ NIST, regression diagnostics: https://www.itl.nist.gov/div898/software/dataplot/refman1/auxillar/regrdiag.htm

Previous: Residence time (statistics) Next: Residual flow