Does LAD require normally distributed errors?

No. Its fitting criterion does not require normality, although inference still relies on assumptions.

Can LAD be regularized?

Yes. A coefficient penalty can be added, but that produces a distinct penalized specification.

Will LAD always outperform least squares?

No. Performance depends on the target, loss, data distribution, model specification and evaluation setting. Their predictions may answer different questions even when both are well fitted.

AI Glossary / Statistics and probability / Intermediate

Least absolute deviation regression

Definition

Least absolute deviation regression, often called least absolute deviations or LAD regression, estimates a model by minimizing the sum of absolute residuals between observed responses and fitted values. For a linear model, it chooses coefficients that minimize the total of |y minus predicted y| across observations. Unpenalized LAD is equivalent to median quantile regression, because the quantile loss at probability 0.5 differs only by a positive constant. It targets a conditional median under an appropriate model, rather than the conditional mean typically associated with squared-error regression.

What it means

Large residuals receive quadratic emphasis in least squares but only linear emphasis in LAD. A response error of twenty units therefore contributes twenty units to an absolute-error objective, rather than four hundred to a squared-error objective. This can reduce the pull of extreme response values. However, changing the loss also changes the statistical target. When responses are skewed, the conditional median and mean can differ for legitimate reasons. Choosing LAD should reflect the business loss and desired prediction, not a blanket assumption that all large observations are mistakes.

How it works

Specify the response, predictors, intercept and any observation weights. Form the sum of absolute residuals and solve the resulting optimization problem. For a model linear in its coefficients, auxiliary variables can represent absolute values, producing a linear program. The objective is convex but has corners, and the minimizing coefficients need not be unique. Evaluate predictions using appropriate held-out data and metrics. If regularization is added, report its penalty separately: an L1 penalty on coefficients is not the same as an L1 loss on residuals. Standard errors also require methods suitable for quantile regression.

Three practical examples

Service-time prediction: estimate a typical conditional completion time when occasional delays produce a long upper tail.
Operational forecasting: fit a baseline aligned with approximately symmetric absolute costs for overprediction and underprediction.
Model comparison: examine how a median-based fit differs from a mean-based fit when unusual responses affect business interpretation. Rare but important outcomes may still require a separate risk or tail model.

Think of it this way

Think of choosing a meeting point that minimizes everyone's total travel distance along a line. A very distant participant matters, but their contribution grows in direct proportion to distance. Squaring distances would give that participant much stronger influence. With predictors, LAD extends this balancing idea to a fitted relationship, although unusual predictor locations create additional leverage that the simple meeting-point analogy does not remove.

Worked example

Consider an intercept-only model for responses 1, 2, 2, 3 and 22. The median is 2, so the LAD prediction is 2. Its absolute residuals are 1, 0, 0, 1 and 20, totaling 22. The least-squares prediction is the mean, 6, whose absolute residuals total 5 + 4 + 4 + 3 + 16 = 32. Conversely, the squared residual sum is 402 at prediction 2 and 322 at prediction 6. Each method wins on the objective it optimizes. If the largest response changes from 22 to 222, the median remains 2 while the mean becomes 46. This is a simple intercept-only illustration of resistance to an extreme response, not a guarantee that a multivariable LAD fit withstands arbitrary contamination or high-leverage predictor values.

Illustrative case study

In a fictional case, Harbor Repairs predicts job duration for customer planning. Most repairs finish within a moderate range, while parts shortages create occasional long delays. Analysts compare mean and median models using future-period validation. They choose a median estimate for the standard scheduling display and separately communicate delay risk. They do not describe the median as the expected average workload, and they retain extreme jobs when they represent real operations. The choice follows the use of the prediction rather than an attempt to make unusual cases disappear.

Common misconceptions and mistakes

LAD is not lasso: the former describes an absolute residual loss, while lasso ordinarily combines squared residual loss with an absolute coefficient penalty. Robustness to large response residuals does not imply protection against influential predictor outliers. LAD does not automatically set irrelevant coefficients to zero. Its fitted values need not equal the sample median within every arbitrary subgroup. Nonunique solutions and ties can affect coefficients. Ordinary least-squares confidence intervals should not be reused unchanged, and a low absolute training error does not establish generalization or causal interpretation.

Business use

For businesses, the main benefit is alignment with a median prediction and an absolute-error criterion. Define whether the decision needs a mean, median or another quantile before selecting the model. Under asymmetric costs, a different quantile may be more appropriate. Inspect residual patterns and influential predictor configurations, compare predictive performance across relevant groups, and preserve a consistent validation procedure. Report uncertainty in a way appropriate to the fitted model. A stable central prediction should not conceal operational exposure to unusually large outcomes.

Check your understanding

Why does prediction 2 have larger squared error than prediction 6 in the example?

Answer: Prediction 2 minimizes absolute error, not squared error. The outlying response has much greater influence under squared loss, pulling the mean to 6. Comparing methods fairly requires identifying which target and loss matter for the intended decision.

Related terms

Learn next

Study quantile regression at probabilities other than 0.5 and connect asymmetric loss to decision costs. Then examine robust regression diagnostics, high-leverage observations and uncertainty estimation for conditional quantiles.

Sources

NIST Dataplot, Weights documentation, LAD and leverage discussion: https://www.itl.nist.gov/div898/software/dataplot/refman1/ch5/weights.pdf . Scikit-learn, QuantileRegressor: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.QuantileRegressor.html . The numerical comparison and fictional case above are original illustrations.

Previous: Learning vector quantization Next: Least Angle Regression