Does quantile regression require normally distributed errors?

No. Normality is not required by the pinball-loss definition.

Are independently fitted quantiles always correctly ordered?

No. Crossing can occur and should be checked or addressed.

AI Glossary / Statistics and probability / Intermediate

Quantile Regression

Definition

Quantile regression models a chosen quantile of an outcome conditional on explanatory variables. Instead of estimating only the conditional mean, it can estimate a conditional median, an upper percentile, or another part of the outcome distribution. For example, a model may predict the 90th percentile of delivery time for a particular route and order type. The method is useful when the distribution's spread or shape changes with the inputs and an average alone does not answer the operational question.

What it means

A conditional quantile describes the outcome distribution for a specified feature pattern. It differs from taking a quantile of all outcomes together. A median regression targets the 0.50 conditional quantile, while a 0.90 regression targets a higher point in that distribution.

The word regression does not restrict the relationship to a straight line. Linear quantile regression uses a linear predictor, but other model classes can optimize the same quantile-oriented objective. The requested quantile should reflect the question or loss associated with underprediction and overprediction. It is not selected merely because a particular percentile produces an attractive result.

How it works

Choose a quantile level tau between zero and one and a model for its dependence on features. For residual u = observed outcome minus predicted quantile, the pinball loss is tau × u when u is nonnegative and (1 - tau) × (-u) when u is negative.

Fit the model by minimizing the sum or average of these asymmetric losses, optionally with a specified regularization penalty. At tau = 0.50, the loss is half the absolute residual. At high tau, underprediction receives more weight than an equally large overprediction, directing the fit toward an upper conditional quantile.

Evaluate on suitable held-out observations using pinball loss, coverage diagnostics, and the operational context. Estimating lower and upper quantiles can form a prediction interval, but the fitted endpoints require validation. Separately trained quantile models can cross, producing an incoherent ordering. Model misspecification, limited data, or distribution shift can prevent the desired conditional coverage even if the training objective was optimized successfully.

Three practical examples

A logistics team predicts upper delivery-time quantiles to plan for slower orders with particular route characteristics.
An AI service models latency quantiles as request size changes, allowing the spread of response times to vary with workload.
A demand-planning team compares conditional median and upper-quantile forecasts when shortages and excess inventory have different consequences.

Think of it this way

Imagine choosing a line through observations while paying different prices for being too low and too high. A higher quantile makes being too low more expensive, so the fitted line shifts toward the upper portion of the conditional distribution. The line is still an estimate, not a ceiling that every future outcome must respect. The cost interpretation applies to the stated asymmetric absolute-loss setup.

Worked example

In an original example, a binary feature identifies two task types. Type zero has outcomes 2, 4, 6, 8, and 30 minutes. Type one has outcomes 5, 7, 9, 11, and 33 minutes. Fit an unpenalized model q(x) = a + bx for tau = 0.75. Because x has only two values, the model can choose a separate prediction for each group.

For type zero, prediction 8 gives pinball losses 1.5, 1, 0.5, 0, and 16.5, totaling 19.5. Prediction 6 gives total loss 21, and prediction 10 gives total loss 20. Between successive observations the loss is linear; its slope changes from negative to positive at 8, establishing the minimum there. The shifted type-one sample has its minimum at 11.

Thus a = 8 and b = 3 minimize this example's objective. The group medians are 6 and 9, while the means are 10 and 13. Quantile regression answers a different question from mean or median prediction. Four of five observations in each group are at or below the fitted quantile; a finite sample need not contain exactly 75% below it. This calculation is a simple saturated two-group fit, not evidence of future calibration.

Illustrative case study

Fictional case study: Aspen Dispatch develops median and upper-quantile delivery forecasts. It evaluates both models on later orders and checks coverage separately for short and long routes.

The team finds that aggregate coverage can hide differences between route groups, so it investigates features and model fit before presenting the upper forecast as an operational planning threshold. This fictional case illustrates conditional evaluation, not a real accuracy result or a universal guarantee that quantile regression produces calibrated intervals.

Common misconceptions and mistakes

Quantile regression does not simply run ordinary least squares on the highest observations. It uses the full training sample with asymmetric loss. Nor is a predicted 90th percentile a 90% confidence interval for a model coefficient.

An interval between predicted lower and upper outcome quantiles is a prediction interval, whose coverage depends on the estimated distribution and evaluation setting. Robustness to large outcome residuals does not imply immunity to influential feature values or bad measurements. Finally, a quantile slope describes an association in a conditional distribution; it does not automatically identify how an intervention changes an individual's outcome.

Business use

The method can align forecasts with decisions that care about specific parts of a distribution. It can reveal differences that a single mean forecast conceals and can make uncertainty more relevant to capacity planning.

Choose quantile levels with stakeholders and record the rationale. Compare against simple baselines, inspect subgroup performance, and monitor changes after deployment. Avoid interpreting a well-performing upper quantile as a maximum or a safety guarantee. Where a decision depends on an interval, assess both its coverage and its width; very wide intervals can achieve coverage while offering little practical precision.

Check your understanding

At tau = 0.75, what is the loss for underpredicting by four minutes, and for overpredicting by four minutes?

Answer: The losses are 0.75 × 4 = 3 and 0.25 × 4 = 1, respectively.

Related terms

Pinball loss
Prediction interval
Regularization
Calibration
Conditional quantile
median regression
heteroscedasticity
quantile crossing
asymmetric loss

Learn next

Study sample quantiles and absolute loss, then derive pinball loss. Compare conditional mean and quantile models and evaluate prediction intervals on appropriately separated data.

Pinball loss

Sources

Scikit-learn's official linear-model guide explains conditional quantiles, pinball loss, regularization, and nonlinear alternatives: https://scikit-learn.org/stable/modules/linear_model.html#quantile-regression

The two-group fit, loss comparisons, and fictional case are original and independently calculated. The example uses unpenalized quantile regression and does not assume that every implementation uses the same defaults.

Previous: Quantile bucketing Next: Quantile regression averaging