Must the quantile model be perfectly accurate?

No. The marginal validity guarantee does not require perfect quantile estimates, though poor estimates can produce unhelpful intervals.

Does 90% coverage apply to every input?

Not under the ordinary marginal guarantee. Conditional or subgroup coverage is a separate question.

Can a neural network supply the quantiles?

Yes. CQR can wrap different quantile-regression methods, provided the procedure's assumptions and data separation are respected.

AI Glossary / Probabilistic machine learning / Intermediate

Conformalized quantile regression

Definition

Conformalized quantile regression, or CQR, combines estimated lower and upper conditional quantiles with a conformal calibration step to produce prediction intervals. Under the required exchangeability assumptions, it provides a finite-sample marginal coverage guarantee while allowing interval widths to vary with the input.

What it means

A single forecast does not show how uncertain an outcome is. Quantile regression can estimate a lower and upper boundary that changes with the situation: one delivery route may be predictable while another has highly variable timing.
Estimated quantiles alone may miss their intended coverage because the fitted model is imperfect. CQR uses separate calibration examples to adjust the initial interval. The resulting guarantee concerns a new exchangeable example averaged over the relevant randomness. It does not promise the stated coverage for every individual route, customer, or other subgroup.

How it works

In the split version, fit lower and upper quantile models on a training set. Keep the calibration set separate from that fitting. For each calibration pair, calculate a score equal to the larger of lower_prediction minus outcome and outcome minus upper_prediction.
A positive score measures how far the outcome lies outside the initial interval. A negative score means it lies inside both boundaries. With m calibration examples and desired miscoverage alpha, choose the score at ordered rank ceil((m + 1)(1 − alpha)), using the appropriate infinite-threshold convention if the rank exceeds m.
Subtract that threshold from a new lower prediction and add it to the upper prediction. The threshold may expand or shrink intervals. Preserve the data-separation and exchangeability conditions when selecting models and applying the procedure.

Three practical examples

Delivery planning: produce wider predicted time intervals for variable routes and narrower ones for more predictable routes, then check subgroup performance separately.
Service demand: present a range for incoming request volume when variability changes with known features, using a calibration design appropriate to the data's dependence.
Production timing: estimate job-duration intervals from machine and job characteristics, evaluating whether observed coverage remains adequate after process changes.

Think of it this way

Think of tailoring a jacket and then adjusting its seams after an independent fitting. The initial shape comes from the quantile model; calibration supplies a final correction. The analogy stops at individual fit: CQR's ordinary guarantee is marginal across exchangeable cases, so it does not ensure every particular subgroup receives a perfectly fitted interval.

Worked example

Consider an original toy calibration with nine examples and desired coverage 80%, so alpha = 0.2. Suppose their sorted scores are −2, −1, −0.5, 0, 0.5, 1, 1.5, 2, and 3.
The corrected rank is ceil((9 + 1) × 0.8) = 8. The eighth score is 2. For a new case with initial quantile predictions 10 and 18, the conformalized interval is [10 − 2, 18 + 2] = [8, 20]. Its width is 12 instead of 8.
For another case with initial predictions 30 and 50, the same calibration threshold gives [28, 52], whose width is 24. The intervals retain different widths because the underlying quantile predictions differ.
To check one score, an outcome 21 against an initial interval [10, 18] has score max(10 − 21, 21 − 18) = 3. The example demonstrates the calculation, not empirical proof that a deployed system achieves 80% coverage.

Illustrative case study

Fictional case: Linden Repairs predicts completion times for equipment jobs. Its analysts fit two quantile models using one dataset and reserve another for calibration. They report both interval coverage and average width on an additional evaluation set.
Although overall coverage is acceptable in this fictional exercise, a newly introduced job type is frequently missed. The team investigates that subgroup and changing data conditions before using the intervals for commitments. It does not reinterpret an overall guarantee as a guarantee for every job category. No actual company performance is claimed.

Common misconceptions and mistakes

Distribution-free does not mean assumption-free. Exchangeability is central to the standard guarantee; ordinary time series and changing environments require additional care or methods suited to those settings.
These are prediction intervals for future outcomes, not confidence intervals for a model parameter or conditional mean. Calibration is not simply taking an uncorrected empirical percentile. Reusing calibration outcomes to repeatedly select the most attractive model can undermine the intended procedure.
A negative correction is possible because calibration scores can be negative. CQR therefore need not only widen intervals, and interval quality still depends on the usefulness of the fitted quantiles.

Business use

CQR can make uncertainty easier to act on when prediction difficulty varies across cases. Businesses should evaluate the tradeoff between coverage and interval width in the units of the actual decision.
Record the target coverage, calibration sample size, splitting procedure, and assumptions about new cases. Monitor coverage by relevant groups and over time without claiming those diagnostics create stronger guarantees automatically. If the data-generating process changes, investigate and recalibrate through an appropriate protocol rather than continuing to quote an old nominal percentage.

Check your understanding

With 19 calibration scores and alpha = 0.1, which ordered score supplies the usual finite-sample correction?

Answer: Rank ceil((19 + 1) × 0.9) = 18. Use the eighteenth smallest score, rather than assuming an arbitrary software percentile convention is equivalent.

Related terms

Quantile Regression
Conformal prediction
Prediction interval
Pinball loss
calibration set
exchangeability
marginal coverage
heteroscedasticity

Learn next

Learn conditional quantiles and the distinction between prediction and confidence intervals. Then calculate calibration scores by hand, including negative scores. Finally, compare overall coverage with group-specific coverage on held-out examples and study why exchangeability matters to the guarantee.

Sources

Romano, Patterson, and Candès, “Conformalized Quantile Regression,” original paper, section 4, Algorithm 1 and Theorem 1: https://papers.neurips.cc/paper_files/paper/2019/file/5103c3584b063c431bd1268e9b5e76fb-Paper.pdf Independent source review: Original CQR section 4, Algorithm 1 and Theorem 1 were read, including signed scores, corrected quantile and exchangeable marginal coverage.

Previous: Conformal prediction Next: Confounded