Does the chain order matter?

Yes. It changes which predicted outputs each regressor can use and can change error propagation.

Must every stage use a linear regressor?

No. A chain can wrap suitable regression estimators, subject to the implementation's requirements.

Does cross-validation inside the chain replace a final test set?

No. It can generate training features for later stages, but the complete selected system still requires independent evaluation.

AI Glossary / AI and machine learning / Intermediate

Regressor chains

Definition

A regressor chain is a method for predicting several numerical outputs by arranging individual regression models in an ordered sequence. Each model receives the original input features and information about outputs earlier in the chain. At prediction time, those additional inputs are predictions from earlier models, allowing later models to use relationships between outputs.

What it means

Suppose an input describes a planned delivery and the outputs are loading time, travel time and unloading time. Independent regressors predict each quantity separately. A chain can pass its predicted loading time into later regressors, alongside the original delivery features. The arrangement is a modeling choice, not a statement that one outcome causes another. All outputs can concern the same event, so a chain need not be a time-series forecast. Whether chaining improves predictions depends on the data, the underlying regressors and how well the training setup matches actual use.

How it works

Choose an output order and a base regression method. Fit the first regressor using original features. Fit the next with original features plus earlier target information, and continue until every output has a model. Training may use observed earlier targets or cross-validated predictions of those targets. During deployment, predict in chain order and append each prediction before evaluating the next model. True earlier targets used during training can be cleaner than the predictions available later, creating a mismatch. Out-of-fold predictions can better represent that situation, but their construction must respect groups, time and all preprocessing boundaries. The entire chain still needs evaluation on separate data.

Three practical examples

A warehouse predicts several handling durations for the same order, testing whether earlier duration estimates help later estimates.
A materials experiment predicts multiple measured properties from a shared input description, using their statistical relationships while avoiding causal claims.
A building model predicts several zone temperatures from weather and operating settings. A chain can connect these numerical outputs, but it does not replace physical checks or guarantee consistency between rooms.

Think of it this way

Think of several estimators completing a planning worksheet in sequence. Each can see the original brief and the estimates already written above. That can provide useful context, but a poor early estimate may influence several later answers. The analogy stops at information flow: actual regressors implement mathematical functions, and the order is selected and validated rather than determined by workplace seniority.

Worked example

Consider a deliberately simple, already-fitted two-output chain. Its first model predicts y1 as 2x. Its second predicts y2 as x + 0.5 times the predicted y1. For x = 3, the first prediction is 6, and the second is 3 + 0.5 × 6 = 6. Suppose the observed outcomes, revealed only afterward, are y1 = 8 and y2 = 7. The absolute errors are therefore 2 and 1. Substituting the true y1 into the second model would produce 7, but that would use information unavailable in the stated prediction task. It cannot be reported as the chain's deployed result. A first-stage error of −2 changes the second prediction by −1 in this particular linear example. Other fitted models can transmit errors differently. These equations illustrate inference; they are not the output of an actual training experiment.

Illustrative case study

In a fictional project, Alder Dispatch compares independent duration regressors with several chain orders. The team initially sees excellent results when later models receive actual earlier durations in an evaluation worksheet. They correct the test to pass predicted durations, then compare models on deliveries from a later period. They also inspect which stages accumulate error and whether extra prediction latency is acceptable. This is an illustration of a fair evaluation process, not a claim that chaining achieved a particular business saving.

Common misconceptions and mistakes

A chain is not automatically superior to independent models. Predicted earlier outputs are functions of the same original inputs, so they do not create new observed information at inference time; they change the model's representation and inductive structure. Correlated targets alone do not guarantee a gain. Avoid assuming the natural spreadsheet column order is best, or selecting an order after inspecting final test results. Also distinguish a regressor chain from a classifier chain: the former predicts numerical quantities, while the latter concerns class labels.

Business use

Start with a multi-output problem whose targets are available together in historical records and whose deployment inputs are clearly specified. Establish an independent-regressor baseline, choose meaningful per-output error measures and compare several justified orders using training-side validation. If targets use different units, a pooled error can conceal a poor result on one output. Document which values are predicted versus observed at each stage. Where an early outcome truly becomes known before later decisions, define that separate operating scenario explicitly rather than silently changing the original prediction task.

Check your understanding

A chain predicts two quantities before either is observed. May its second stage use the actual first quantity when calculating test performance?

Answer: No. It must receive the first-stage prediction under that task definition. Actual values would evaluate a different, better-informed system and give a misleading picture of deployment performance.

Related terms

Classifier chains
Error propagation
multi-output regression
classifier chain
out-of-fold prediction
stacking
target leakage

Learn next

fit a two-target toy problem with independent regressors and both possible chain orders. Evaluate all three on identical held-out rows, record each target's errors, and compare training with true earlier targets against training with out-of-fold predictions.

Sources

Sources: scikit-learn, RegressorChain API, especially order, cv and prediction behavior, https://scikit-learn.org/stable/modules/generated/sklearn.multioutput.RegressorChain.html ; multi-output regression guide, https://scikit-learn.org/stable/modules/multiclass.html#regressorchain . The worked equations and fictional dispatch scenario are original illustrations of sequential prediction and evaluation boundaries.

Previous: Regression with ARIMA errors Next: Regret