Is a generated regressor always biased?

No. Bias and consistency depend on the model and estimator; uncertainty is a related but distinct issue.

Is a fixed logarithm of an observed variable the same problem?

Not by itself. The distinctive issue here is dependence on estimated upstream quantities.

Can a bootstrap help?

Often, if it is valid for the estimator and repeats the relevant stages with an appropriate sampling scheme.

AI Glossary / Statistics and probability / Advanced

Generated regressor

Definition

A generated regressor is an explanatory variable constructed using quantities estimated in an earlier statistical step, then used in a later model. Examples include fitted values, estimated residuals, propensity scores, or estimated expectations. Because the regressor depends on estimated quantities, inference in the later model may need to account for earlier-stage estimation uncertainty. Simply treating the generated values as ordinary observed measurements can give inappropriate standard errors.

What it means

Many analyses need a predictor that is not directly observed or is represented through a fitted model. An analyst might first estimate expected demand and then study how staffing relates to that expectation. The demand estimate is a generated regressor in the second analysis.

The issue is not merely that a column was created with software. Squaring an observed measurement using a fixed formula does not introduce an estimated first-stage parameter in the same way. A predictor built from fitted coefficients carries uncertainty from the data used to estimate those coefficients.

Generated regressors are not inherently invalid. Their consequences depend on the estimator, dependence between stages, sample sizes, and target quantity. Some constructions have special properties that eliminate a first-order correction; others require substantial adjustments.

How it works

Describe the first-stage model and estimate its parameters. Use those estimates to calculate the generated predictor for the observations entering the second stage. Fit the final model and identify the inferential target: prediction conditional on a fixed upstream model, or inference accounting for estimation of that upstream model.

For suitable estimators, analytical two-step covariance formulas or joint estimating equations can propagate first-stage uncertainty. An appropriate bootstrap may repeat the entire estimation pipeline within each resample, including regeneration of the predictor. The resampling scheme must respect clustering, time dependence, and whether the stages use shared or separate samples.

With flexible machine learning, overfitting and data reuse require additional care. Sample splitting or cross-fitting can address some dependence problems, but they do not automatically repair every bias or establish valid inference without the relevant conditions.

Three practical examples

Demand research: use an estimated expected-demand measure in a later model of inventory choices, accounting for how that expectation was fitted.
Econometrics: use first-stage fitted values or residuals within a properly specified instrumental-variable or control-function estimator.
AI-assisted analysis: use an estimated customer score as a predictor in a downstream statistical model, tracking both prediction quality and uncertainty from its training process.

Think of it this way

Think of measuring a room with a ruler whose scale was itself estimated in another experiment. The final calculation may be straightforward, but uncertainty about the ruler can affect uncertainty about the room. Whether that extra uncertainty matters depends on how the ruler was constructed and how the final quantity uses it.

Worked example

Take four original records with z = (1, 2, 3, 4), a first-stage outcome x = (2, 5, 5, 8), and a later outcome y = (5, 8, 10, 13). Regress x on an intercept and z. The fitted equation is x_hat = 0.5 + 1.8z, giving generated values (2.3, 4.1, 5.9, 7.7).

Now regress y on an intercept and x_hat. The fitted equation is y_hat = 16/9 + (13/9)x_hat, so the slope is approximately 1.4444. Substituting the first-stage equation yields y_hat = 2.5 + 2.6z. Independently fitting y on z gives exactly that same line, confirming the arithmetic.

The numerical agreement does not justify using the ordinary second regression’s uncertainty for every intended two-stage interpretation. The x_hat values depend on the estimated first-stage coefficients. A full-pipeline resample would refit x on z, regenerate x_hat, and refit the second regression. This tiny example illustrates construction and algebra; it supplies neither a reliable inference exercise nor evidence that z is a valid instrument or that x causes y.

Illustrative case study

Fictional case: Willow Retail estimates customer demand scores from historical purchases and then relates staffing decisions to those scores. A draft report treats each score as if it were measured without estimation error.

The analyst documents which records trained the score, checks whether outcomes leaked into its construction, and clarifies the target of the second-stage analysis. A suitable uncertainty procedure repeats all relevant stages and preserves store-level grouping. Management receives a description of both stages and their limitations, rather than an apparently precise coefficient detached from the score’s origins.

Common misconceptions and mistakes

Assuming every computed feature is a generated regressor in the same inferential sense.
Ignoring first-stage uncertainty because the generated column has already been saved to a file.
Bootstrapping only the second regression when the intended procedure requires repeating both stages.
Assuming corrections always increase standard errors; the relationship depends on the complete estimator and covariance structure.
Treating a fitted value as a valid instrument without checking identification assumptions.
Believing cross-fitting alone makes arbitrary downstream causal claims valid.

Business use

Generated regressors are common in multi-model business workflows, so data lineage should include model lineage. Record the training sample, model version, feature processing, tuning, prediction date, and relationship between upstream and downstream datasets.

Separate predictive deployment questions from inferential research questions. A fixed, externally supplied score can be assessed conditionally for a particular prediction task, while broader claims about the score-generation process may require additional uncertainty analysis. Before publishing a coefficient or policy conclusion, ensure that the procedure answers the intended question and that validation covers the whole pipeline.

Check your understanding

Answer: 1.

Why can saving first-stage predictions fail to remove their uncertainty?

Answer: Storage does not change how the values were estimated. 2.

What must a full-pipeline bootstrap regenerate?

Answer: The first-stage estimates and resulting regressors, as well as the later fit. 3.

Does the worked example prove an instrumental-variable interpretation?

Answer: No; that requires additional assumptions about z, x, and the outcome.

Related terms

Instrumental variables
Control function (econometrics)
Propensity scores
Bootstrap
Cross-fitting
Measurement error
Two-step estimator
control function
propensity score
Murphy–Topel correction

Learn next

Study ordinary regression uncertainty, then compare a naive second-stage covariance with a suitable two-step adjustment. Map the data flow before choosing resampling or sample-splitting procedures, especially when flexible models generate the predictors.

Sources

Hahn and Ridder, “The Asymptotic Variance of Semiparametric Estimators with Generated Regressors”: https://economics.ucr.edu/wp-content/uploads/2019/10/Ridder-paper-for-4-14-10-seminar.pdf — primary definition and first-stage uncertainty discussion read.

Stata, bootstrap with panel data: https://www.stata.com/support/faqs/statistics/bootstrap-with-panel-data/ — preserving group structure in resampling. The example is an original two-regression illustration, not an identification argument.

Previous: Generalized total variation minimization Next: Generated text