Can future predictors be unknown?

Yes, but they then need forecasts or scenarios.

AI Glossary / Time series and signal processing / Advanced

Regression with ARIMA errors

Definition

Regression with ARIMA errors is a forecasting model that combines a regression relationship with an ARIMA model for the regression's errors. Predictors explain part of the response, while the error model represents remaining dependence over time. The final forecast combines forecasts from both parts.

What it means

Ordinary regression can leave a pattern in its errors: an unusually high outcome may be followed by another unusually high outcome. This model explicitly represents such persistence instead of treating every regression departure as unrelated noise.

Distinguish the regression error from the innovation in its ARIMA model. The former may be autocorrelated. The latter should have the assumed unpredictable, white-noise behavior after the model has accounted for time dependence. Neither component automatically establishes a causal relationship between a predictor and the outcome.

How it works

Write the response as y_t = beta_0 + beta_1 x_t + u_t, and specify an ARIMA process for u_t. Estimate the regression and error structure appropriately, then examine innovation residuals and forecast performance.

Check whether differencing or other treatment of nonstationarity is needed. When differencing a regression relationship, handle the response and predictors consistently; cointegrated relationships require particular care. Merely adding predictors to a trending series does not make the resulting relationship trustworthy.

Future predictor values must be supplied, forecast or set as scenarios. Intervals conditional on those values do not automatically incorporate uncertainty in their forecasts. Software parameterizations also differ, especially in how an intercept is represented, so compare equations as well as labels.

Three practical examples

1. Service demand: forecast workload using known calendar features while modeling persistent departures from the calendar-based expectation.

2. Production planning: relate output requirements to supplied demand scenarios and account for serially correlated deviations.

3. Energy forecasting: use temperature assumptions alongside a model for remaining time dependence, documenting uncertainty in the weather inputs.

Think of it this way

Think of estimating a journey from its planned route while also accounting for a traffic delay that persists for a while. The route explains one part of the travel time; the lingering delay explains another. Ignoring either can distort the forecast.

Worked example

Consider an invented weekly workload index modeled as:

y_t = 100 + 3x_t + u_t,

u_t = 0.5u_(t-1) + e_t.

Assume the innovations e_t are independent Gaussian variables with mean zero and variance 4. All parameters are known for this teaching example. The error process is ARIMA(1,0,0), an autoregressive special case of ARIMA. No differencing or moving-average term is required in this stipulated model.

At the forecast origin, the latest regression error is known to be u_T = 8. Suppose the next three predictor values are fixed at 10, 12 and 11. These are scenario inputs, not values inferred from future outcomes.

The regression-only forecasts are 100 + 3 × 10 = 130, then 136 and 133. The expected regression errors are 0.5 × 8 = 4, then 0.5^2 × 8 = 2 and 0.5^3 × 8 = 1. Adding the two parts produces forecasts of 134, 138 and 134.

The persistent departure shrinks because its coefficient is 0.5. Setting all future errors to zero immediately would omit the information contained in the latest observed departure. Conversely, carrying the full departure of eight into every future week would ignore the specified decay.

Now calculate conditional forecast uncertainty. One week ahead, only the next innovation is unknown, so forecast-error variance is 4. Two weeks ahead, the error is 0.5e_(T+1) + e_(T+2). Independence makes its variance 4 × (0.5^2 + 1) = 5.

Three weeks ahead, the error is 0.25e_(T+1) + 0.5e_(T+2) + e_(T+3). Its variance is 4 × (0.25^2 + 0.5^2 + 1) = 5.25. We square the coefficients when adding independent variance contributions; adding the coefficients themselves would give the wrong result.

Using the conventional rounded Gaussian multiplier 1.96, approximate 95% prediction intervals are 134 ± 1.96 × 2, or [130.08,137.92], and 138 ± 1.96 × sqrt(5), approximately [133.62,142.38]. The three-week interval is approximately [129.51,138.49]. These intervals concern individual future index values, conditional on the stated predictor path, known parameters and latest error.

Suppose the second predictor input changes from 12 to 14, with everything else held fixed. The second forecast rises by 3 × 2 = 6, from 138 to 144. This scenario calculation does not prove that intervening to increase the predictor would cause workload to rise by six.

There is also an equivalent equation useful for checking parameterization. Substitute u_(t-1) = y_(t-1) - 100 - 3x_(t-1) into the error equation. The result is y_t = 50 + 0.5y_(t-1) + 3x_t - 1.5x_(t-1) + e_t. The constant is now 50, while the original regression intercept was 100. The equations describe the same stipulated model; the changed constant does not indicate a conflicting estimate.

Illustrative case study

Fictional Cedar Operations forecasts workload from a planned activity index. Its analyst notices that recent departures remain positive for several weeks and considers the model above.

Cedar compares forecasts against simpler alternatives using past forecast origins. It reports the assumed future activity path alongside the intervals and checks the innovation residuals. This invented case describes an evaluation process, not a claim that ARIMA errors always improve business forecasts.

Common misconceptions and mistakes

"All residuals must be uncorrelated." The innovation residuals are the relevant target; regression departures may persist.

"Predictor scenarios are certain future observations." They are assumptions unless genuinely known.

"Any reported intercept has the same meaning." Check the model equation.

"A forecasting coefficient proves causation." It does not.

Business use

Record the forecast date, available inputs and scenario assumptions. Evaluate forecasts using information that would actually have been available at each historical origin. Explain whether intervals include predictor and parameter uncertainty, rather than presenting a conditional interval as a complete account of future risk.

Check your understanding

What is the first combined forecast?

Answer: 134.

Why is the two-week variance five?

Answer: Two independent innovations contribute with squared weights.

Does the constant 50 contradict the intercept 100?

Answer: No; they belong to equivalent parameterizations.

Related terms

Autocorrelation
Prediction interval
Stationarity
Cointegration
ARIMA
dynamic regression
innovation
exogenous predictor

Learn next

Replace the latest regression error of eight with minus four. Recalculate the three conditional means, keeping the model parameters and predictor path unchanged. Explain why the innovation-based forecast variances remain the same.

Conditional means

Sources

Rob J Hyndman and George Athanasopoulos, Forecasting: Principles and Practice, Dynamic regression models: https://otexts.com/fpp3/dynamic-regression.html

Statsmodels developers, SARIMAX and ARIMA: Frequently Asked Questions, model equations and intercept parameterizations: https://www.statsmodels.org/stable/examples/notebooks/generated/statespace_sarimax_faq.html

Previous: Regression tree Next: Regressor chains