Does linear prediction require normal data?

No. The best-linear calculation relies on second moments; normality is relevant to stronger claims about the full conditional distribution.

Must weights add to one?

Not in general. Particular unbiasedness constraints or applications can impose that condition.

Is it always autoregression?

No. Autoregression uses a variable's own past values; linear prediction can use other variables or spatial measurements.

AI Glossary / Statistics and probability / Intermediate

Linear prediction

Definition

Linear prediction estimates an unknown quantity using a weighted sum of available observations or features, often with an intercept. In time-series and signal processing, it commonly predicts a current or future value from past values of the same series. More broadly, it includes predicting an outcome from other variables through a linear rule. The best linear predictor minimizes an expected loss, usually mean squared error, within the chosen class of linear rules; it need not be the best possible nonlinear predictor.

What it means

The task is to choose useful weights for information already available at prediction time. An observation can receive a positive, negative, or zero weight, depending on its relationship with the target and with other inputs.

Linear prediction is not necessarily a straight-line extrapolation through time. A weighted combination of several lagged values can track oscillations or other patterns. Nor does it necessarily involve forecasting: estimating a missing spatial measurement from nearby sensors is also prediction.

How it works

With target Y and predictor vector X, use a + b-transpose X. For finite second moments and an invertible predictor covariance matrix Sigma, the minimum-mean-square coefficients satisfy b = Sigma inverse Cov(X,Y). The intercept is E(Y) - b-transpose E(X).

This calculation accounts for relationships among predictors rather than choosing each weight independently. In practice, means and covariances are usually estimated, introducing additional uncertainty. A singular covariance matrix requires special handling, such as removing redundant predictors or choosing an appropriate generalized inverse.

Time-series versions use lagged observations and their autocovariances. Training and validation must respect what information would actually have been available. The signal-processing convention for stored filter coefficients may reverse their signs relative to the prediction weights, so implementation formulas must be checked.

Three practical examples

A monitoring system predicts a sensor reading from its recent history and investigates unusually large prediction errors.
A speech-processing system models predictable signal structure using past samples, leaving a residual that carries information not captured by the predictor.
A spatial analyst estimates an unavailable measurement from neighboring observations using a stated covariance structure, while reporting prediction uncertainty rather than presenting an estimate as a direct measurement.

Think of it this way

Think of combining reports from several informants. If two informants usually repeat the same information, counting each as fully independent can overstate their contribution. A linear predictor allocates weights using the target relationships and redundancy among inputs. Even the best combination can miss information that none of the selected informants supplies.

Worked example

Construct a population example with E(Y) = 10, E(X1) = 2, and E(X2) = 4. Suppose Var(X1) = 4, Var(X2) = 9, Cov(X1,X2) = 1, Cov(X1,Y) = 3, and Cov(X2,Y) = 5. Let Var(Y) = 16.

The predictor covariance matrix has rows (4,1) and (1,9), with determinant 35. Solving the two equations 4b1 + b2 = 3 and b1 + 9b2 = 5 gives b1 = 22/35 and b2 = 17/35.

The intercept is 10 - (22/35)x2 - (17/35)x4 = 6.8. For observed inputs X1 = 3 and X2 = 6, the prediction is 6.8 + (22/35)x3 + (17/35)x6 = 11.6.

Its minimum expected squared error is 16 - [3x(22/35) + 5x(17/35)] = 409/35, approximately 11.6857. The positive error confirms that useful prediction is not perfect prediction. These are assumed population moments, not estimates from a sample. The calculation does not require a normal distribution, although normality would support additional distributional conclusions.

Illustrative case study

Fictional case: Rivergate Utilities uses a weighted combination of recent sensor readings to predict the next reading. It evaluates the predictor chronologically and compares it with simply repeating the last value.

During a change in equipment operation, prediction errors become persistently positive. The team checks whether the mean level has shifted and whether previously estimated lag relationships still apply. It does not automatically label every error as a machine fault.

The predictor remains an operational baseline, with retraining and alert thresholds assessed on later data. Engineers distinguish unusual observations from predictable changes in operating conditions, keeping the model's role and limitations visible.

Common misconceptions and mistakes

Do not confuse best linear prediction with the unrestricted conditional expectation E(Y | X). They coincide under suitable conditions, including a jointly Gaussian setting, but need not coincide generally.

Do not use future observations to evaluate a purported real-time forecast. Also avoid assuming that a zero residual correlation with included predictors means complete independence or absence of nonlinear structure. Adding many highly correlated lags can destabilize estimated coefficients. A small in-sample prediction error is not evidence that performance will persist after a process changes.

Business use

Linear prediction supports forecasting baselines, signal compression, sensor validation, and missing-value estimation. It provides a clear benchmark for deciding whether a more complex AI method adds useful predictive information.

Document the forecast horizon, input availability, coefficient convention, and evaluation period. Compare errors against operational costs: a model that reduces average squared error may still miss rare expensive failures. If uncertainty calculations assume known covariances, explain whether estimating those covariances adds uncertainty in the actual application.

Check your understanding

If two predictors are strongly correlated, can their optimal weights be calculated using only each predictor's covariance with the target?

Answer: Generally no. The covariance between predictors matters too. The worked example solves a joint system that accounts for their shared information.

Related terms

Learn next

Next, compare a best linear predictor with the conditional mean. Then examine lag selection, chronological validation, and prediction intervals when model parameters must be estimated.

Sources

Sources: Shalizi, Linear Prediction for Spatial and Spatio-Temporal Fields, primary university lecture formulas: https://www.stat.cmu.edu/~cshalizi/dst/26/lectures/06/lecture-06.html . MathWorks, lpc, official linear-prediction documentation and coefficient convention: https://www.mathworks.com/help/signal/ref/lpc.html . Both sources were read. The covariance example and fictional case are original; the coefficient system, prediction and expected squared error were independently calculated.

Previous: Linear planning Next: Linear predictive coding