Does it always use its own predictions?

During generation often yes, but forecasting and training also use observed history.

Must it be a neural network?

No. Classical autoregressive models are widely used statistical models.

Does AR(p) specify p future predictions?

No. It specifies the number of included lags.

AI Glossary / Statistics and probability / Intermediate

Auto-regressive model

Definition

An auto-regressive model predicts or models the next element of a sequence using earlier elements of that sequence. In classical statistics, an autoregressive time-series model expresses the current value using lagged values and a new disturbance. In generative AI, an autoregressive model produces successive tokens or other components by conditioning each new prediction on the preceding context.

What it means

The central idea is that the sequence's own history supplies information about what comes next. That history may contain observed values, previously generated values, or a mixture, depending on the task and stage of operation.

The supplied machine-learning glossary emphasizes generation based on previous predictions. The broader statistical meaning also includes forecasting from actual past observations. A classical linear AR model and a neural language model share the conditioning idea but have very different representations, parameters, and output distributions.

Autoregressive does not mean that every prediction is correct, that the model understands causality, or that it must use a particular neural architecture. It describes how sequential dependencies are modeled and used.

How it works

For a classical AR(p) model, p is the number of lagged values included. A common form is X_t = c + phi_1 X_(t−1) + ... + phi_p X_(t−p) + epsilon_t, where epsilon_t is a new disturbance. Parameters are estimated from time-ordered observations, and residual checks assess whether predictable structure remains.

For a standard causal stationary AR(1) with finite innovation variance, the lag coefficient has magnitude less than one. Its long-run mean is c/(1−phi), rather than generally being equal to the intercept c. More elaborate models have their own stationarity conditions.

A generative sequence model instead represents a joint probability as a product of conditional probabilities: P(x_1,...,x_T) = product of P(x_t given x_1,...,x_(t−1)). A neural network can estimate those conditionals. Generation repeatedly selects or samples a next element and appends it to the context.

Training can use observed preceding elements, while generation conditions on the model's own outputs. Errors can therefore change the context for later predictions. Sequential generation also creates latency dependencies, although implementation techniques can reduce practical cost. Neither the chain-rule factorization nor a large neural network guarantees factual accuracy or useful long-range behavior.

Three practical examples

A workload forecaster uses recent hourly measurements to predict the next hour under a fitted time-series model.
A text-generation system samples successive tokens conditioned on a prompt and the text already generated.
An image-generation research model orders pixel components and models each component conditional on those preceding it.

Think of it this way

Think of continuing a melody one note at a time. The notes already played constrain what sounds plausible next. A simple model may remember only a few recent notes; a more elaborate one can use a longer context. A plausible continuation is still a prediction, not a guarantee of what the composer intended.

Worked example

Consider an original AR(1) teaching model: X_t = 2 + 0.6 X_(t−1) + epsilon_t. Assume future innovations have conditional mean zero, are independent over time, and have variance 4. The latest observed value is 10.

The one-step conditional mean forecast is 2 + 0.6 × 10 = 8. Without a new observation, the two-step forecast uses that forecasted value: 2 + 0.6 × 8 = 6.8. The stationary mean is 2/(1−0.6) = 5, and the forecasts move toward it.

With known parameters, one-step forecast-error variance is 4. Two-step error contains epsilon_(t+2) + 0.6 epsilon_(t+1), so its variance is 4 + 0.6 squared × 4 = 5.44. The additional uncertainty is not captured by merely printing the point forecast 6.8.

For a separate discrete illustration, suppose a model assigns probability 0.4 to the first symbol A and probability 0.7 to B after A. The joint probability of the two-symbol sequence AB is 0.4 × 0.7 = 0.28. This is a conditional-product calculation, not a claim that token generation follows the linear numerical AR equation.

Illustrative case study

Fictional case: Birchline Support forecasts a continuous workload measure and also prototypes a text assistant. The forecasting team uses a small autoregressive baseline; the assistant generates tokens sequentially using a neural model.

The team documents the common idea of conditioning on history while keeping evaluation separate. Forecasts are checked on future time periods with uncertainty estimates. Generated responses are checked for task usefulness and factual support rather than judged only by fluent continuation.

The fictional project illustrates two uses of autoregression without claiming that one model can be substituted for the other. It also preserves the distinction between actual observations used for updating forecasts and generated outputs used as future context.

Common misconceptions and mistakes

Common mistakes include assuming auto-regressive means linear, confusing it with an autoencoder, and treating the intercept as the mean of every AR model. Another error is using future observations during evaluation, which leaks information unavailable at prediction time.

Not every Transformer-based model generates autoregressively; architecture and generation objective are separate choices. Likewise, a process is not autoregressive merely because its software runs in several steps. The defining feature is the specified dependence on earlier sequence elements, and classical stationarity assumptions should not be transferred indiscriminately to neural token models.

Business use

Autoregressive models support forecasting and sequential generation. A simple AR baseline can reveal how much predictive value comes from recent history, while neural autoregressive models can represent complex conditional distributions.

Choose a model and evaluation procedure suited to the sequence and intended use. Monitor changes in the data-generating process, evaluate beyond one-step performance when longer sequences matter, and account for latency and uncertainty. For generated factual content, add appropriate evidence and verification; predictive likelihood alone does not establish truth.

Check your understanding

Why is the two-step forecast 6.8 rather than another 8?

Answer: Without a new observation, the model conditions the second forecast on the first forecast of 8, giving 2 + 0.6 × 8.

Related terms

Learn next

compare AR and moving-average models, study time-series validation, and examine conditional sequence likelihood and sampling in neural generation.

Sources

Sources: Google Machine Learning Glossary, auto-regressive model, https://developers.google.com/machine-learning/glossary; Hyndman and Athanasopoulos, Forecasting: Principles and Practice, autoregressive models, https://otexts.robjhyndman.com/fpp3/arima.html; van den Oord, Kalchbrenner, and Kavukcuoglu, Pixel Recurrent Neural Networks (2016), https://arxiv.org/pdf/1601.06759. Numerical examples are original; architecture and autoregressive objective are deliberately distinguished.

Previous: Authorship attribution Next: Autoassociative memory