AI Glossary / Time series and signal processing / Intermediate
ARMA model
Definition
An ARMA model combines autoregressive and moving-average components to describe a stationary time series. The autoregressive component uses previous series values, while the moving-average component uses current and previous innovations. ARMA(p, q) identifies the orders of those two components. It corresponds to an ARIMA model with zero ordinary differencing under the matching specification.
What it means
An observation can be related to recent history in two ways: previous levels may persist, and recent unexpected shocks may continue to affect the series. ARMA represents both mechanisms in one linear model. It is often used for series fluctuating around a stable mean after any appropriate preprocessing.
The innovations are unpredictable disturbances relative to the model's information set. They are not directly observed in the way the series is; fitted models infer them. The moving-average terminology does not mean calculating a simple rolling mean of observations. A model can contain only an AR component or only an MA component as a special case.
How it works
For a centered ARMA(1, 1), one common convention is y_t - mu = phi(y_(t-1) - mu) + e_t + theta e_(t-1). Here mu is the mean, phi controls the autoregressive term, and theta controls the previous innovation's contribution. Sign conventions for the MA coefficient vary between references.
Parameters are estimated from the observed series, typically with likelihood-based methods. Stationarity constraints control the AR dynamics, and invertibility supports a suitable innovation representation. In the first-order example these commonly require |phi| < 1 and |theta| < 1. Higher-order conditions involve polynomial roots. Analysts then inspect residual dependence, compare alternative orders and evaluate forecasts in chronological order rather than selecting solely for training fit.
Three practical examples
1. Process monitoring: Engineers model a stable sensor series after removing known deterministic effects. ARMA captures remaining short-term dependence before unusual deviations are investigated.
2. Workload analysis: A team models fluctuations around a stable baseline in regularly observed service activity. It checks whether changing business conditions make the stationarity assumption implausible.
3. Regression diagnostics: Researchers find serial dependence after explaining a time series with external variables. An ARMA error component can represent residual dynamics while the regression describes the systematic predictors.
Think of it this way
Think of a room's temperature retaining some of yesterday's deviation from its usual level while a recent unexpected heat pulse also has a lingering effect. AR and MA terms describe different pieces of that memory. The analogy is statistical: fitted coefficients do not establish the physical mechanism that caused the persistence.
Worked example
Use an original specified ARMA(1, 1) model with mean mu = 10, phi = 0.6 and theta = 0.3 under the plus-sign convention above. Suppose the latest observation is 14 and the latest inferred innovation is 2. Future innovations have conditional mean zero.
The one-step forecast is 10 + 0.6(14 - 10) + 0.3 × 2 = 10 + 2.4 + 0.6 = 13. For the two-step forecast, substitute the first forecast for the unknown next observation and set the future innovation's conditional expectation to zero. This gives 10 + 0.6(13 - 10) = 11.8.
The three-step forecast is 10 + 0.6(11.8 - 10) = 11.08. The forecast path moves toward the mean rather than remaining at the last observation. The equivalent intercept in the uncentered equation is (1 - 0.6) × 10 = 4, not 10. These are conditional point forecasts with specified parameters and innovation history, not estimates obtained from the few numbers shown.
Illustrative case study
Fictional case: Maple Operations monitors a production measurement that fluctuates around a stable operating target. Its first dashboard treats consecutive measurements as independent, causing a long run above the target to be interpreted poorly. The analytics team investigates an ARMA description of the normal operating dynamics.
It fits candidates on an established baseline period, checks residual patterns and evaluates later forecasts. During a subsequent maintenance change, the mean shifts and the old model becomes unsuitable. The team rebuilds the baseline only after reviewing the process change. It does not use an ARMA fit to explain away every persistent deviation as harmless correlation.
Common misconceptions and mistakes
ARMA is not the same as smoothing a series with a rolling average. Its MA component models dependence on innovations. The mean is not generally equal to the intercept in the recursive equation, as the worked example demonstrates.
A high autoregressive coefficient does not establish causality, and a stationary fitted model does not prove the real process will remain stable. Adding many AR and MA terms can create unstable estimates or nearly canceling components. Normality is not part of the basic definition of a weak white-noise innovation sequence, although particular estimation and interval procedures may impose distributional assumptions.
Business use
ARMA provides a compact description of short-term dependence that can support forecasting, monitoring and more realistic uncertainty calculations. It is most useful when the series' statistical behavior is sufficiently stable for the intended period and when a small number of terms captures relevant dependence.
Business users should request the actual equation, observation frequency, baseline period and forecast validation results. If trend, seasonality, external events or changing variance dominate, those features need appropriate treatment. A technically valid fit can still be a poor business tool when it is evaluated at the wrong horizon or applied after the process changes.
People also ask
Does ARMA difference the series?
The ARMA model itself has no integrated differencing component; preprocessing may have produced the series being modeled.
Are innovations the same as raw observations?
No. They represent model-relative unexpected contributions.
Must both p and q be positive?
No. Pure autoregressive and pure moving-average models are special cases.
Does stationarity imply independence?
No. A stationary series can have substantial serial correlation.
Check your understanding
What is the one-step forecast in the example?
Answer: It is 13.
Why is the uncentered intercept 4?
Answer: It equals (1 - phi) times the mean for this first-order model.
Why is the unknown next innovation replaced by zero in the two-step expectation?
Answer: Its conditional mean is assumed to be zero.
Related terms
- ARIMA model
- Stationarity
- Autocorrelation function
- White noise
- autoregression
- moving-average process
- innovation
- invertibility
Learn next
Learn next how AR and MA components produce different autocorrelation patterns. Then study likelihood estimation, residual checking and forecast uncertainty, paying attention to software sign and intercept conventions.
More in Time series and signal processing
- Adaptive response rate - Adaptive response rate describes a forecasting approach that changes how strongly it reacts to new observations according to evidence of changing conditions.
- Analog Filter Design - Analog filter design is the process of specifying a continuous-time system that changes the amplitude or phase of different frequency components of a signal.
- Assumed-density filter - An assumed-density filter is a sequential Bayesian approximation method that keeps the evolving probability distribution within a chosen manageable family.
- Augmented Lagrangian - An augmented Lagrangian is an optimization function that combines the original objective, multiplier terms for constraints and an additional penalty for...
- Autocorrelated errors - Autocorrelated errors are model errors that are correlated with errors at other times or ordered positions.
- Autocorrelation - Autocorrelation is the correlation between a variable and a time-shifted version of itself.
Sources
Hyndman and Athanasopoulos, Forecasting: Principles and Practice, autoregressive, moving-average and ARIMA models: https://otexts.com/fpp3/arima.html . statsmodels developers, ARIMA specification: https://www.statsmodels.org/stable/generated/statsmodels.tsa.arima.model.ARIMA.html . The example explicitly fixes its plus-sign MA convention, mean and coefficients; its recursive forecast arithmetic was independently checked.
Previous: ARKA descriptors in QSAR Next: ARPAbet