Does lower AIC mean lower forecast error?

Not on every future sample or horizon.

Yes. Its absolute sign is not a quality threshold; valid relative comparisons matter.

Does BIC always choose fewer parameters?

No. A large enough likelihood improvement can outweigh its penalty.

Should ARIMA differencing be selected by ordinary AIC comparison?

Usually use separate reasoning or tests because differing transformations can make the likelihood basis incompatible.

AI Glossary / Time series and signal processing / Intermediate

Order selection criteria

Definition

Order selection criteria are rules for choosing a model's structural complexity, such as how many autoregressive or moving-average terms a time-series model contains. Common criteria combine goodness of fit with a penalty for estimated parameters. Their purpose is to guide comparison among suitable candidates, not to prove that a selected model is true.

What it means

An order describes a model's structure rather than the numerical values of its fitted coefficients. An autoregressive order of two, for example, allows two past observations to enter the prediction equation. Increasing order can improve historical fit while adding estimation uncertainty and unnecessary flexibility.
A criterion turns that tradeoff into a comparison score. Different criteria emphasize different objectives and can legitimately choose different models. The result also depends on the candidate set, estimation method, data, and parameter-counting convention.

How it works

For comparable maximum-likelihood fits, AIC is minus twice the maximized log likelihood plus twice the number of estimated parameters. BIC instead uses the parameter count multiplied by the logarithm of sample size as its penalty. Smaller values are preferred within a valid comparison.
A commonly used small-sample correction is AICc = AIC + 2k(k+1)/(n-k-1), with k the total parameter count under the relevant model convention. The denominator must be positive; the correction grows when the sample is small relative to complexity. Include estimated variance and other fitted quantities where the model's definition requires them.
Compare compatible likelihoods fitted to the same response observations. In routine ARIMA selection, differencing order is usually chosen separately before comparing autoregressive and moving-average orders. Changing differencing can change the likelihood's data basis. Residual checks and evaluation on later observations remain necessary.

Three practical examples

A staffing analyst compares several autoregressive orders using AICc after fixing the data preparation and differencing choices.
A production team uses BIC to prefer a more compact candidate when additional lag terms offer only a small likelihood improvement.
A forecasting service combines an information-criterion shortlist with rolling-origin checks because the best historical score may not give the most useful forecasts at its operating horizon.

Think of it this way

Think of selecting a bag for a journey. More compartments allow more things to fit, but each adds weight. A selection criterion balances capacity against a specified cost. Changing the cost rule can change the preferred bag; it does not establish one universally best choice.

Worked example

For an original illustration, suppose two compatible models use the same 50 observations and the same likelihood convention. Model A has maximized log likelihood -80 and three estimated parameters. Model B has log likelihood -77.5 and five parameters. These counts include every parameter required by the stated convention.
A's AIC is 160 + 6 = 166. B's is 155 + 10 = 165, so AIC prefers B. The AICc corrections are 24/46 = 0.521739 for A and 60/44 = 1.363636 for B. Thus AICc values are 166.521739 and 166.363636: AICc also prefers B, but only narrowly.
For BIC, log(50) is approximately 3.912023. A scores 171.736069 and B scores 174.560115, so BIC prefers A. The likelihood gain is enough for AIC's penalty but not BIC's stronger penalty here.
These are stipulated fit summaries, not results from an invented dataset. The small AICc difference does not establish a practically important forecast advantage. Independently recomputing each penalty confirms the rankings.

Illustrative case study

Fictional case: Willow Dispatch chooses a weekly workload model. Its automated search finds a higher-order candidate with a slightly lower AICc, while a simpler candidate has lower BIC. The analysts record both scores and compare forecasts at the four-week horizon used for hiring decisions.
They also inspect residual dependence and stability across several historical cutoffs. Management chooses a model only after reviewing that evidence and maintenance requirements. This invented case shows how criteria inform a decision; it does not report an accuracy improvement or imply that one criterion always wins.

Common misconceptions and mistakes

Do not compare scores solely because they share the same label. Different observations, transformations without appropriate likelihood treatment, or incompatible likelihood constants can invalidate the comparison. Check software conventions before combining output from different packages.
Do not choose the model with the best in-sample fit and call that penalized selection. Do not treat a minimum information criterion as a residual diagnostic or a significance test. Searching a limited neighborhood can also miss a lower-scoring candidate elsewhere.

Business use

Order selection helps teams automate routine forecasting while making complexity choices reproducible. Save the candidate set, fitted sample, transformations, parameter counts, estimation settings, and criterion values with the selected model.
Set reasonable operational limits and review candidates that are unstable, difficult to estimate, or inconsistent with the process being forecast. Use evaluation periods that reflect deployment and avoid repeatedly choosing models on the final test set. A transparent score is useful evidence, but business costs may require additional comparison.

Check your understanding

Which criterion prefers A in the example?

Answer: BIC.

Why does AIC prefer B?

Answer: The improvement in minus twice log likelihood is five, exceeding the extra penalty of four.

Why are parameter conventions important?

Answer: Omitting fitted quantities changes the penalty and potentially the ranking.

Does the narrow AICc difference guarantee better staffing decisions?

Answer: No; operational forecast evaluation is still needed.

Related terms

Likelihood
Parsimony
Overfitting
model order
AIC
AICc
BIC
ARIMA
rolling-origin evaluation
residual diagnostics

Learn next

compare a small fixed family of time-series models on one training period. Recalculate their parameter penalties, inspect residuals, and check whether rankings persist across later forecast origins and business-relevant horizons.

Time series

Sources

Sources: Hyndman and Athanasopoulos, Forecasting: Principles and Practice, ARIMA information criteria: https://otexts.com/fpp3/arima.html . Hyndman, Glossary of Forecasting Terms: https://robjhyndman.com/mwh3/FG4.pdf . Relevant primary textbook and glossary passages consulted. Numerical summaries and fictional case are original.

Previous: Order of integration Next: Order statistic