Does parsimony mean always choosing a linear model?

No; a compact nonlinear model can be justified by the process and evidence.

Is fewer parameters always better?

No; insufficient structure can cause systematic errors.

Can information criteria help?

Yes. Their penalties formalize one tradeoff, although criterion choice and valid comparisons still matter.

Is interpretability identical to parsimony?

No. A small model can contain difficult transformations, and a larger structured model may be easier to explain in a particular domain.

AI Glossary / Time series and signal processing / Intermediate

Parsimony

Definition

Parsimony is the principle of using no more model complexity than is needed for the task and the evidence. In forecasting, it often means preferring a model with fewer estimated parameters when it adequately represents the useful structure. It is a disciplined preference for economical explanation, not a rule that the smallest model always performs best.

What it means

A model can reproduce historical data more closely by adding terms that capture accidental variation. Those terms may be unstable when new observations arrive. Parsimony encourages analysts to ask what each additional component contributes and whether that contribution survives a fair evaluation.
Adequacy matters as much as simplicity. A model that ignores a strong seasonal pattern can be too simple, even if it is easy to describe. The intended forecast horizon, available data, operating constraints, and cost of error determine what complexity is justified.

How it works

Start with a credible baseline and add structure for a reason: persistent trend, seasonal dependence, relevant predictors, or another reproducible pattern. Assess the resulting forecasts and residual behavior rather than relying solely on historical fit.
Complexity can be controlled through a limited model family, parameter penalties, regularization, or explicit restrictions on unnecessary terms. Parameter count is one useful measure, but effective flexibility, tuning effort, and data requirements also matter. A large regularized model can behave more conservatively than its raw parameter count suggests.
Parsimony is especially useful when data are limited because many fitted quantities compete for the same information. However, removing an essential predictor or dependency can increase bias. Selection should therefore balance fit, generalization, stability, and the practical demands of deployment.

Three practical examples

A weekly demand team retains a seasonal model but removes extra lag terms that add unstable estimates without improving later forecasts.
A maintenance analyst compares a short predictor list with a larger one using evaluation periods that respect time order.
A small organization begins with a transparent baseline and requires evidence that a more elaborate forecasting pipeline improves decisions enough to justify its upkeep.

Think of it this way

Think of packing the tools required for a repair. Carrying every tool makes the bag heavier, but removing the one necessary tool prevents the repair. Parsimony means choosing sufficient equipment with a clear purpose for each item, rather than minimizing weight regardless of the job.

Worked example

Consider an original illustration with three training pairs: x = -1, 0, 1 and y = 1, 0, 1. A constant least-squares model predicts the mean, 2/3. Its training squared error is (1/3)^2 + (2/3)^2 + (1/3)^2 = 2/3.
A quadratic model y = x squared passes through all three points, giving zero training error. Its three freely fitted coefficients allow the intercept, linear, and quadratic terms, even though two happen to be zero in this example.
Now stipulate two later evaluation cases: x = -2 and 2, each with observed y = 2/3. The constant model has zero error. The quadratic predicts four for each, producing total squared error 2 times (4 - 2/3)^2 = 200/9, approximately 22.2222.
This constructed example proves neither that the constant model is generally superior nor that quadratic models are unsuitable. If later values followed the quadratic, the ranking would reverse. It demonstrates why perfect training fit alone cannot justify complexity, especially when extrapolating beyond observed predictor values. The coefficients and both error totals were independently checked.

Illustrative case study

Fictional case: Cedar Repairs forecasts weekly service requests using a model with a calendar effect and a small number of lags. A proposed replacement adds many interactions and fits the training history more closely. Across several later evaluation windows, its gains are inconsistent and its estimates change sharply.
The team keeps the smaller adequate model while gathering more evidence. It also retains the seasonal component because removing it clearly damages forecast behavior. This invented case illustrates justified simplification, not a claim of measured savings or an argument against complex models in general.

Common misconceptions and mistakes

Do not interpret parsimony as excluding every variable whose individual coefficient is uncertain. Correlated predictors, grouped effects, and hierarchical constraints can require joint consideration.
Do not treat a short equation as automatically robust. A simple model selected after extensive unreported experimentation can still be overfit to the evaluation process. Conversely, raw parameter count does not fully describe a constrained or regularized system's effective flexibility.
Do not simplify away structure that matters for the forecast target merely to make the presentation easier.

Business use

A parsimonious model can reduce data dependencies, monitoring burden, and difficulty explaining a forecast. Those advantages are useful when forecast quality and operational suitability are comparable, but they do not replace evidence about accuracy or risk.
Document why each important component is present, compare against appropriate baselines, and check performance at the horizons used by the business. Revisit the decision when the process changes. A model that was adequate last year can become too simple after a new seasonal pattern or operating regime appears.

Check your understanding

Why does the quadratic win on training data?

Answer: It interpolates all three observed points.

Why does it lose on the stipulated later cases?

Answer: Its extrapolated predictions do not match those later outcomes.

Does this establish the constant model as the true relationship?

Answer: No; the example only illustrates a possible generalization failure.

What should be retained in a parsimonious forecast?

Answer: Structure supported by evidence and needed for the task, rather than complexity added without adequate justification.

Related terms

Model selection
Overfitting
Underfitting
Regularization
bias-variance tradeoff
AIC
BIC
baseline forecast
Occam's razor

Learn next

remove one component from an existing model and assess the consequences on later observations and residual diagnostics. Record both forecast changes and operational dependencies before deciding whether the simplification is worthwhile.

Sources

Sources: Hyndman, Glossary of Forecasting Terms, parsimony: https://robjhyndman.com/mwh3/FG4.pdf . Hyndman and Athanasopoulos, Forecasting: Principles and Practice, ARIMA order selection and information criteria: https://otexts.com/fpp3/arima.html . Relevant primary passages consulted. The numerical demonstration and fictional case are original.

Previous: PARSEVAL Next: Parsing