Must a statistical model be linear?

No. Nonlinear and hierarchical models are common.

Does a fitted model prove the data-generating process?

No. It provides an approximation whose adequacy must be assessed.

Are machine-learning models and statistical models completely separate categories?

No. The terminology overlaps; many learning methods have explicit statistical interpretations, while some predictive algorithms specify less of a probability model.

AI Glossary / Statistics and probability / Beginner

Statistical model

Definition

A statistical model is a mathematical description of how observed data can arise, expressed through probability distributions and assumptions about relationships, variation, and dependence. It may contain unknown parameters estimated from data and may be used for description, prediction, or inference. A model is a structured approximation to the process being studied, not a claim that every detail of reality has been captured. Its usefulness depends on the question, the data, and whether its assumptions are adequate for the intended use.

What it means

A model connects observations to quantities that can be reasoned about. For example, a binomial model describes a count of successes under assumptions about a fixed number of independent trials with a common success probability. A regression model describes how an outcome's distribution or selected features relate to predictors.

The fitted model is more than a formula displaying estimated coefficients. Its assumptions also include what was measured, which observations are treated as independent, how missingness or selection is handled, and which population or conditions the model represents. These assumptions can matter as much as the numerical estimates.

How it works

Define the outcome, observational units, predictors, and purpose. Choose a family of probability descriptions appropriate to those variables and the design. Specify the parameters, dependence structure, and any restrictions on how outcomes relate to predictors.

Estimate unknown quantities using a suitable method, such as maximum likelihood, least squares under an associated model, or Bayesian inference. Assess identifiability: different parameter values should not be observationally indistinguishable for the quantities being claimed. Examine fit, residual patterns, uncertainty, and predictive performance using data and checks appropriate to the task.

Distinguish model specification from estimation and validation. A successful fitting algorithm only finds a result within the chosen setup; it does not establish that the setup is suitable. Statistical models can include nonlinear relationships, interactions, and hierarchical structure. Although some introductory or author-specific descriptions emphasize additive effects, additivity is not a universal requirement of every statistical model.

Three practical examples

1. A manufacturer models the number of defective items in a fixed inspection sample, checking whether a common independent defect probability is plausible.

2. A service organization models resolution time conditional on ticket characteristics and evaluates the resulting prediction intervals.

3. A research team uses a hierarchical model to represent observations grouped within sites, rather than treating every record as independent.

Think of it this way

Think of a map drawn for a particular journey. It omits details while preserving relationships needed for navigation. A road map can be useful without depicting every tree, but it becomes misleading if used for a task requiring information it leaves out. Models likewise need a defined purpose and an understood scope.

Worked example

Consider an original model for four independent binary outcomes with a shared success probability p. The observed sequence is success, success, failure, success. Under the model, its probability is p cubed times (1 - p).

At p = 0.5, the sequence probability is 0.5 to the fourth power = 0.0625. At p = 0.75, it is 0.75 cubed times 0.25 = 0.10546875. The latter likelihood is 1.6875 times the former for this observed sequence. Maximizing the Bernoulli likelihood gives the sample proportion, p_hat = 3/4 = 0.75.

If only the count of three successes is recorded, the binomial probability includes a factor C(4,3) = 4 for the possible sequences. Both candidate likelihoods receive that same factor, so their ratio and maximizing p remain unchanged.

Using the fitted p as a plug-in value, the predicted probability of two future independent successes is 0.75 squared = 0.5625. This prediction ignores uncertainty in p and assumes the same process continues. The four observations do not prove independence, a common probability, or future stability; those are model assumptions requiring additional justification.

Illustrative case study

In a fictional maintenance company, analysts fit a simple model for fault counts across service visits. The average prediction looks reasonable, but residual checks reveal that visits to the same machine have related outcomes and that some machine types are much more variable.

The team revises the model to represent those features and evaluates it on later visits. It reports which operating conditions are covered and monitors changes after deployment. The revised analysis treats the original model as a useful starting approximation rather than defending it solely because its software fit completed successfully.

Common misconceptions and mistakes

Do not confuse a statistical model with the observed dataset or with a fitting algorithm. The same data can support several models, and the same model can be estimated in different ways. A high fit score does not establish correct assumptions, causal interpretation, or good future predictions.

More complex models are not automatically better; they can overfit or make important quantities difficult to identify. Simple models are not automatically trustworthy either. An association model does not become causal just because it includes many predictors. Uncertainty conditional on a fitted model may also omit uncertainty about model choice, data quality, and future changes.

Business use

Statistical models help businesses organize evidence, quantify uncertainty, and compare scenarios. They are useful when their outputs are linked to clear decisions and tested against realistic conditions. Document assumptions, target populations, training and evaluation data, and known failure modes. Monitor performance after changes in operations or customer behavior, and distinguish predictions within the observed range from extrapolation beyond it.

Check your understanding

A model assumes independent observations, but all records come from repeated measurements of one device. Does a successful numerical fit validate independence?

Answer: No. Dependence must be considered from the design and evidence. A fitting routine can return estimates even when a crucial assumption is unsuitable.

Related terms

Probability model
Parameter
Likelihood
Model validation
Overfitting
estimator
regression
identifiability
predictive distribution

Learn next

Learn how a probability model assigns probabilities to observations, then study estimation and model checking as separate tasks. Compare uncertainty within one model with uncertainty about the model itself.

Sources

Frank Harrell, Glossary of Statistical Terms, statistical model: https://hbiostat.org/glossary/

NIST/SEMATECH, Typical underlying assumptions in process modeling: https://www.itl.nist.gov/div898/handbook/pmd/section2/pmd21.htm

The primary terminology and assumption guidance were consulted. This entry uses the broad probability-model meaning and notes that additivity is not universal. The Bernoulli calculations and fictional case are original.

Previous: Statistical machine translation Next: Statistical model specification