Is logistic regression parametric?

With a fixed feature representation, yes: it uses a fixed finite coefficient vector.

Can a parametric model be Bayesian?

Yes. A prior and posterior can describe uncertainty over its finite parameter vector.

Does selecting a polynomial degree make every polynomial procedure nonparametric?

Not automatically. A fixed-degree family is parametric; a procedure allowing complexity to grow with the dataset requires examining the broader learning setup.

AI Glossary / Graphs and symbolic AI / Intermediate

Parametric model

Definition

A parametric model is a model family described by a fixed finite set of parameters, for a specified model structure. Learning estimates those parameters from data. Adding more observations does not by itself require increasing the number of parameters in that fixed family.

What it means

For example, a straight-line predictor has an intercept and a slope. Two observations and two million observations can both be used to estimate those same two quantities. A Gaussian probability distribution has a mean and variance; collecting more measurements refines their estimates without adding a separate parameter for every measurement.

“Parametric” does not mean “simple,” “linear” or “easy to explain.” A neural network with a fixed architecture can have many parameters and nonlinear behavior while remaining parametric in this sense.

The relevant contrast is with model families whose flexibility is not restricted to a fixed finite-dimensional parameterization. Nonparametric methods can still contain numerical settings and fitted quantities; their name does not mean that they have no parameters of any kind.

How it works

The modeler first specifies a functional or distributional family and its parameter space. Training then uses observations to estimate a parameter vector through a chosen loss, likelihood or Bayesian update.

The family imposes assumptions. A straight line cannot represent every curved relationship exactly, regardless of how accurately its two coefficients are estimated. More data may reveal that limitation rather than remove it.

Changing the feature set, polynomial degree or network architecture can change the number of parameters. That is a change to the selected family or structure, not merely refitting the same fixed model with additional observations.

A Bayesian version can represent uncertainty over the same finite parameter vector. Being Bayesian therefore does not automatically make a model nonparametric. Conversely, storing a finite representation of a nonparametric fit on a computer does not erase the broader family's ability to grow with data.

Three practical examples

A retailer fits a straight-line relation between a chosen input and demand, estimating two coefficients.
A quality team models measurement noise with a Gaussian distribution and learns its mean and variance.
A developer trains a fixed neural-network architecture on an expanded dataset while keeping the architecture's parameter count unchanged.

Think of it this way

Think of a camera with a fixed set of adjustable controls. More photographs can help choose better settings, but they do not automatically add another control to the camera.

The analogy is limited because mathematical parameters may interact and need not correspond to recognizable physical controls. A fixed set can also be extremely large. The point is the distinction between estimating existing quantities and changing the set of quantities that defines the model.

Worked example

Suppose an invented deterministic regression model predicts y = a + bx. Its two free parameters are a and b. Three training pairs are (0,2), (1,5) and (2,8).

The choice a=2 and b=3 fits all three pairs exactly: predictions are 2, 5 and 8. A fourth observation at x=3 with y=11 would agree with the same parameter values. The number of parameters remains two.

Now suppose the fourth outcome is instead 14. A straight line cannot pass through all four points. Least squares refits the coefficients to a=1.4 and b=3.9, producing predictions 1.4, 5.3, 9.2 and 13.1. The residuals are 0.6, −0.3, −1.2 and 0.9, and their squared sum is 2.70.

There are still two coefficients. Adding a quadratic term would create a different, three-coefficient family; it is not an automatic consequence of adding the observation. These invented data do not establish which family is best for future cases.

Illustrative case study

The fictional Cedar Demand team begins with a fixed linear model because its relationship is easy to inspect. As more data arrive, prediction errors show a systematic curve rather than random scatter.

Cedar checks data quality and compares the linear model with alternative families using an appropriate validation design. It does not assume that refitting the same line indefinitely will remove the mismatch. It also avoids assuming that the most flexible alternative will generalize best.

The fictional case illustrates a distinction between uncertainty in estimated coefficients and limitations of the selected family. More evidence can reduce the first while exposing the second.

Common misconceptions and mistakes

Parametric means linear. Many nonlinear distributions and predictors have fixed finite parameterizations.
Parametric means only a few parameters. A fixed neural network can have a very large parameter vector.
More data must mean more parameters. The count stays fixed within the specified family, although estimated values can change.
Nonparametric means assumption-free. Such methods still depend on choices and assumptions about smoothness, distance, sampling or other structure.
A parametric model cannot overfit. Flexible finite models can fit noise and generalize poorly.

Business use

The distinction helps organizations reason about model flexibility, storage, updating and the assumptions behind predictions. A compact parametric model can be convenient, but the label alone does not guarantee speed, interpretability or accuracy.

Ask whether observed errors arise from uncertain parameter estimates, inappropriate inputs, optimization problems or a family unable to capture the relationship. These possibilities call for different responses.

Document the feature definition and structure alongside fitted parameters. A statement that “the same model was retrained” is incomplete if the feature expansion or architecture changed. Evaluation should assess the complete modeling choice, including any process used to select the family.

Check your understanding

The worked example increases from three observations to four. Why does the parameter count remain two?

Answer: The selected family remains y=a+bx. Additional observations change the evidence used to estimate a and b, not the number of coefficients in that family.

Related terms

Parameter learning
Nonparametric model
Linear regression
Neural network
model family
model misspecification
bias–variance tradeoff
Bayesian estimation

Learn next

Fit the same two-coefficient line to two datasets of different sizes. Record parameter values and residuals separately from the parameter count. Then compare a fixed quadratic family and explain which modeling assumption changed when the extra coefficient was introduced.

Sources

Stanford statistics course materials distinguish parametric and nonparametric approaches: https://web.stanford.edu/~lmackey/stats202/content/lec2.pdf

Carnegie Mellon lecture search excerpts corroborate fixed parameter sets; the full file could not be opened during this review: https://www.andrew.cmu.edu/course/18-661/lectures/NN.pdf

The line-fitting calculation and Cedar Demand case are original examples. The definition concerns statistical and machine-learning models, rather than every use of “parametric” in design software.

Previous: Parametric index Next: Parametric test