Are all model parameters learned?

No. Some are fixed by design or prior knowledge, and only a subset may be trained during adaptation.

Is learning rate a model parameter?

It is usually a hyperparameter controlling optimization, rather than a coefficient in the predictive mapping.

Can different parameter values give the same predictions?

Yes. Rescaling inputs with compensating coefficients or symmetries within a model can produce equivalent behavior.

AI Glossary / Statistics and probability / Foundation

Model parameter

Definition

A model parameter is a quantity that helps specify a particular model within a family of possible models. In machine learning, parameters are commonly estimated from training data: examples include regression coefficients, neural-network weights and biases, and the probabilities in a fitted categorical distribution. Changing parameter values usually changes the model's predictions or the distribution it represents. Parameters differ from input features, which describe individual cases, and from hyperparameters, which typically control model design or the learning procedure. Some parameters may be fixed rather than learned.

What it means

A model family defines a form, such as a straight line. Its parameters choose one member of that family, such as a particular slope and intercept. Learning estimates suitable values according to data, assumptions and an objective. The resulting values summarize aspects of the training process, but they are not necessarily understandable facts about the world. Several parameter settings can sometimes produce identical predictions. In statistical analysis, parameters may describe an underlying population distribution, while an estimated value is the result of fitting that distribution to a sample.

How it works

Specify the model structure and its allowed parameter values. Choose an estimation or optimization method, such as least squares, maximum likelihood or gradient-based training. The procedure compares model behavior with available observations or another defined objective and updates or solves for parameter values. Constraints and regularization can restrict the solution. Save the resulting parameters together with the architecture, feature definitions and preprocessing needed to interpret them. Evaluate the fitted model on appropriate unseen data. For Bayesian models, inference can represent uncertainty through a distribution over parameters rather than only one fitted value.

Three practical examples

Forecasting: a linear model learns coefficients linking specified predictors to an expected quantity.
Image recognition: a neural network learns arrays of weights used to transform pixels into class scores.
Reliability analysis: a probabilistic model estimates a distribution's rate or shape from observed data. A useful explanation states the parameter's units and assumptions instead of presenting its numerical value without context.

Think of it this way

Think of a model family as a radio with adjustable controls. The design determines what the controls can do; their settings determine the particular behavior. Training chooses settings using evidence and an objective. The analogy has limits: real models can contain interacting parameters whose individual effects cannot be understood by turning one control in isolation.

Worked example

Consider the model y-hat = b + wx and the three training observations (0,1), (1,3) and (2,5). The parameter values b = 1 and w = 2 fit all three points exactly: predictions are 1, 3 and 5. For x = 3, the model predicts 7, although this extrapolation is not guaranteed by the training fit. There are two parameters, b and w, and one input feature, x. If x represents meters and is converted to centimeters, the equivalent slope is 0.02 per centimeter, with the same intercept. At 300 centimeters the prediction remains 1 + 0.02 times 300 = 7. The changed numerical coefficient does not imply changed behavior. Separately, a fully connected layer with three inputs, two outputs and one bias per output contains 3 times 2 + 2 = 8 parameters. These original calculations distinguish parameter values, units and parameter count.

Illustrative case study

In a fictional case, Willow Operations compares two versions of a delivery-time model. A manager notices that one coefficient has become much smaller and initially assumes the associated factor matters less. The analyst finds that the input unit changed from kilometers to meters, explaining the rescaling. The team compares predictions under consistent inputs and documents both units and preprocessing alongside the saved parameters. It then evaluates the models on held-out deliveries before deciding whether either version improves planning.

Common misconceptions and mistakes

More parameters do not automatically mean better predictions, stronger reasoning or greater business value. Parameter count is not the same as training-data size or the number of input features. A large coefficient is not directly comparable with another coefficient measured on a different scale. A learned association is not automatically a causal effect. Parameters can remain fixed during ordinary inference, but online-learning systems may update them through an explicit learning process. Values stored in a model artifact can also include nonparameter state, so file size is not a precise parameter count.

Business use

Businesses should record parameterized model versions with their architecture, training lineage and feature-processing requirements. Parameter counts can inform memory and compute estimates, but actual resource use also depends on numerical precision, execution strategy and temporary state. For interpretable statistical models, report uncertainty and units where appropriate. For complex networks, evaluate behavior rather than treating individual weights as explanations. Monitor performance after deployment and retrain through controlled processes when evidence supports it. The practical question is whether the fitted system behaves reliably for its intended decisions.

Check your understanding

A straight-line model has one input, a slope and an intercept. How many fitted parameters does it have?

Answer: Two, assuming both the slope and intercept are estimated. The input value is not an additional fitted parameter.

Related terms

Weight
Bias
Parameter space
Regularization
Parameter estimation
hyperparameter
identifiability
statistical inference
model complexity

Learn next

Learn how a loss function guides parameter estimation, then examine overfitting and regularization. Practice tracking units through a simple regression and comparing predictions before interpreting changes in fitted coefficients.

Sources

The Aalto Dictionary of Machine Learning, entries on model parameters and parameter space: https://aaltodictionaryofml.github.io/ADictML_Main.pdf Google Machine Learning Glossary, parameter: https://developers.google.com/machine-learning/glossary/#parameter

Previous: Model parallelism Next: Model poisoning