Is logistic regression a linear model?

In common machine-learning terminology, yes: its log-odds score is linear in features, although predicted probability is transformed nonlinearly.

Can a linear model use categories?

Yes, through appropriate indicator or contrast coding.

Must errors be normally distributed?

Normality is not required simply to compute least-squares coefficients. Particular exact inference procedures require additional assumptions, which should be stated separately.

AI Glossary / Statistics and probability / Beginner

Linear model

Definition

A linear model combines specified features through a weighted sum, usually with an intercept. Its basic score is b + w1 x1 + ... + wp xp, where the weights and intercept are model parameters. In machine learning, the term also includes models that transform this score, such as logistic regression. In classical statistical usage, a linear model commonly describes an outcome whose conditional mean is linear in coefficients. State which meaning and output scale are intended.

What it means

Each feature contributes its value multiplied by a weight. The intercept supplies the starting value when all features equal zero. In a model without interactions, changing one feature while holding the others fixed changes the raw score at a constant rate.

The word linear refers to the model's mathematical construction, not to whether the real world is simple. Features can include category indicators, squared measurements, or interactions. A model can therefore be linear in its fitted coefficients while tracing a curved relationship with an original measurement.

How it works

Prepare features consistently, choose an output and loss function, and estimate the parameters from training data. Ordinary least squares minimizes squared errors for a continuous outcome. Other linear-score models use different losses and output transformations.

For one observation with feature vector x, calculate the score from the intercept and weighted features. For many observations, write the scores as X beta, with an intercept column included if needed. Regularization can constrain coefficients to reduce instability or overfitting.

Evaluate performance on suitable held-out data and inspect where predictions fail. Statistical uncertainty requires assumptions about the data and errors; merely using a linear expression does not supply them. Nor does the model automatically establish that changing a feature would cause the predicted change.

Three practical examples

A warehouse predicts packing time from item count and a fragile-item indicator, using the resulting minutes to plan staffing.
A document classifier uses a weighted sum of word-based features as a score, then applies a specified classification rule or probability transformation.
A calibration team fits an output against a measurement and its square. The fitted curve can bend even though the unknown coefficients enter linearly.

Think of it this way

Think of a recipe with fixed contributions for each measured ingredient. The model adds those contributions to a starting amount. If a feature represents a special combination of ingredients, that combination can have its own weight. The recipe analogy explains the calculation, but the weights still need evidence and validation before the recipe is trusted.

Worked example

Suppose a fictional packing-time model is minutes = 8 + 2.5 x item count + 6 x fragile, where fragile equals one for a fragile order and zero otherwise.

For four items and a fragile order, the prediction is 8 + 2.5 x 4 + 6 x 1 = 24 minutes. For five items with the same fragile status, it is 26.5 minutes. The one-item change contributes 2.5 minutes, holding the indicator fixed. A nonfragile four-item order has a prediction of 18 minutes.

Now consider a separate curve, predicted response = 3 + 2x + 0.5x squared. At x = 0, 1, and 2, its values are 3, 5.5, and 9. The successive changes are 2.5 and 3.5, so the curve is not a straight line in x. Nevertheless, it is linear in the three coefficients multiplying features 1, x, and x squared.

These are specified teaching models, not coefficients fitted to a dataset. Prediction arithmetic does not measure their accuracy or justify extrapolation.

Illustrative case study

Fictional case: Parcelway builds a complex AI system for packing-time forecasts. Before adopting it, the operations team compares it with a linear baseline using item count, package size, and handling requirements.

The baseline is easy to inspect and performs adequately for ordinary orders, but errors increase for unusually large packages. Analysts inspect residuals and consider a size transformation or interaction instead of assuming the simple model is universally suitable.

The company compares revised candidates on future-like orders and retains explanations of their operating ranges. Managers gain a practical benchmark for deciding whether a more complex system delivers enough improvement to justify maintenance effort.

Common misconceptions and mistakes

Do not confuse linearity in coefficients with a straight-line relationship in every original variable. Do not assume the intercept is meaningful if zero-valued features lie far outside the observed data.

A large coefficient is not automatically the most important feature because measurement units and correlations matter. Strongly related features can make individual coefficients unstable even when predictions remain useful. A simple formula also does not guarantee fairness, robustness, or causal interpretation. Feature selection, missing data, and deployment changes still affect results.

Business use

Linear models provide useful baselines for forecasting, scoring, and operational estimation. They are often inexpensive to calculate and straightforward to inspect, which can help teams investigate unexpected outputs.

Their value should be judged against the business target and plausible alternatives. Document feature units, category coding, transformations, and the output scale. Monitor errors after deployment: a stable calculation can become inaccurate when customer behavior, pricing, equipment, or data collection changes. Simplicity helps diagnosis but does not replace ongoing evaluation.

Check your understanding

Is 3 + 2x + 0.5x squared a linear model?

Answer: It is linear in the coefficients associated with the specified features 1, x, and x squared. It is not a straight-line function of x. Naming the features and the sense of linearity resolves the apparent conflict.

Related terms

Learn next

Next, study how least squares estimates weights and how residuals reveal model shortcomings. Then compare regularized linear models with nonlinear alternatives using the same validation data and decision-relevant metric.

Least squares

Sources

Sources: Google Machine Learning Glossary, Linear Model, including its distinction between raw score and transformed output: https://developers.google.com/machine-learning/glossary#linear_model . NIST/SEMATECH, Linear Least Squares Regression, for linearity in parameters: https://www.itl.nist.gov/div898/handbook/pmd/section1/pmd141.htm . Both primary explanations were consulted. The packing model, polynomial calculation and fictional case are original teaching examples; numerical predictions were independently checked.

Previous: Linear map Next: Linear network coding