AI Glossary / Statistics and probability / Intermediate

Regression model

Definition

A regression model represents a relationship between predictors and an outcome, commonly for predicting a numerical quantity or describing a conditional outcome feature. A statistical regression model includes both the relationship and assumptions about variation around it, while machine-learning usage sometimes refers more narrowly to the fitted prediction function. The model may be linear, nonlinear, tree-based or otherwise flexible. Its form, target and uncertainty representation should be stated rather than inferred from the word regression alone.

What it means

A model is more than one predicted number. It specifies how predictions change with inputs and, in a probabilistic formulation, how outcomes vary around those predictions. Different regression models can target the conditional mean, median, quantiles or an entire distribution. Terminology has historical exceptions: logistic regression is commonly used for classification probabilities. A numerical category code, such as a postal code, does not become a meaningful continuous regression target merely because it is stored as a number.

How it works

Define the outcome, available predictors and intended prediction or inference target. Select a functional form and fitting objective suited to that target. Estimate the model from development data and assess assumptions, residual patterns and out-of-sample performance as appropriate. Document preprocessing, missing-data handling and the range of supported inputs. If the model represents predictive uncertainty, evaluate that uncertainty as well as point accuracy. Monitor changes after deployment because a previously useful relationship may no longer describe the target setting.

Three practical examples

Example 1: A delivery model predicts a duration from route and workload features. Example 2: A manufacturing model describes a continuous quality measurement under different operating conditions. Example 3: A quantile regression model estimates a high conditional demand level rather than the average. The same predictors and data can support different targets, but the fitting loss and evaluation should match the chosen question. A low average error does not automatically imply reliable upper-tail predictions.

Think of it this way

Think of a regression model as a structured rule for translating a situation into a prediction, together with whatever uncertainty assumptions the model contains. The fitted coefficients or tree are the visible machinery. The target definition and validation explain what the machinery is intended to do. Without those, two systems both called regression models may produce numbers with very different meanings.

Worked example

Suppose a stated probabilistic model is Y given x normally distributed with mean 5 + 2x and standard deviation 3. At x = 3, the conditional mean is 11. The probability of Y exceeding 14 is the probability that a standard normal exceeds (14 - 11)/3 = 1, approximately 0.1587. An approximate central 95% interval for a new outcome is 11 ± 1.96 × 3, or [5.12, 16.88], when these model parameters are treated as known. The equation y-hat = 5 + 2x alone would provide the mean prediction but not this interval. If parameters were estimated from limited data, their uncertainty could require additional treatment. The normal outcome assumption also needs checking rather than being accepted because the arithmetic is convenient.

Illustrative case study

Fictional case study: Cedar Dispatch asks for a model that supports delivery commitments. An initial mean-regression model predicts typical duration well but does not answer how often a promised deadline will be missed. Analysts clarify the target and evaluate a probabilistic or quantile model against that question. The case is fictional and does not claim a measured performance improvement. It illustrates why choosing the model's target is as important as selecting its algorithm.

Common misconceptions and mistakes

Common mistakes include equating regression with ordinary least squares, confusing a prediction function with a full probabilistic model, and assuming every model output is an expected value. A flexible model can overfit, while a simple model can miss important patterns. Accurate prediction does not by itself establish causation or fairness. Training error can be optimistic, especially after extensive model selection. Models can also produce impossible values if their output range is incompatible with the target.

Business use

Business use: Regression models support planning and estimation when a clearly defined numerical outcome matters. Choose evaluation measures that reflect the actual decision and compare with transparent baselines. Document how predictions are produced and when they should not be used. Communicate uncertainty without implying that the model includes every possible source of error. For proposed interventions, use a causal design or justified identification strategy rather than treating predictive relationships as intervention effects.

Check your understanding

What does the worked model provide beyond its mean equation?

Answer: A conditional normal distribution with stated standard deviation, allowing probability and interval calculations.

Why does the interval calculation say parameters are treated as known?

Answer: Estimating parameters introduces another source of uncertainty that the simplified calculation does not include.

Related terms

Regression analysis is the broader process of specifying
fitting and evaluating models

Learn next

Compare a mean prediction, a median prediction and an upper-quantile prediction for the same skewed outcome. Evaluate each with a loss appropriate to its target. Then add a probabilistic distribution and inspect calibration of intervals. Keep model specification, fitted parameters and observed validation performance distinct in your explanation.

Sources

Sources: Google Developers, Machine Learning Glossary, regression model and probabilistic regression model: https://developers.google.com/machine-learning/glossary ; Penn State STAT 501, Regression Methods: https://online.stat.psu.edu/stat501/index ; TensorFlow Probability, Probabilistic Layers: Regression: https://www.tensorflow.org/probability/examples/Probabilistic_Layers_Regression . These primary educational and software sources ground the model distinctions. The conditional-normal calculation and fictional dispatch case are original illustrations.

Previous: Regression line Next: Regression planning