Does multiple regression require only numerical predictors?

No. Categorical predictors can be represented using an appropriate coding scheme.

Can it include curved relationships?

Yes. Transformed terms such as a squared predictor can represent curvature while remaining linear in coefficients.

Does adjusting for many variables establish causation?

No. Causal interpretation requires a justified design and assumptions about confounding, measurement and which variables should be included.

AI Glossary / Statistics and probability / Intermediate

Multiple regression

Definition

Multiple regression models the relationship between one outcome and two or more predictors. In the common multiple linear regression form, the expected outcome is represented by an intercept plus a weighted combination of predictor terms. The model can support prediction or describe conditional associations, depending on the study and analysis. The word multiple refers to the predictors, not to several outcomes. Regression coefficients do not automatically represent causal effects, and including more variables does not guarantee that confounding or model misspecification has been resolved.

What it means

An outcome can vary with several measured factors at the same time. Multiple regression estimates their associations within one specified model. In an additive linear model, a coefficient describes the predicted change in the outcome for a one-unit change in its predictor while the other included predictors remain fixed. This is a conditional comparison, which may differ from the simple association seen when the predictor is examined alone. The interpretation depends on the included variables, their coding and the functional form. Holding variables fixed algebraically is not the same as conducting a controlled experiment.

How it works

Define the outcome and predictors using information appropriate to the prediction or explanatory question. Specify the model, including any justified transformations or interactions. For ordinary least squares, estimate coefficients by minimizing the sum of squared residuals. Inspect residual patterns, influential observations and relationships among predictors. Use uncertainty estimates suited to the design, addressing dependence or unequal variance where necessary. Validate predictions on appropriate unseen cases when prediction is the goal. Interpret each coefficient in the context of units and model terms, and distinguish a useful fitted relationship from an unsupported causal story.

Three practical examples

Operational forecasting: predict completion time from workload, staffing and other available inputs.
Resource planning: estimate expected consumption from measured job characteristics.
Observational analysis: examine how an outcome is associated with one factor after accounting for specified additional variables. These uses require different validation and interpretation choices, even when the fitted regression equation has the same mathematical form.

Think of it this way

Think of a mixing board with several controls contributing to one output. The equation describes how changing a control affects the predicted output while the other controls remain fixed. Real observational data are more complicated: controls may move together, some may be missing and the board's behavior may not match the assumed equation. The analogy helps explain the coefficients without proving that they are causal control knobs.

Worked example

Consider the artificial fitted equation y-hat = 5 + 2X1 - 3X2. At X1 = 4 and X2 = 1, the prediction is 5 + 8 - 3 = 10. Increasing X1 to 5 while keeping X2 at 1 gives 12, a two-unit increase. Instead increasing X2 to 2 while keeping X1 at 4 gives 7, a three-unit decrease. If both predictors rise by one unit, the predicted change is 2 - 3 = -1, producing 9 from the original value 10. This original calculation illustrates an additive model. It does not imply that independently changing either predictor is feasible or causal. If the model included an interaction X1 times X2, the slope of one predictor would depend on the other, so the simple constant-coefficient interpretation would need revision.

Illustrative case study

In a fictional case, Willow Logistics models delivery duration using distance, load and a route-category indicator. The team documents the reference category and measurement units before interpreting coefficients. Residual review reveals unusually large errors on a small set of routes, prompting investigation of omitted congestion information. Managers use the model for planning only after evaluating a later time period. They avoid interpreting the distance coefficient as a universal effect that would remain unchanged under every route or scheduling policy.

Common misconceptions and mistakes

Multiple regression is not the same as multioutput regression. Linear means linear in the fitted coefficients, so polynomial terms can still appear in a linear regression. Predictor variables do not need to be mutually independent or normally distributed. Strong correlations among predictors can nevertheless make individual coefficients unstable. A high R squared does not establish a correct functional form, accurate extrapolation or causality. Adding inappropriate controls can create bias rather than remove it. Residual assumptions used for inference should be distinguished from the algebraic procedure that computes least-squares coefficients.

Business use

Businesses should decide whether the primary purpose is forecasting, association or causal explanation before selecting variables and evaluation methods. Preserve data lineage and ensure every predictor is available at the intended decision time. Report uncertainty and errors in meaningful units, with attention to important cases. Examine whether coefficient interpretations are stable and supported by the observed combinations of predictors. Keep simple baselines and avoid adding variables solely to increase training fit. A useful regression analysis communicates both the model's practical contribution and the conditions under which its predictions or interpretations are credible.

Check your understanding

In an additive model, the coefficient of X1 is 2. What does that mean?

Answer: The model predicts a two-unit outcome increase for a one-unit increase in X1 while the other included predictors remain fixed, within the model's applicable context.

Related terms

Learn next

Review simple regression, then interpret coefficients with two predictors and a categorical reference group. Study residual diagnostics and causal diagrams to understand why prediction and causal adjustment require different reasoning.

Sources

Penn State STAT 501, Multiple Linear Regression: https://online.stat.psu.edu/stat501/Lesson05 Penn State STAT 501, Multicollinearity and Other Regression Pitfalls: https://online.stat.psu.edu/stat501/Lesson12

Previous: Multiple instruction, single data Next: Multiple treatments