Does linear regression require normal predictors?

No. Normality assumptions, when used for particular inference procedures, concern errors rather than requiring every predictor to be normal.

Is R-squared the percentage of correct predictions?

No. It compares squared errors with a reference based on the outcome mean, under the usual definition.

Can it use categories?

Yes. Appropriate indicator or contrast coding represents categories, with coefficients interpreted relative to the coding scheme.

AI Glossary / Statistics and probability / Beginner

Linear regression

Definition

Linear regression models a numerical outcome using a sum of explanatory features multiplied by coefficients, usually with an intercept. In ordinary least squares, the coefficients are chosen to minimize the total squared difference between observed outcomes and fitted values. The method can summarize associations or generate predictions. Its coefficients do not automatically describe causal effects, and a good fit to historical data does not guarantee accurate future predictions.

What it means

For one feature, a familiar form is predicted y = intercept + slope x. The slope describes the fitted change in the predicted outcome for a one-unit increase in x. With several features, a coefficient describes a change while other included features are held fixed, subject to the model's specification.

Linearity is in the coefficients. Features can include indicator variables, squares, or interactions, so linear regression is not restricted to a single straight line plotted against an original measurement. The choice of features determines the relationships the model can express.

How it works

Define the outcome, select features available at prediction time, and prepare a training dataset. Ordinary least squares chooses coefficients to minimize the sum of squared residuals, where each residual is observed minus fitted outcome.

For one predictor with an intercept, the slope is the sum of centered cross-products divided by the sum of squared centered predictor values. The intercept makes the fitted line pass through the sample means. With several predictors, numerical linear-algebra methods solve the corresponding least-squares problem.

Check residual patterns, influential observations, and performance on appropriate validation data. Assumptions for interpreting coefficients, constructing intervals, and making causal claims are separate from the mechanics of computing the fit. Dependence or unequal error variances can require different uncertainty calculations.

Three practical examples

A service team models handling time from case characteristics to estimate workload, checking whether unusually complex cases need a different specification.
A manufacturing analyst relates a measured output to controlled process settings, using the fitted relationship within the tested range.
An AI development team uses linear regression as a baseline for numerical prediction, comparing a more complex system against it on the same held-out cases and business-relevant error measure.

Think of it this way

Think of placing a ruler through a cloud of points so that the squared vertical gaps are as small as possible. The ruler captures a pattern, not every individual point. With multiple features the geometry becomes a plane or higher-dimensional surface, but the same principle of balancing prediction errors remains.

Worked example

Take four fictional observations: x values 1, 2, 3, and 4, with outcomes 2, 4, 5, and 7. The predictor mean is 2.5 and the outcome mean is 4.5.

The sum of centered cross-products is 8, while the sum of squared centered x values is 5. The fitted slope is therefore 8/5 = 1.6. The intercept is 4.5 - 1.6 x 2.5 = 0.5, giving predicted y = 0.5 + 1.6x.

The fitted outcomes are 2.1, 3.7, 5.3, and 6.9. Residuals are -0.1, 0.3, -0.3, and 0.1, whose squared sum is 0.20. The total squared deviation of outcomes from their mean is 13, so training R-squared is 1 - 0.20/13 = 0.9846.

At x = 5 the fitted prediction is 8.5. That value lies beyond the observed predictor range and is an extrapolation. The high training R-squared does not establish that it will be accurate. Four observations also provide very limited evidence for checking a model's assumptions or judging deployment performance.

Illustrative case study

Fictional case: Clearpath Services predicts resolution time for customer requests. Its first regression uses only request length and fits the training data reasonably well.

Residual review shows repeated underprediction for cases requiring an external supplier. The team adds a supplier-involvement indicator that is known when a case opens, then evaluates the revised model on later requests. It avoids including final-resolution notes because those would not exist at prediction time.

Managers use the forecast for staffing with an allowance for uncertainty. They do not interpret the request-length coefficient as proof that shortening a customer's message would shorten the work required.

Common misconceptions and mistakes

Do not mistake correlation or a regression coefficient for causation. Omitted variables, selection, reverse causality, and measurement error can change interpretation.

Do not judge a model only by training R-squared: adding features can improve apparent fit without improving prediction. Outliers and high-leverage observations can strongly influence least squares. Collinear predictors can make separate coefficients unstable. A negative prediction for an inherently nonnegative outcome is a reason to examine the model and target, not simply to assume the calculation is defective.

Business use

Linear regression is useful when teams need a transparent numerical baseline, a compact description of associations, or an estimate within a well-understood operating range. It can make disagreements about feature definitions and units easier to investigate.

For deployment, record the training population, outcome timing, feature availability, and validation design. Compare errors across important groups and operating conditions. Monitor changes in data collection and business processes, because a previously useful relationship can deteriorate even when the calculation and software remain unchanged.

Check your understanding

The worked model has training R-squared about 0.985. Does that prove its prediction at x = 5 is accurate?

Answer: No. The score describes fit to the four observed cases. Prediction beyond their range requires additional justification, and future accuracy must be evaluated using relevant new data or defensible process knowledge.

Related terms

Learn next

Next, learn to read residual plots and distinguish confidence intervals for a mean from prediction intervals for an individual outcome. Then compare ordinary, weighted, and regularized regression for different data conditions.

Residual

Sources

Sources: NIST/SEMATECH, Linear Least Squares Regression, for the model and fitting criterion: https://www.itl.nist.gov/div898/handbook/pmd/section1/pmd141.htm . Google Machine Learning Glossary, Linear Regression: https://developers.google.com/machine-learning/glossary#linear_regression . These primary definitions were read. The four-point dataset and fictional case are original; coefficients, residual sum of squares and R-squared were independently checked with a numerical least-squares solution.

Previous: Linear-quadratic regulator rapidly exploring random tree Next: Linear relationship