Is polynomial regression a nonlinear estimation problem?

Not when the unknowns are ordinary polynomial coefficients; the model is linear in those coefficients.

Does a higher degree always reduce training squared error?

For nested, unconstrained least-squares fits on the same observations, the minimum cannot increase, though numerical issues can affect computation.

Must the fitted curve pass through every point?

No; approximation is usually the aim.

Are coefficients comparable after rescaling x?

Not directly; their interpretation changes with the basis and units.

AI Glossary / Time series and signal processing / Intermediate

Polynomial fitting

Definition

Polynomial fitting estimates a relationship using a sum of powers of an input variable, such as a + bx + cx squared. The coefficients are chosen to match observations according to a fitting rule, commonly least squares. It can describe curvature or a time trend, but a close historical fit does not guarantee reliable forecasts.

What it means

A degree-one polynomial is a straight line; degree two adds a squared term; degree three adds a cubic term. Increasing degree expands the shapes the model can represent. With an intercept, a full degree-d polynomial has d + 1 coefficients, so degree and number of terms are not the same.
Although a fitted curve can be nonlinear in its input, polynomial regression remains linear in its coefficients. Ordinary linear regression methods can therefore fit it after constructing the required powers, subject to the usual numerical and statistical considerations.

How it works

Choose the input scale and polynomial degree, construct columns for one, x, x squared, and any higher powers, then estimate the coefficients. Least squares minimizes the sum of squared vertical residuals between observations and fitted values. Weighted fitting can assign different influence to observations when justified.
Interpolation is a special case. For n distinct input values, there is a unique polynomial of degree at most n minus one passing through any n specified outputs. Repeated inputs with conflicting outputs cannot be interpolated by a single-valued polynomial. A polynomial fit need not interpolate at all.
Large powers and poorly scaled inputs can make coefficient estimation numerically unstable. Centering, scaling, suitable polynomial bases, and stable linear algebra help. They do not prevent statistical overfitting or make distant extrapolation trustworthy.

Three practical examples

A calibration team fits a low-degree curve between a sensor reading and a reference measurement within a validated operating range.
An analyst models a smooth time trend over a limited historical interval and compares it with alternative forecasting methods.
A product team uses a quadratic response model to summarize curvature in experimental observations, while respecting the experimental range and checking whether the model's assumptions are adequate.

Think of it this way

Think of shaping a flexible ruler through a set of marks. A little flexibility captures a bend; too much can twist between marks or swing sharply beyond them. Polynomial degree controls part of that flexibility, but the marks alone do not establish what happens outside the observed region.

Worked example

For an original example, take three observations: (-1, 2), (0, 1), and (1, 4). The least-squares straight line is y_hat = 7/3 + x. Its fitted values are 4/3, 7/3, and 10/3, and its residuals are 2/3, -4/3, and 2/3.
The residual sum of squares is 4/9 + 16/9 + 4/9 = 24/9 = 8/3. The residuals sum to zero and their products with x sum to zero, independently checking the intercept and slope normal equations.
The quadratic y_hat = 1 + x + 2x squared gives values 2, 1, and 4, matching all three observations exactly. It has degree two and three fitted coefficients. At x = 2, the line predicts 13/3, approximately 4.3333, while the quadratic predicts 11.
Neither extrapolated value is validated by these three observations. The contrast shows how the chosen degree changes behavior beyond the data. The zero quadratic training residual is an interpolation result, not proof that the underlying process follows a quadratic.

Illustrative case study

Fictional case: Meadow Instruments fits a calibration curve to readings collected across a specified temperature range. A high-degree curve passes through every calibration point, but its predictions oscillate between points and become implausible just outside the range.
The engineers compare lower-degree fits with additional reference measurements and document the range over which the chosen calibration is supported. They do not advertise the interpolating curve's zero training error as perfect measurement accuracy. This invented case illustrates validation and scope control rather than a real product result.

Common misconceptions and mistakes

Do not confuse degree with coefficient count. A full quadratic contains three coefficients, not two. Do not claim that any collection of n observations can be perfectly fitted by degree n minus one without checking distinct input values and the single-valued relationship.
Do not interpret tiny residuals as reliable long-range forecasting. Higher powers can dominate rapidly outside the observed range. Also distinguish numerical conditioning from model adequacy: a stable computation can still produce a poor scientific or business model.

Business use

Polynomial fitting offers a transparent way to capture modest curvature, particularly for calibration and bounded operating ranges. In forecasting, it should be assessed against methods that handle seasonality, serial dependence, or changing trends when those features matter.
Record the degree, basis, scaling, fitting sample, and validated prediction range. Evaluate on later observations or appropriate held-out measurements. If errors are serially correlated, ordinary regression uncertainty formulas may be inappropriate. Use domain constraints and diagnostics to reject impossible or unsupported extrapolations.

Check your understanding

How many coefficients does the example's quadratic contain?

Answer: Three.

What is the straight line's residual sum of squares?

Answer: Eight thirds.

Why does the quadratic achieve zero training error?

Answer: Three distinct inputs support interpolation by a polynomial of degree at most two.

Which prediction at x = 2 is proven correct?

Answer: Neither; new evidence is needed to assess extrapolation.

Related terms

Polynomial regression
Least squares
Overfitting
interpolation
model degree
extrapolation
numerical conditioning
orthogonal polynomials
spline

Learn next

fit degrees one through three to a modest dataset, inspect both the observed region and a wider plotting range, and compare later prediction errors. Repeat after centering the input to distinguish numerical stability from statistical generalization.

Sources

Sources: NumPy polynomial.polyfit documentation: https://numpy.org/doc/stable/reference/generated/numpy.polynomial.polynomial.polyfit.html . Hyndman, Glossary of Forecasting Terms: https://robjhyndman.com/mwh3/FG4.pdf . Relevant primary documentation and glossary consulted. Worked values and fictional case are original.

Previous: Polyglot persistence Next: Polynomial kernel