AI Glossary / Statistics and probability / Intermediate

Regression spline

Definition

A regression spline models a relationship using polynomial pieces joined at specified locations called knots. The pieces are constrained to meet with an intended degree of smoothness, allowing the fitted function to bend without using one high-degree polynomial across the entire range. The spline can be represented by basis functions whose coefficients are estimated in a regression model. Its flexibility depends on the polynomial degree, knot placement and any constraints or penalties imposed.

What it means

Splines provide a structured way to model nonlinearity. A linear spline is continuous but can change slope abruptly at knots. A conventional cubic spline usually joins cubic pieces with continuous first and second derivatives at ordinary interior knots. Natural cubic splines add boundary restrictions that produce linear behavior beyond the boundary knots. A regression spline need not pass through every observation: it is fitted to a regression objective, unlike an interpolation curve required to hit specified points.

How it works

Choose the predictor, knot locations, polynomial degree and any boundary conditions. Construct a spline basis evaluated at each observation, then fit its coefficients using a suitable regression method. Select complexity with design-appropriate validation or other justified criteria. Inspect the fitted curve and uncertainty over the observed range, especially near boundaries and sparse regions. For new predictions, use the same basis definition and knot locations. Penalized spline approaches add a smoothness penalty and should be distinguished from an unpenalized fixed-knot fit.

Three practical examples

Example 1: A service model lets the relationship between workload and delay bend across the observed workload range. Example 2: A researcher models a nonlinear association between age and an outcome without imposing a single global polynomial. Example 3: A forecasting model uses a smooth function of a continuous predictor when a straight line is inadequate. These applications require enough data to support flexibility, and the spline alone does not establish that the association is causal.

Think of it this way

Think of shaping a flexible strip with a limited set of controlled bends. Knots determine where polynomial pieces meet, while smoothness rules prevent arbitrary breaks in the strip. Too much flexibility can follow noise; too little can miss real shape. The analogy concerns the mathematical function, not a guarantee that a visually smooth curve is a reliable prediction or explanation.

Worked example

Consider the linear spline f(x) = 2 + x + 3 max(x - 2, 0), with a knot at two. For x at most two, the hinge term is zero and f(x) = 2 + x. For x above two, f(x) = 2 + x + 3(x - 2) = 4x - 4. Predictions at x = 0, 1, 2, 3 and 4 are 2, 3, 4, 8 and 12. Both pieces equal four at the knot, so the function is continuous. Its slope changes from one to four. The basis functions are 1, x and max(x - 2, 0); with the knot fixed, their coefficients can be estimated by ordinary linear regression even though the resulting curve is not globally a straight line. These coefficients are stipulated for illustration, not fitted from observations.

Illustrative case study

Fictional case study: Fir Planning sees a curved relationship between workload and completion time. Analysts compare a straight-line baseline with spline models of different complexity using held-out periods. They inspect the tails, where few observations support the curve, and avoid claiming a sharp operational threshold simply because a knot was placed at a convenient value. This fictional case illustrates model evaluation and does not claim a measured accuracy gain or that one knot configuration is generally best.

Common misconceptions and mistakes

Common mistakes include treating knots as automatically discovered real-world thresholds, assuming every spline interpolates the data, and interpreting individual basis coefficients as simple marginal effects. Different bases can represent the same fitted function with different coefficients. Too many knots can overfit, while boundary behavior can be misleading during extrapolation. Natural cubic splines constrain tail shape but do not make distant extrapolation trustworthy. Smoothing splines and fixed-knot regression splines are related methods with different complexity controls.

Business use

Business use: Regression splines can capture nonlinear patterns while retaining a structured and inspectable function. Define the target and select flexibility based on validation rather than visual appeal alone. Plot meaningful predicted changes and uncertainty instead of presenting a table of opaque basis coefficients. Document knots and transformations so deployment reproduces the fitted model. Review whether important decisions fall in sparse regions where the curve depends strongly on modeling choices.

Check your understanding

Why is the worked spline continuous at x = 2?

Answer: Both polynomial pieces give the same value four there.

What changes at the knot?

Answer: The slope changes from one to four. A cubic spline with standard smoothness constraints would impose additional derivative continuity that this simple linear spline does not have.

Related terms

Knots are locations where spline pieces join

Learn next

Plot the worked hinge-basis function and calculate its slopes on each side of the knot. Fit coefficients to a small dataset, then compare one knot with several knots using held-out error. Study cubic and natural cubic constraints before interpreting their boundary behavior. Always retain the original knot and basis specification when predicting new observations.

Sources

Sources: R documentation, Regression Spline Functions and Classes: https://stat.ethz.ch/R-manual/R-devel/library/splines/html/00Index.html ; Introduction to Statistical Learning, Non-Linear Modeling lab: https://intro-stat-learning.github.io/ISLP/labs/Ch07-nonlin-lab.html . These primary software and author-maintained educational sources describe spline bases and regression fitting. The hinge-spline calculation and fictional workload case are original illustrations.

Previous: Regression planning Next: Regression surface