AI Glossary / AI and machine learning / Advanced
Basis function regression
Definition
Basis function regression is a way of fitting curved relationships by transforming an input into a set of fixed building-block functions, such as powers or waves, and then fitting a linear combination of them.
What it means
Ordinary linear regression fits a straight line. But many real relationships curve: sales rise with price promotion, then flatten; energy use falls and rises with temperature.
Basis function regression keeps the simplicity of linear fitting but gives it curved ingredients. Instead of using x directly, it builds new features from x, such as x squared, x cubed, or a set of smooth bumps, and fits weights for them.
The model is linear in its weights, so it can be fitted with the same reliable methods as a straight line, yet the resulting curve can bend.
Common basis choices include polynomials, sine and cosine waves, and localized bumps such as splines or radial functions. The choice shapes what curves are easy to fit.
For a business, this is a useful bridge between straight-line models, which are easy to explain, and complex models, which are not.
Splines are the most practical family for business data. A spline ties together several gentle polynomial pieces, each local to a range of the input, so the curve can bend where the data bends without the wild swings a single high-degree polynomial produces.
How it works
Choose a set of basis functions, for example the first four powers of the input. Each one turns the raw input into a new column of numbers.
Fit ordinary regression on these new columns. The method finds the weight for each basis function that best matches the data, usually by minimizing squared errors.
The final prediction is a weighted sum of the basis functions evaluated at the input, which traces out a smooth curve.
Too few basis functions underfit, and too many overfit, wiggling through the noise. Penalties on the weights, or checking performance on held-out data, help pick a sensible level.
Regularization shrinks the weights of basis functions that contribute little, which is a second way to control overfitting alongside choosing fewer functions.
Three practical examples
- Pricing: modeling how demand curves rather than falling in a straight line as price rises.
- Energy: representing usage that dips in mild weather and rises in heat and cold.
- Seasonality: using sine and cosine terms to capture yearly cycles in sales.
Think of it this way
Think of drawing a curve with a set of flexible rulers. A single ruler gives a straight line, but bending and combining several in the right proportions lets you trace a smooth curve.
The limit: with enough rulers you can trace every wobble in the data, including noise, which does not help predict anything new.
Worked example
modeling a seasonal sales cycle
Suppose a shop sees sales peak in December and dip in summer. A straight line over the year cannot capture this.
The analyst adds basis functions: one sine and one cosine of the day of year. Linear regression on these two columns fits a smooth yearly wave.
She compares fit on the last year, held out from training, and adds a second pair of waves only if it clearly improves. The figures are illustrative.
The final model is still a sum of weighted terms, so she can explain which part is the yearly cycle and which is the underlying trend.
Illustrative case study
A fictional utility models demand against temperature with a straight line. It underpredicts in cold snaps and heat waves.
The analysts switch to spline basis functions for temperature. Predictions improve at extremes, which is when errors cost the most. They keep the number of basis functions modest and test on a held-out winter.
The invented lesson: a flexible curve on a linear framework captures nonlinearity while staying explainable.
Common misconceptions and mistakes
- It is a non-linear method in the fitting sense. It is linear in the weights, with non-linear building blocks.
- More basis functions always fit better. They fit the training data better, but may predict new data worse.
- You must use polynomials. Many families are used; splines and waves are often better behaved.
Business use
Use basis functions when a straight line misses a clear curve but you still want a model you can explain and check.
Keep the number of terms small, validate on held-out data, and plot the fitted curve for a sense check before using it.
Plotting the fitted curve is not optional. A curve that rockets beyond the observed range is a warning that the model is extrapolating, and predictions outside the data range should be treated with care.
People also ask
Is this the same as polynomial regression?
Polynomial regression is one example, using powers of the input as basis functions.
Why is it still linear regression?
Because the model is a linear combination of the basis functions, with fitted weights.
How do I stop overfitting?
Limit the number of terms, add a penalty on weights, and test on unseen data.
Check your understanding
Why can you fit a curve using ordinary linear regression machinery?
Answer: Because the model is linear in its weights: the curve comes from non-linear basis functions of the input, but the weights are found by linear fitting.
Related terms
Learn next
AI for Business Owners covers using forecasts and models responsibly in business decisions: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]
More in AI and machine learning
- Bagging - Bagging, short for bootstrap aggregating, is an ensemble method that trains many models on different random resamples of the training data and combines...
- Ball tree - A ball tree is a data structure that organizes points into nested regions defined by a center and a radius.
- Base rate - A base rate is the underlying frequency or probability of an event in a relevant population before considering a particular test result or signal.
- Batch - In machine learning, a batch is a group of training examples processed together in one step before the model updates its parameters.
- Batch size - Batch size is the number of training examples a model works through before it makes one adjustment to itself, a setting that trades speed, memory use and...
- Bayes factor - A Bayes factor compares how well two specified models predict observed data.
Sources
[S13] Minka, Statistical Learning/Pattern Recognition Glossary; [S12] Aalto University, Dictionary of Machine Learning. Full URLs are in the source ledger. Verified 11 October 2026.