Is PPR a linear model?

Its projections are linear, but the functions applied to them can make the overall prediction nonlinear.

Does ridge function mean ridge regularization?

No. It refers to dependence on a linear projection, not the familiar squared-coefficient penalty.

Can PPR model interactions?

Yes. A nonlinear function of a combination of predictors can create interactions among the original variables.

AI Glossary / Statistics and probability / Advanced

Projection pursuit regression

Definition

Projection pursuit regression, or PPR, models a numerical response as a sum of flexible functions of linear combinations of predictors. Each component first projects the predictors onto a direction and then applies a one-dimensional function, often called a ridge function. By learning both the directions and the functions, PPR can represent nonlinear relationships and interactions while organizing the model into a collection of lower-dimensional components.

What it means

Ordinary linear regression combines predictors with fixed coefficients to produce a linear response surface. A conventional additive model allows a separate nonlinear function of each original predictor. PPR instead applies functions to learned combinations of predictors, so one component can involve several variables together.

The word ridge here describes a function that varies along a projection direction and is constant across directions perpendicular to it. It does not mean the coefficient penalty used in ridge regression. The fitted components can aid interpretation, but the combined model may still be complicated, and different component representations can describe similar predictions.

How it works

A common single-response form is an intercept plus a sum of terms f_m(a_m-transpose x). Each direction a_m is constrained or normalized to remove avoidable scale ambiguity. The functions f_m are estimated using flexible smoothing methods rather than assumed to be straight lines.

Fitting searches for directions and functions that reduce prediction error, commonly squared error. Algorithms may add terms sequentially, update existing components, and remove less useful terms. Implementation choices determine how thoroughly the components are refitted and how much smoothing is applied.

Select the number of terms and smoothing settings using suitable validation rather than training fit alone. Check sensitivity to initialization, predictor scaling, and outliers. Because optimization can be nonconvex, a fitted solution need not be the global optimum. Predictions outside the observed projected ranges can be especially uncertain, so inspect extrapolation behavior before using the model in new operating conditions.

Three practical examples

A service model predicts processing time using nonlinear functions of combinations of workload and request characteristics.
A manufacturing analyst explores whether several measured inputs jointly influence a numerical quality outcome.
A forecasting team compares a compact PPR model with a linear baseline and other nonlinear methods on held-out observations.

Think of it this way

Think of creating several custom measuring scales from the original inputs. Each scale combines the inputs in a learned direction, and a curve translates that scale into a contribution to the prediction. Adding the contributions creates the final estimate without requiring a full lookup table over every input combination.

Worked example

Consider an original two-predictor model with projected coordinates u = (x1 + x2)/sqrt(2) and v = (x1 − x2)/sqrt(2). Both direction vectors have unit length. Define the prediction as 5 + 2u² − v².

For (x1, x2) = (2, 0), both u and v equal sqrt(2), so the prediction is 5 + 2 × 2 − 2 = 7. For (1, 1), u equals sqrt(2) and v equals zero, giving 9. For (1, −1), u equals zero and v equals sqrt(2), giving 3.

Expanding the formula gives 5 + 0.5x1² + 3x1x2 + 0.5x2². The interaction term appears even though the model is additive in its two projected functions.

This is a specified PPR-form response surface, not the output of an optimization run. It shows representational capability, not proof that a fitting algorithm would discover these directions from a small or noisy dataset. The quadratic functions are chosen only to make the calculations transparent.

Illustrative case study

Fictional case study: Cedar Scheduling finds that a linear model misses curved relationships between request complexity, workload, and processing time. Its analysts try PPR with a small number of terms and inspect the resulting ridge-function plots.

They compare predictive errors on later periods, vary smoothing, and examine whether a few extreme observations dominate the learned directions. The team retains the model only if its performance and interpretation suit the operational use. This fictional case illustrates evaluation choices without asserting that PPR is the best model for real scheduling data.

Common misconceptions and mistakes

PPR is not the same as projecting onto principal components and then fitting a regression. Its directions are learned for the response-fitting task, whereas PCA ordinarily uses predictor variance without the response.

The additive sum of ridge functions does not imply additivity in the original variables; the worked example explicitly contains an interaction. A direction coefficient alone is also not a simple marginal effect because its contribution passes through a nonlinear function. Finally, adding more terms can improve training fit while worsening generalization. Flexibility, smoothing, and validation must be considered together.

Business use

PPR offers a way to model nonlinear business relationships using a limited collection of projected components. It can be useful as an interpretable alternative to a large unrestricted response surface when a small number of directions captures important predictive structure.

For practical use, preserve preprocessing, chosen directions, smoothing settings, and the training domain. Show representative prediction comparisons rather than presenting direction weights as causal effects. Compare with simpler methods and consider maintenance costs. A modest accuracy gain may not justify unstable components or difficult extrapolation in the intended workflow.

Check your understanding

A component is f(x1 + x2), with f(t) = t². Is it necessarily the sum of a function of x1 and a function of x2 alone?

Answer: No. Expanding gives x1² + 2x1x2 + x2², including a joint term. Additivity in projected components differs from additivity in the original predictors.

Related terms

Learn next

Study linear projections and additive models, then inspect how a nonlinear ridge function creates interactions. Learn smoothing and model-selection methods before fitting many components. Compare ridge-function plots with actual predictions and validate performance outside the training sample.

Sources

The R statistics documentation describes projection pursuit regression, ridge-term fitting and removal, smoothing methods, and projection directions, with references to Friedman and Stuetzle: https://www.stat.ethz.ch/R-manual/R-devel/library/stats/html/ppr.html

The two-direction quadratic model, evaluations, and fictional case are original. This entry explains the standard regression form; the example is a hand-specified function and does not claim to reproduce R's fitting algorithm.

Previous: Projection pursuit Next: Projective Rule