Can PLS handle more predictors than observations?

It can construct a limited component representation in such settings, but this does not remove the need for validation and appropriate complexity control.

Is PLS automatically nonlinear?

Standard PLS regression is linear in the supplied predictors; nonlinear variants or transformed inputs are separate choices.

Can it predict several responses?

Yes. PLS methods can use a multivariate response matrix, though performance should be checked for each response.

AI Glossary / Probabilistic machine learning / Intermediate

Partial least squares regression

Definition

Partial least squares regression, or PLS regression, predicts one or more responses using a smaller set of components constructed from the predictors with information from the responses. It is especially associated with data containing many correlated predictors. The component construction seeks useful predictor-response covariance rather than only large variation in the predictors.

What it means

A dataset may have hundreds of measurements that largely move together. Fitting a separate unconstrained coefficient for every measurement can be unstable. PLS builds weighted combinations of those measurements and uses the combinations for prediction.
Unlike principal component regression, the response participates in learning the representation. A predictor direction with large variance is not necessarily useful for predicting the response. Conversely, a lower-variance direction may be useful. This supervised construction also means that fitting PLS components before separating training and validation data can leak response information into evaluation.

How it works

Prepare the predictor matrix X and response matrix Y, usually centering them and considering whether scaling is appropriate. Extract component directions using relationships between X and Y. Form scores by projecting observations onto these directions, then remove the explained components according to the chosen PLS algorithm and continue if additional components are required.
Fit the response relationship through the resulting low-dimensional representation. Standard PLS regression still produces a linear prediction rule in the original predictors, although the way its coefficients are estimated differs from ordinary least squares.
Choose the component count using a suitable validation procedure. PLS variants differ in their normalization and deflation details, so identify the implementation when exact numerical replication matters. Preprocessing, component fitting, and component-count selection must all respect the training-data boundary.

Three practical examples

Spectral prediction: combine many correlated wavelength measurements to predict a measured property, checking performance on samples outside the fitting set.
Sensor regression: reduce a large collection of related sensor features into predictive components before estimating a continuous response.
Multiple-response analysis: model several related response variables from shared predictors, comparing whether a joint representation helps each response rather than only an aggregate score.

Think of it this way

Think of mixing several overlapping microphone signals into a few channels chosen to help hear a particular speaker. The desired speaker influences which combinations are useful. The analogy has limits: PLS components are statistical linear combinations, not necessarily separate physical sources, and a predictive component does not identify a causal mechanism.

Worked example

Consider an original centered, unscaled example with three observations. The two predictor columns are x_1 = (−1, 0, 1) and x_2 = (−1, 1, 0). The response is y = (−2, 1, 1).
For a one-response, first-component covariance construction, X-transpose y = (3, 3). Normalizing this vector gives weights w = (1/square-root(2), 1/square-root(2)). The component scores are t = Xw = (−square-root(2), 1/square-root(2), 1/square-root(2)).
Regressing y on this single score gives coefficient q = (t-transpose y) / (t-transpose t) = (3 square-root(2)) / 3 = square-root(2). The fitted values are therefore t q = (−2, 1, 1), exactly matching this deliberately simple response.
For a new centered predictor pair (2, −1), the score is 1/square-root(2), producing prediction 1. Here the original-variable rule is y_hat = x_1 + x_2. This example verifies one component under stated preprocessing; its perfect training fit does not show generalization or imply that one component always suffices.

Illustrative case study

Fictional case: Fir Materials predicts a laboratory property from many spectral measurements. The team compares several PLS component counts and a simpler regression baseline. All scaling and component extraction are fitted inside training folds.
A model with many components fits the training records closely but performs worse on held-out production batches. Analysts choose the complexity using those held-out results and inspect whether a change in measurement equipment alters predictions. They avoid naming components as physical causes without independent evidence. This case illustrates model selection, not a claimed laboratory outcome.

Common misconceptions and mistakes

PLS is not principal component regression. Principal components are constructed from predictor variation without using the response; PLS uses predictor-response information.
Do not confuse PLS regression with partial regression or partial correlation. Those terms concern adjustment for other variables and do not describe the same component method.
More components do not guarantee better prediction, especially with limited observations. Scaling choices can materially affect covariance-based component construction.
Finally, a component with strong predictive value is not necessarily interpretable as one real-world factor. Correlated predictors can distribute weights in ways that make causal or feature-importance claims unreliable.

Business use

PLS offers a useful baseline for predicting outcomes from numerous related continuous measurements. Its value depends on predictive performance and stability, not merely on compressing many columns into a small component count.
Assess error on realistic held-out groups or periods and compare with regularized regression and other suitable methods. Document centering, scaling, missing-data treatment, and selected components. Where interpretation matters, distinguish the model's predictive use from claims about which measured variable causes the outcome.

Check your understanding

A team fits PLS components using all observations and their responses, then cross-validates only the final regression. Is that an independent evaluation?

Answer: No. The response-informed components have already used information from the validation observations. Component fitting and related preprocessing must be repeated within the training folds.

Related terms

Learn next

Compare predictor-only principal components with response-informed PLS directions on a small centered dataset. Then examine how scaling and component count affect predictions. Build validation around the entire fitting pipeline before interpreting component weights or comparing model performance.

Sources

scikit-learn, Cross decomposition, official explanation of PLS methods and PLS regression: https://scikit-learn.org/stable/modules/cross_decomposition.html The official cross-decomposition algorithm and regression distinctions were independently consulted.

Previous: Partial least squares path modeling Next: Partial-order planning