Is it the same as dimensionality reduction?

No. Selection keeps a subset of original features; methods such as principal component analysis create transformed combinations.

Does univariate mean there is only one feature in the final model?

No. Each feature is scored separately, but many can be retained.

Can it detect every nonlinear relationship?

That depends on the score, and interactions between features remain a particular limitation of separate evaluation.

AI Glossary / AI and machine learning / Intermediate

Univariate feature selection

Definition

Univariate feature selection evaluates input features one at a time and selects those that score well under a chosen statistical criterion. It is a way to reduce the set of inputs before fitting a model.

What it means

A dataset may contain hundreds of candidate columns, many of which add little useful information. Univariate selection provides a relatively simple screening step by examining each feature separately.

In supervised settings, a score often measures the relationship between one feature and the target. The method can retain a fixed number of top-scoring features or apply a statistical threshold, depending on the procedure.

The word univariate is the key limitation. The screen does not directly assess whether two weak-looking features become useful together. It can also select several redundant features that each show a similar individual association.

For business readers, the result is a shortlist under a specific test, not proof that selected variables cause an outcome or that rejected variables have no possible value.

How it works

The team chooses a scoring method appropriate to the feature and target types. A classification task and a numerical prediction task may need different tests or estimators. Some methods have assumptions or input restrictions.

Each feature receives a score using the training examples. A selection rule then retains features, such as the best k scores or those meeting a chosen criterion.

Selection belongs inside the training pipeline and inside each cross-validation training fold. Screening the entire dataset before splitting can allow evaluation labels to influence which features survive.

The resulting model is compared with alternatives using independent data. The team checks predictive performance, stability and operational cost, remembering that a statistically associated feature may still be unavailable or inappropriate at prediction time.

Three practical examples

Survey analysis: screen many questionnaire fields before fitting a prediction model, while treating results as associations.
Text classification: select informative word-count features from a large vocabulary to reduce model size.
Sensor modeling: compare individual sensor measurements with a target before examining more complex combinations.

Think of it this way

Think of auditioning musicians one at a time before forming an ensemble. Individual ability matters, but the audition does not reveal every useful combination or duplication of roles.

The analogy has a limit. Statistical feature scores measure a specified relationship in data, not overall quality. A feature with a low individual score may be valuable through an interaction that the screening procedure cannot see.

Worked example

Why individual screening can miss an interaction

Consider an invented dataset with two binary features, A and B. The target is one when exactly one feature is one, and zero otherwise. The four equally common combinations are 00 to 0, 01 to 1, 10 to 1 and 11 to 0.

Looking at A alone, half the records with A = 0 have target one, and half with A = 1 do too. B has the same pattern. Neither feature individually distinguishes the target in this balanced example.

Together, however, they determine it perfectly through the stated rule. A univariate screen can therefore discard information needed for a useful interaction.

This teaching example does not imply that screening is always harmful. It shows why its assumptions must match the kinds of relationships the task may contain.

Illustrative case study

A fictional manufacturer screens sensor columns using all available labeled records before evaluating a model. The selected features appear strong, but the evaluation has already influenced their selection.

The team moves the screening step inside the training pipeline so each validation fold remains independent of its feature-selection decision. It also tests whether interaction-capable models benefit from retaining some features excluded by the simple screen.

The lesson is that feature selection is part of learning. Treating it as harmless preparation can leak target information and overstate performance.

Common misconceptions and mistakes

A selected feature causes the target. The score usually measures association, not causation.
Rejected features are useless in every model. Interactions and alternative representations may reveal value that individual screening misses.
Screening before cross-validation is always safe. If target information guides selection across all records, the validation procedure is contaminated.

Business use

Univariate selection can reduce storage, computing and maintenance when many candidate inputs are weak. It is also an interpretable starting point for discussing which columns deserve closer investigation.

Ask which score was used, what assumptions it makes and whether selection was nested within evaluation. Check stability across samples and time. The operational benefit may come from removing expensive data dependencies, but only if the resulting model still meets performance requirements.

Check your understanding

Two individually uninformative features jointly determine the target. Is a one-feature-at-a-time screen guaranteed to retain them?

Answer: No. Their value comes from an interaction the screen may not detect. The evaluation should consider whether such relationships are plausible for the task.

Related terms

Learn next

Read feature selection and feature interaction. Check where screening occurs in your evaluation pipeline, then compare a screened model with a suitable unscreened baseline.

Feature Selection

Sources

scikit-learn, Univariate feature selection: https://scikit-learn.org/stable/modules/feature_selection.html#univariate-feature-selection. Primary documentation consulted for method scope. The binary interaction example and manufacturing case are original.

Previous: Univariate design Next: Universal approximation theorem