Can the same feature appear in several learners?

Yes. Independently sampled subsets can overlap.

Is this limited to decision trees?

No. The general strategy can be applied to other suitable base learners.

Does it guarantee better accuracy?

No. The result depends on feature relevance, redundancy, learner behavior and the task's data distribution.

AI Glossary / AI and machine learning / Advanced

Random subspace method

Definition

The random subspace method is an ensemble-learning approach in which different models are trained on randomly selected subsets of the available features. Their predictions are then combined. By giving models different views of the same observations, the method aims to create useful diversity among their errors.

What it means

If a dataset contains many measurements, every model need not see all of them. One model might use a subset of sensor readings, while another sees a different subset. The resulting models may rely on different patterns, making their combined prediction more stable than a single model in suitable conditions. The benefit is conditional: removing important features can also weaken individual models. The method is therefore a balance between the information available to each learner and the diversity created across the ensemble.

How it works

Choose a base learning algorithm, an ensemble size and a feature-subset size. Sample a set of columns for each learner, train it using those columns, and store the feature selection with the fitted model. At prediction time, route the corresponding columns to each learner and aggregate outputs through voting or averaging. Scikit-learn's documentation distinguishes random subspaces from bagging over observations and random patches over both observations and features. Random-forest implementations often select candidate features at individual splits, which is related but not identical to fixing one feature subset for an entire learner.

Three practical examples

A hypothetical sensor-classification team trains several models on different groups drawn from a large set of available measurements, then compares their combined error with a full-feature baseline.
A document-classification experiment uses different subsets of text features for its component models to investigate whether ensemble diversity improves held-out results.
A research workshop varies the number of features per learner, showing how very small subsets can create diverse but weak models and larger subsets can produce more similar predictions.

Think of it this way

Think of asking several reviewers to examine different selections of evidence before combining their judgments. Their different views can expose patterns that one fixed perspective misses. The analogy has a limit: reviewers who all miss a crucial fact can still agree on the wrong answer. Diversity is useful only alongside enough relevant information and sound aggregation.

Worked example

Suppose a dataset has six features, A through F. Three learners receive subsets {A,B,D}, {B,C,E} and {A,E,F}. For a new observation, they predict class 1, class 0 and class 1. An equal majority vote selects class 1. This does not imply a calibrated probability of two-thirds. If feature C contains an essential signal, only the second learner can use it in this tiny ensemble. The example shows both the voting mechanism and the risk of excluding important information. A real evaluation would vary subsets and test whether the ensemble improves results across many unseen observations.

Illustrative case study

A fictional equipment-monitoring team has dozens of partially redundant measurements. It tests a random-subspace ensemble with several feature-subset sizes, reserving later operating periods for assessment. Some configurations improve average classification but miss an uncommon state detected by a particular measurement. The team investigates representation of that signal and compares alternative ensemble designs rather than accepting the average score alone. It stores each learner's feature mapping and validates the complete prediction pipeline. The selected approach must meet the monitoring requirement across relevant conditions, with a documented trade-off between ensemble size, latency and error.

Common misconceptions and mistakes

Random subspaces sample features, not necessarily observations. They do not automatically identify the best feature subset, and a feature's absence from one learner is not evidence that it is unimportant. More learners cannot reliably repair a fundamentally unsuitable representation. Random feature selection during training does not automatically make the deployed system tolerant of missing input values. Each learner still expects its defined inputs unless missingness is handled explicitly. Finally, related random-forest procedures should not be described as identical without specifying where and how the feature sampling occurs.

Business use

The method can be a useful comparison when a task has many features and enough redundancy to support different informative views. Evaluate whether it improves reliability at an acceptable serving cost. Track performance for rare but important cases and inspect whether essential signals reach enough learners. Keep feature order and preprocessing consistent, and save random seeds or explicit subsets for reproducibility. Compare with a single well-tuned model as well as other ensembles. The business case depends on the complete improvement, including maintenance and explanation costs.

Check your understanding

What is sampled in a random-subspace ensemble?

Answer: The feature columns used by each learner.

How does that differ from bootstrap bagging?

Answer: Bootstrap bagging samples observations with replacement.

Why can an ensemble of very diverse models still perform poorly?

Answer: Each model may lack enough useful information. Explain why the three-vote example should not be reported as a two-thirds chance that the selected class is correct.

Related terms

Learn next

Compare sampling rows, columns and both on the same dataset. Then examine the relationship between individual-model quality and error correlation. Learn how to evaluate feature-subset size and ensemble size without repeatedly tuning against the final test set.

Test set

Sources

Scikit-learn, ensemble methods and the distinction between random subspaces, bagging and random patches: https://scikit-learn.org/stable/modules/ensemble.html Tin Kam Ho, The Random Subspace Method for Constructing Decision Forests, 1998: https://doi.org/10.1109/34.709601 The six-feature vote and equipment-monitoring case are original illustrations, not published performance measurements. Independent review consulted the official ensemble documentation's sampling taxonomy and voting rule. These distinguish random feature subspaces, row bagging and random patches. The three-learner illustration was traced separately, including which learner can access feature C.

Previous: Random start Next: Random surfer model