Is the same subset used at every node?

In the common random-forest form described here, a new subset is sampled at each node. Other variants use different sampling levels.

Does a sampled feature automatically become the split?

No. It becomes eligible for comparison; the split criterion selects among the candidates.

Can attribute sampling work without row sampling?

Yes, feature randomization and row resampling are distinct mechanisms. The exact combination depends on the ensemble algorithm and configuration.

AI Glossary / Generative AI and language / Intermediate

Attribute sampling

Definition

Attribute sampling is a decision-tree training technique that randomly selects a subset of input features to consider when choosing a split. In a common random-forest design, a new subset is selected at each node. Attribute here means feature, such as quantity, duration or product type.

What it means

A tree normally searches for a useful condition that divides the examples reaching a node. Without attribute sampling, it can consider every eligible feature. With sampling, it searches only the randomly chosen candidates for that node.

This introduces diversity among trees. If one strong feature is always available, many trees may repeatedly select similar early splits. Occasionally withholding it gives other features a chance to influence the structure.

The term in this entry follows the source glossary's decision-forest meaning. It does not refer to sampling the next token from a language model, selecting people for a survey, or auditing whether items possess an attribute. It changes which columns are considered during tree construction, not which rows are observed.

How it works

Choose a candidate-feature count or fraction as part of the training configuration. At a node, draw a subset of eligible features using the method's randomization rule. Evaluate possible split conditions using those features and choose a suitable condition according to the tree's criterion.

Repeat as the tree grows, commonly drawing a fresh subset at each node. Other ensemble designs may sample feature subsets at different levels, such as per tree, so identify the exact convention.

The candidate count balances two effects. A small subset can make trees less similar, which may help the ensemble, but can also make individual splits weaker by excluding useful variables. A larger subset offers more options but can increase similarity among trees. Validation should assess the resulting forest, not just the quality of one split.

Attribute sampling can operate alongside bootstrap sampling of training examples. The two randomize different dimensions of the problem. Neither guarantees independent tree errors, prevents data leakage or removes the need for a credible evaluation.

Three practical examples

A demand model uses a random forest whose nodes consider only a subset of the available historical and product features.
A defect-classification team tunes the candidate-feature count while measuring the complete ensemble's performance on held-out parts.
A model researcher compares row resampling and feature sampling separately to understand which source of randomness changes prediction diversity.

Think of it this way

Think of several analysts preparing decision rules from different temporary menus of evidence. Restricting the menu can encourage them to build different rules instead of all starting with the same obvious clue.

The analysts are only an analogy. A tree applies a numerical split criterion, and random restrictions can omit essential evidence. Diversity is useful only when the combined predictions remain sufficiently informative.

Worked example

Sampling candidates before selecting a split

An invented tree has five eligible features: A, B, C, D and E. At a particular node, the training method randomly selects two candidates, B and D. Suppose B's best available split improves the criterion by 0.12, while D's improves it by 0.08.

The node chooses B, assuming all other split requirements are satisfied. Feature A might have achieved 0.20, but it was not in this node's candidate subset. A later node can draw a different subset and use A.

With uniform sampling of two distinct features from five, any given feature has probability 2 / 5 of being included at this draw. That is not its probability of becoming the selected split feature: selection also depends on its competing split scores and the examples reaching the node.

Illustrative case study

A fictional inventory company, Elm Stock, observes that many trees in its ensemble start with the same highly predictive historical quantity feature. The developers experiment with smaller candidate subsets to encourage alternative structures.

One setting improves aggregate validation behavior, while an excessively small subset weakens the model on product groups with few informative inputs. The team selects using representative validation evidence and examines the affected groups rather than assuming maximum randomness is best.

It also checks whether the strong feature was available at prediction time. Attribute sampling would only sometimes omit a leaked feature; it would not make that leakage acceptable. The fictional case separates legitimate diversity tuning from correction of an invalid input.

Common misconceptions and mistakes

Attribute sampling is not permanent feature removal. A feature excluded from one node can be considered elsewhere in the tree or forest.

It is not the same as bootstrap sampling, which resamples examples. It also differs from randomly choosing a split threshold: some algorithms add that separate form of randomization.

Do not assume fewer candidates always improve accuracy or reduce deployment input requirements. The completed forest may still use many features across its nodes. A feature's frequency of selection is also not a causal measure of its importance, particularly when features overlap in information.

Business use

Treat candidate-feature count as a tunable part of an ensemble training procedure. Compare predictive quality, important error types and training cost under a consistent evaluation design.

Record the sampling settings and random seed so an experiment can be reproduced. If the business objective is to reduce data-collection costs, inspect the features actually used by the final forest and test a revised feature set. Training-time random exclusion alone does not guarantee that expensive inputs can be omitted from production.

Check your understanding

A feature is absent from the random candidate subset at a tree's root. Does that mean the feature is absent from the entire trained forest?

Answer: No. It may be selected as a candidate and used at other nodes or in other trees. One local exclusion is not global feature removal.

Related terms

Feature
Random forest
Decision tree
Bagging
Feature Selection
split criterion
bootstrap sample
random subspace
ensemble diversity

Learn next

Build a tiny split-selection example with and without a restricted candidate set. Distinguish the random draw from the subsequent best-split choice. Then compare candidate counts using the whole forest's validation results and inspect whether all required production features remain necessary.

Sources

Sources: Google Machine Learning Glossary, attribute sampling: https://developers.google.com/machine-learning/glossary#attribute-sampling Google, random forests and feature sampling: https://developers.google.com/machine-learning/decision-forests/random-forests Breiman, Random Forests: https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf

Previous: Attribute Preservation Property Next: Attribute - value system