Does zero impurity mean the tree is perfect?

It means the evaluated node contains only one class under its counts or weights; future predictions can still be wrong.

Is Gini impurity the same as entropy?

No. Both measure class mixing and can guide splits, but they use different formulas and can prefer different candidate splits.

Does changing class names change impurity?

No. Only the proportions matter, not whether classes are named A and B or use numerical codes.

AI Glossary / Causal inference and experimentation / Intermediate

Gini impurity

Definition

Gini impurity measures how mixed the class labels are within a group of observations, commonly a node in a classification tree. If class proportions are p1 through pK, it equals 1 minus the sum of their squares. A pure node containing only one class has impurity zero. Decision-tree algorithms can use reductions in weighted child-node impurity to choose splits. Gini impurity is not the same as the Gini coefficient of inequality or a Gini-style ranking measure derived from ROC AUC.

What it means

A classification tree repeatedly asks questions that divide examples into groups. A useful split often produces child groups with more consistent labels than the parent. Gini impurity quantifies that mixing without depending on the names assigned to the classes. In a binary node, impurity is largest at an equal class mix and decreases as one class dominates. It describes the labels within a node, not the model's future error rate or the causal importance of the splitting feature.

How it works

At a node, estimate each class's fraction using observation counts or declared sample weights. Calculate G = 1 - sum(p_k squared). For a candidate split, compute each child's impurity and average the child values using their proportions of the parent weight. Subtract this weighted child impurity from the parent's impurity to obtain the local reduction. Compare valid candidate splits under the tree's constraints. Stopping rules, minimum leaf sizes, depth limits, and pruning help prevent the algorithm from growing a tree that simply memorizes training examples.

Three practical examples

Decision-tree training: choose a split that makes child nodes more consistent in their target labels.
Model explanation: inspect a node's class distribution to understand why it is mixed or nearly pure.
Training diagnostics: compare impurity reduction with held-out performance to detect when increasingly pure training leaves fail to improve generalization.

Think of it this way

Think of jars containing colored beads. A jar with one color is pure; a jar containing several colors is mixed. Gini impurity summarizes that mixture. The analogy stops where purity is treated as usefulness: separating beads perfectly in a small sample does not prove a rule will sort new beads correctly or that color is the right property to predict.

Worked example

Suppose a parent node contains ten examples: six in class A and four in class B. Its Gini impurity is 1 - 0.6 squared - 0.4 squared = 0.48. A proposed split sends four A examples to the left child, which is pure and has impurity zero. The right child contains two A and four B examples. Its proportions are 1/3 and 2/3, so impurity is 1 - 1/9 - 4/9 = 4/9, approximately 0.4444.
The weighted child impurity is (4/10) times 0 + (6/10) times 4/9 = 4/15, approximately 0.2667. The local impurity reduction is 0.48 - 0.2667, approximately 0.2133. Simply averaging the two child impurities would give about 0.2222 and would be wrong for this unequal-size split. The reduction describes improvement on the node's training distribution; it is not a 21.33-percentage-point improvement in classification accuracy and does not guarantee the same benefit on new examples.

Illustrative case study

A fictional routing team builds a classification tree and finds a feature that creates many tiny pure leaves. The feature nearly identifies individual training records, so the tree's impurity falls sharply while held-out routing accuracy does not improve. The team limits leaf complexity and checks more realistic evaluation splits. It also treats impurity-based feature importance cautiously. The example shows why a split criterion is a training tool, while deployment quality needs separate evidence from data the model did not use to fit its rules.

Common misconceptions and mistakes

Gini impurity is not a probability that the majority-class prediction is wrong. It can be interpreted as the probability that two independent class draws from the node's distribution disagree, which is a different quantity. Its maximum depends on the number of classes: for K equally frequent classes it is 1 - 1/K. It is not always bounded by 0.5; that upper bound applies to binary classes. A feature with high impurity-based importance is not necessarily causal and may benefit from many possible split points.

Business use

Use impurity values to understand tree construction and node composition. Compare candidate splits using the correct child weights and keep sample-weight choices explicit. Evaluate the fitted tree separately on an appropriate held-out sample, including important classes and error types. When using feature importance, complement training-based impurity reductions with other evidence, such as held-out permutation analysis and domain review. Avoid presenting low impurity as proof of fairness, reliability, or a meaningful underlying mechanism.

Check your understanding

Why must the child impurities be weighted by child size in the worked example?

Answer: The six-example child represents more of the parent's observations than the four-example child. Weighting preserves the average impurity experienced by the parent sample rather than giving unequal groups identical influence.

Related terms

Decision tree
Entropy
CART
Pruning
Overfitting
classification node
impurity reduction
sample weights
feature importance
class distribution

Learn next

Calculate impurity for pure, balanced, and uneven class mixtures. Then compare two candidate splits using weighted child impurity, including one with very unequal child sizes. Study tree pruning and held-out evaluation next so training purity is kept separate from generalization and practical predictive quality.

Pruning

Sources

Scikit-learn, decision trees and classification criteria: https://scikit-learn.org/stable/modules/tree.html Scikit-learn, DecisionTreeClassifier: https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html Scikit-learn, permutation importance: https://scikit-learn.org/stable/modules/permutation_importance.html

Previous: Gibbs sampling Next: Gini's mean difference