What happens when split information is zero?

The ratio is undefined; a test sending all cases into one branch supplies no useful partition and needs appropriate exclusion or handling.

Can gain ratio be used with numeric attributes?

Yes, through candidate threshold tests, although the exact selection and correction procedures depend on the learner.

Is gain ratio the same as Gini impurity?

No. Gini-based criteria use a different impurity calculation and do not apply this information-gain-to-split-information ratio.

AI Glossary / Graphs and symbolic AI / Intermediate

Gain ratio

Definition

Gain ratio is a decision-tree split criterion that divides information gain by the split information of the candidate test. It adjusts for how the test partitions the training cases, helping reduce information gain's preference for attributes with many distinct outcomes.

What it means

Information gain measures how much a split reduces uncertainty about class labels in the training data. A record identifier can appear excellent under this measure: splitting into one branch per record creates pure branches even when the identifier has no useful predictive meaning.

Gain ratio considers the information needed to identify the branch as well as the reduction in class uncertainty. A large number of small branches can have substantial split information, reducing the resulting ratio.

The criterion is associated with Quinlan's C4.5 family. It is a local heuristic for choosing a split, not a guarantee of the best complete tree, useful generalization or causal relevance of the selected attribute.

How it works

Compute the entropy of class labels at the current node. For each candidate test, calculate the weighted average class entropy in its child branches. Subtracting that average from the parent entropy gives information gain.

Next calculate split information from the branch proportions themselves: minus the sum of p times log base two of p across nonempty branches. Divide information gain by this quantity when the denominator is positive.

A very small denominator can make a ratio misleadingly attractive even when absolute information gain is weak. The standard C4.5 selection described by Quinlan therefore considers tests with at least average gain before choosing the largest ratio. Other details, including handling numeric thresholds, missing values and pruning, depend on the implementation and version.

Three practical examples

A classification-tree lesson contrasts a meaningful category split with a unique-record identifier that memorizes the training cases.
A data-science team reviews why a C4.5-style learner chose one attribute over another at a particular node.
A model-audit exercise checks whether high-cardinality columns distort a tree's split choices and whether predictive validation supports the resulting model.

Think of it this way

Think of organizing files into folders. Giving every file its own folder removes all mixing within folders, but it may provide no reusable organizing principle.

Gain ratio asks about the class information gained relative to the information in the partition. The analogy does not mean every small folder is bad or that the criterion measures human convenience, storage cost or business value directly.

Worked example

Comparing a useful split with an identifier

Suppose a node contains eight training cases: four positive and four negative. Its class entropy is one bit.

Test A creates two equally sized branches. One contains all four positive cases and the other all four negatives. Child entropies are zero, so information gain is 1 - 0 = 1 bit. The branch proportions are one-half and one-half, giving split information one bit. Gain ratio is therefore 1/1 = 1.

Test B splits on eight unique identifiers, producing eight one-case branches. They are also pure, so information gain is again one bit. Each branch has proportion one-eighth. Split information is -8 × (1/8) × log2(1/8) = 3 bits, giving gain ratio 1/3, approximately 0.333.

If these are the only two candidates, both meet their average-gain threshold of one bit, and A has the higher ratio. The calculation illustrates the intended penalty for fragmentation. It does not prove that A will remain predictive on unseen data; that requires validation beyond this training table.

Illustrative case study

The fictional Cedar Analytics team builds a tree to classify service requests. An early model obtains pure training leaves after using a request number. Reviewers recognize that each request has a unique number and that future requests will have new values.

They examine the split criterion and compare information gain with gain ratio using a small diagnostic dataset. They also remove inappropriate identifier features from the actual modeling specification and evaluate predictions on held-out requests.

Cedar does not rely on the ratio alone to detect every leakage problem. A field that reveals the eventual outcome could still look highly informative under several criteria. The case demonstrates feature review and validation, without claiming an accuracy improvement or treating a split score as a complete audit.

Common misconceptions and mistakes

Gain ratio is information gain divided by the number of branches. The denominator is the entropy of branch proportions, which depends on sizes as well as count.
The highest ratio must always be selected without qualification. Standard C4.5 includes a gain eligibility safeguard, and implementations have additional rules.
Gain ratio eliminates overfitting. Pruning, stopping decisions, feature review and validation remain relevant.
A large score establishes causation. The criterion measures association in the training partition, not a causal effect.

Business use

Understanding the criterion helps teams interpret a tree's construction and identify why high-cardinality attributes need attention. It also prevents misleading comparisons between reported split scores and actual business outcomes.

Ask which algorithm and settings produced the tree, how numeric thresholds and missing values were handled, and what data was available at prediction time. Evaluate predictive performance using an appropriate held-out or cross-validation design. A transparent split calculation supports review, but the final model must still be assessed for its intended use.

Check your understanding

Answer: A test has information gain 0.24 bits and split information 1.2 bits.

What is its gain ratio, and does that alone determine the chosen split?

Answer: The ratio is 0.24/1.2 = 0.2. Selection also depends on competing tests and the algorithm's eligibility and other rules.

Related terms

Information gain
Entropy
Decision tree
Overfitting
Gini impurity
C4.5
split information
high-cardinality feature

Learn next

Modify the worked example so a candidate creates four equally sized pure branches. Calculate its split information and gain ratio, then explain why purity alone does not distinguish it from the two-branch split.

Sources

Quinlan's primary paper defines gain ratio, its average-gain safeguard and refinements for continuous attributes: https://arxiv.org/pdf/cs/9603103

The C4.5 verbose-output manual describes the implemented selection criterion: https://www2.cs.uregina.ca/~hamilton/courses/831/notes/ml/dtrees/c4.5/verbose.html

The eight-case arithmetic and Cedar Analytics case are original illustrations. They demonstrate the unadjusted categorical-test calculation and do not claim that all tree implementations use identical threshold or missing-value procedures.

Previous: Gain parameter Next: Gale - Church alignment algorithm