Is a random forest an ensemble?

Yes, of decision trees; forests and boosting are the most common ensembles in practice.

Do ensemble members need to be different algorithms?

No; different data samples or settings often suffice, though algorithm diversity helps.

Can ensembles overfit?

Much less easily than single models, but weighted or stacked combiners can overfit if tuned carelessly.

What is a stacked ensemble?

One where a small meta-model learns how to combine the members' outputs, rather than using a plain vote or average.

AI Glossary / AI and machine learning / Beginner

Ensemble

Definition

An ensemble is a model made by combining several models, so that their collective answer is better than any of them alone.

What it means

Single models have single points of failure: each has its own blind spots, favorite mistakes and quirks inherited from its training data. An ensemble hedges by asking several models and combining.

The combination is usually simple: a vote for categories, an average for numbers. The sophistication lies in making the members different, because an ensemble of identical models is one expensive model.

Diversity comes from different training samples, different algorithms, different input fields or different random starts. Anything that makes members err in different directions lets the errors cancel.

Ensembles dominate practical machine learning for a reason: decision forests, boosting machines and most competition winners are ensembles wearing different names.

For a business, ensemble thinking is also a management idea: a committee of diverse forecasters outperforms the best single forecaster, and the model version is the same arithmetic.

There is a quiet governance benefit as well: an ensemble's members can be audited separately, and when the committee's answer shifts you can ask which member moved, something a single monolith never lets you do.

How it works

Train several models with deliberate differences: different data samples, algorithms, feature sets or settings.

Check that members actually disagree: measure how often their errors coincide, because correlated errors cancel nothing.

Combine predictions by majority vote for categories or averaging for numbers, sometimes with weights reflecting member quality.

Evaluate the ensemble, not just the members, on held-out data; the combination can behave differently from any part.

Mind the operating cost: ten models mean ten predictions per case, so production ensembles balance accuracy against latency and maintenance.

Version the ensemble as a unit: the members, their weights and the combining rule together are the deployed asset, and changing any member without revalidating the whole is a silent model change.

Three practical examples

Decision forests: hundreds of trees vote, the canonical ensemble.
Forecasting: averaging a statistical model, a machine learning model and a judgmental estimate beats each alone.
Fraud screening: a rules engine, a neural network and an anomaly scorer vote, with disagreements routed to review.

Think of it this way

Think of estimating the weight of a cow at a fair. One farmer guesses badly; the average of a hundred guesses is famously close. Each guesser errs differently, and the average erases the individual errors.

The limit: if every guesser read the same wrong rumor, the average inherits it. Shared information is shared error, which is why diversity is the whole game.

Worked example

combining three demand forecasts

Suppose a chain forecasts demand three ways: a statistical baseline, a machine learning model and the category manager's judgment. The figures are illustrative.

Alone, their monthly errors average 14, 12 and 15 per cent. Averaged together, the error falls to 9 per cent, because their bad months rarely coincide.

The ensemble also provides a disagreement signal: when the three spread widely, the planner investigates before committing stock.

The chain keeps all three models running, because the diversity, not any single model, is the asset.

Illustrative case study

A fictional lender replaced its committee of three modest models with one giant model of higher test accuracy, and discovered in production that the giant had a systematic blind spot for a whole customer segment that one of the modest models used to catch.

Restoring an ensemble, giant plus the segment-savvy veteran, recovered the coverage.

The invented lesson: the best single member is not the best team. Test the team.

A second invented effect: after the blind spot incident, the lender instituted a rule that no single model, however accurate, may serve a regulated decision alone; every committee must contain at least one member built on different assumptions.

Common misconceptions and mistakes

Ensembles must be complicated. Vote and average are the standard combiners; the cleverness is in the diversity, not the arithmetic.
More members always help. Members help while they add diversity; the fortieth correlated model adds cost and no accuracy.
Ensembles cannot be explained. Individual members and their disagreements can be examined; explanation is harder than for one tree but far from impossible.

Business use

Default to ensemble methods for table-shaped prediction problems; they are strong, stable and cheap to try.

Use member disagreement as an operational signal: cases where members split are cases for human review.

Track the ensemble's production cost honestly; every member is a model to monitor, retrain and version.

When replacing an ensemble with a single stronger model, run them in parallel for a full cycle first; the ensemble's coverage of odd cases only becomes visible when it is gone.

Check your understanding

Why does an ensemble beat its best single member only when members make different errors?

Answer: Because combining cancels errors that differ across members; identical errors survive the vote unchanged, so diversity is what the combination spends.

Related terms

Learn next

AI for Business Owners explains how to combine several imperfect forecasts into a decision you can defend: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S2] Google, Machine Learning Glossary; [S3] NIST AI Resource Center glossary; [S12] Aalto University, Dictionary of Machine Learning. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: English auction Next: Ensemble Kalman filter