How do I recognize overfitting?

By the gap: strong performance on training data and noticeably weaker performance on data the model never saw.

A model flexible enough to memorize, trained too long, on data too small or noisy to force genuine learning.

More data, simpler models, complexity penalties, pruning, early stopping and honest held-out evaluation.

Is it only a technical problem?

No: it is a governance problem, because overfit scores are how bad models get funded, launched and trusted.

Can big models avoid overfitting?

Yes, with enough data, regularization and early stopping; size creates the appetite, but discipline decides whether memorizing wins.

AI Glossary / AI and machine learning / Beginner

Overfitting

Definition

Overfitting is the failure where a model learns its training data too well, memorizing its noise and accidents along with its real patterns, so it performs brilliantly on data it has seen and poorly on anything new.

What it means

Learning always balances two goals: fitting the known data and staying simple enough to fit unseen data. Overfitting is what happens when the first goal wins outright. The model becomes a detailed memoir of the training set instead of a usable summary of the world.

The signature is a gap: training scores keep improving while scores on fresh data stall or degrade. A model that reports ninety-nine per cent on its own homework and sixty on the exam is overfit, and the homework score is the lie that must not be quoted.

The causes are capacity and time: a model flexible enough to memorize, trained long enough to do it, on data small or noisy enough that memorizing is easier than learning. The cures are the opposites: more data, simpler models, penalties on complexity, and early stopping.

For a business, overfitting is the reason the demo dazzles and the deployment disappoints, and the reason every claimed accuracy must be asked one question: on data the model never saw?

How it works

Split data before training, keeping a test set locked away so final scores are measured on unseen cases.

Watch the gap between training and validation performance during training; a widening gap is overfitting arriving.

Prefer simpler models when data is limited, since complexity is the appetite that overfitting feeds.

Penalize complexity through regularization, prune trees, and stop training when validation performance peaks.

Gather more data where the model fails, since fresh examples are the most honest cure for memorization.

Three practical examples

A demand model predicts last year's sales almost perfectly but misses next quarter badly, because it learned one unusual promotional calendar by heart.
A resume-screening model aces historical hiring data but ranks new applicants oddly, having memorized quirks of past reviewers rather than real signals.
A tiny pilot of fifty customers produces a beautiful model that collapses on the full base, having learned fifty individual stories instead of any pattern.

Think of it this way

Think of a student who memorizes last year's exam paper, answers and all. They score perfectly on that paper and are lost on this year's. The training set is last year's paper.

Worked example

the perfect backtest

Suppose a trading model shows a ninety per cent win rate on five years of historical data. The figures are illustrative.

Testing reveals the model was tuned through dozens of iterations on that same history, so its score measures memory of the past, not skill at the future.

The fix is discipline: tune on one period, validate on a later untouched period, and quote only the untouched number, which turns out to be a modest fifty-five per cent.

The general lesson: any score measured on data used to build the model is an advertisement; only scores on held-out data are evidence.

Illustrative case study

A property-tech startup pitched investors a valuation model with three per cent error, measured, on inspection, on the very sales used to fit it. A technical advisor insisted on a temporal split: train on older sales, test on newer. Error tripled, still respectable, and the honest number changed the funding conversation from magic to margin. The startup survived because the gap was found in diligence, not in production. The practice that stuck: every model card in the company now lists not just the score but exactly which data produced it, and any number without a held-out pedigree is treated as unmeasured.

Common misconceptions and mistakes

A complex model is always overfit. Complexity only creates the capacity; overfitting is confirmed by the training-versus-new-data gap, not by the model's size.
More training always helps. Past the validation peak, more training is precisely the mechanism of overfitting.
Overfitting means the model is useless. It means its quoted scores are inflated; the model may still be good, but only held-out testing can say how good.

Business use

Never accept a performance number without asking which data produced it; training-set scores are advertisements, held-out scores are evidence.

Insist on a locked test set or a time-based split before any model goes live, and re-lock it after every tuning round.

Treat the training-versus-validation gap as a headline metric in project reviews, not a footnote.

Budget for data collection in weak segments, because fresh examples cure overfitting more honestly than clever algorithms.

Ask for the gap itself: a report showing training and held-out scores side by side tells you more than either number alone.

Check your understanding

What is overfitting, in one sentence, and what is its unmistakable sign?

Answer: Overfitting is a model memorizing its training data, noise included, instead of learning patterns that transfer. Its unmistakable sign is the gap between performance on data the model trained on and performance on fresh data: a wide gap means the quoted score measures memory, and only the fresh-data number should be believed.

Related terms

Learn next

Be Productive Using AI gives you the question that punctures inflated claims: on data the model never saw? https://aiforbusinessandlife.com/academy/be-productive-using-ai [C1]

Sources

[S12] Aalto University, Dictionary of Machine Learning; [S2] Google, Machine Learning Glossary; [S14] KNIME, Data Science Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Outstar Next: Overlap - add method