What is ridge regression?

Linear regression with an L2 penalty, the classical setting where the shrinkage effect was first formalized.

What is weight decay?

The deep-learning name for L2: weights are decayed toward zero at every update, the same penalty in different clothing.

How do I choose between L1 and L2?

Choose L1 when you want field selection and a compact survivor list, L2 when you want all fields kept but restrained, or blend both.

AI Glossary / AI and machine learning / Intermediate

L 2 regularization

Definition

L2 regularization is a penalty added to training that charges the model for the square of its weights, shrinking all weights toward zero to keep any single field from dominating, without evicting fields entirely.

What it means

Large weights are how overfitting shows up: the model leans hard on a few fields to nail the training data, and the leaning breaks on new cases. L2 taxes bigness itself, so training prefers many small contributions over a few large ones.

The effect is shrinkage, not selection: every weight is pulled toward zero proportionally, but none is driven exactly to it. All fields stay in the model, each speaking more softly, which stabilizes the model and usually improves held-out performance.

L2 is the default regularizer across machine learning, present under names like ridge, weight decay and lambda in nearly every toolkit, because it is cheap, smooth and almost always worth having at some strength.

The strength dial matters as with L1: too weak and memorization proceeds untaxed, too strong and the model is all whisper, underfitting because no field may speak loudly enough to be heard.

For a business, L2 is the seatbelt of modeling: rarely the reason a project succeeds, often the reason it does not crash, and the dial setting belongs in every model's documentation alongside its justification.

How it works

Add the squared-weight penalty to the training objective at a chosen strength, the lambda or weight-decay setting.

Train as usual: the optimizer now balances fitting the data against keeping weights small.

Tune the strength on validation data across orders of magnitude, since the useful range is wide and problem-dependent.

Inspect the resulting weight sizes: a healthy L2 fit shows many moderate weights rather than a few extreme ones.

Record the chosen strength and the curve that justified it, because the setting is part of the model, not an implementation detail.

Three practical examples

A demand model whose wild swings on promotions are tamed once L2 strength rises, because no single promotion field may dominate.
A text classifier under weight decay generalizing better with 40,000 small weights than it did with 4,000 enormous ones.
A deep network whose only regularization is weight decay, holding the overfitting gap to two points on a well-fed problem.

Think of it this way

Think of a panel discussion with a strict chair: every voice gets heard, but nobody may shout. The extreme opinions soften into contributions, and the group's conclusion is steadier for it. L2 is the chair, taxing loudness so the committee decides.

The limit: a chair who allows no emphasis at all produces mush. Too strong a penalty silences the fields that genuinely deserve the floor, which is underfitting with good manners.

Worked example

finding the dial

Suppose a retailer tunes L2 strength for a sales model across six orders of magnitude. The figures are illustrative.

Held-out error traces a U: high at both extremes, bottoming at a strength of 0.01, where the gap between training and test error closes from 9 points to 2.

The surviving model's largest weight is a tenth of the unpenalized version's, and its live forecast is noticeably calmer.

The curve goes into the model file: one page showing why 0.01, so next year's retuning starts from evidence.

Illustrative case study

A fictional bank's model team, burned by an overfitting incident, cranked L2 to extreme strength and shipped a model so smoothed it barely separated good loans from bad.

Losses rose at both tails, the model approving too freely at the top and declining too harshly at the bottom, until validation found the middle setting the incident had skipped.

The invented lesson: regularization is a dial, not a direction. More is not safer; right is safer, and right is found on validation data.

Common misconceptions and mistakes

L2 removes useless features. It shrinks everything and removes nothing; L1 is the one that evicts.
It is a deep-learning trick. It predates deep learning by decades as ridge regression, and it lives in nearly every classical toolkit too.
The strongest safe setting is best. Strength trades fit against stability on a curve with a best point; both extremes lose.
L2 makes models slower. The penalty is cheap at training time and changes nothing at prediction time; the model runs exactly as fast, just calmer.

Business use

Default to some L2 in every model fit, and tune the strength rather than accepting the toolkit's default.

Report the validation curve across strengths in model reviews, so the setting is a decision with evidence.

Compare L1 when the survivor list itself matters, L2 when all fields should stay but behave; the choice is a business preference as much as a technical one.

Check your understanding

Why does L2 regularization improve held-out performance without removing any fields?

Answer: Because taxing large weights forces the model to spread credit across many fields in smaller, stabler contributions, which fits the noise less and the pattern better.

Related terms

Learn next

AI for Business Owners covers how to read the evidence behind a model's settings instead of taking them on faith: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: L 2 loss Next: L1-based feature selection