What is the difference from L2?

L1 pushes weights to exactly zero, selecting fields; L2 shrinks weights toward zero without evicting, keeping every field at reduced size.

The classical name for L1-penalised regression, the setting where the selection effect was first made famous.

How is the strength chosen?

By validation across a range of values, picking the point where held-out accuracy and simplicity balance as the business prefers.

AI Glossary / AI and machine learning / Intermediate

L 1 regularization

Definition

L1 regularization is a penalty added to training that charges the model for the size of its weights, with the distinctive effect of pushing unneeded weights all the way to zero, performing automatic feature selection.

What it means

Every weight in a model is a claim that this field matters. L1 charges rent on each claim: keep the weight only if it earns its cost in reduced error, otherwise it is driven to exactly zero and the field leaves the model.

The result is sparsity. Where other penalties shrink weights toward zero without reaching it, L1 evicts: a model trained under L1 typically keeps a fraction of its candidate fields, having quietly done feature selection as a side effect of fitting.

That side effect is often the point. In a problem with ten thousand candidate fields and fifty that matter, L1 finds the fifty, giving a smaller, faster, more interpretable model in one training run.

The penalty has a strength dial, and it is a real trade: too little and nothing is selected, too much and genuinely useful fields are evicted along with the noise, so the strength is tuned on validation data like any other dial.

For a business, L1 is the tidy-house option: when the field list is bloated and nobody knows which columns earn their keep, L1's evictions are a defensible first answer, and the survivors list is a finding in itself.

How it works

Assemble the candidate fields, then add the L1 penalty to the training objective with a chosen strength.

Train: the optimizer balances fit against the penalty, driving weak fields' weights to zero.

Inspect the survivors: which fields kept non-zero weights, at what sizes, with what signs.

Tune the penalty strength by validation across a range, tracing how the survivor count and accuracy move together.

Refit the final model at the chosen strength, and record the survivor list as part of the model's documentation.

Three practical examples

A text classifier under L1 keeping 800 of 40,000 word features, with the survivor list reviewed for sense by a human.
A pricing model whose L1 fit evicts 60 of 75 candidate fields, ending a two-year argument about which ones mattered.
A genomics-style screen where L1 isolates a handful of markers from thousands of candidates for follow-up.

Think of it this way

Think of a moving company charging per box. Every possession is re-examined: is this worth shipping? The furniture survives, the broken treadmill does not, and you arrive with a smaller, better household. L1 is the per-box charge applied to model weights.

The limit: the charge knows nothing about meaning. Two fields that share the work may each look dispensable, and the eviction of a field you know matters is a sign to look at correlated rivals, not to overrule the arithmetic blindly.

Worked example

the survivors list

Suppose a bank's scorecard team feeds 300 candidate fields into an L1-penalised logistic regression. The figures are illustrative.

The fitted model keeps 41 fields, and the survivor list contains every field the senior underwriters expected plus four surprises.

Two surprises prove to be leakage on inspection and are removed; two are genuine, previously ignored signals.

The final 39-field model validates as well as the 300-field unpenalized version, at a tenth of the monitoring burden.

Illustrative case study

A fictional marketing team dialled the L1 strength up chasing simplicity, and the model obliged by evicting everything but the two strongest fields, delivering a beautifully simple model with mediocre accuracy.

A gentler penalty kept fifteen fields and recovered the performance; the team had briefly optimized for tidiness over truth.

The invented lesson: sparsity is a means, not the goal. The strength dial trades simplicity against accuracy, and the trade belongs on validation data, not on aesthetics.

Common misconceptions and mistakes

L1 deletes fields, so it loses information. It deletes fields whose predictive rent was not being earned; the information that mattered survives in the kept weights.
It is only for linear models. The penalty applies wherever weights exist, though the eviction effect is most famous in linear and logistic models.
Zeroed fields were useless. They were redundant given the kept fields, which is different: a zeroed field may predict well alone and still lose its seat to a stronger rival.

Business use

Reach for L1 when the candidate field list is large and the survivor list itself is valuable knowledge.

Tune the penalty strength on validation data and report the accuracy-sparsity trade to stakeholders explicitly.

Review zeroed fields of known business meaning: their eviction usually signals a stronger correlated field, worth understanding.

Check your understanding

Why does L1 regularization perform feature selection while other penalties do not?

Answer: Because its penalty grows with the absolute size of each weight, so the cheapest way to reduce the penalty is to drive weak weights exactly to zero, evicting their fields from the model.

Related terms

Learn next

AI for Business Owners explains how to read a feature list and ask which fields are earning their place: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: L 1 loss Next: L 2 loss