What does the method produce?

A small set of variables with fitted weights that together explain the outcome nearly as well as a much larger set.

Why one variable at a time?

Speed and simplicity: testing all combinations is impossible at scale, and greedy picks recover most of the value in practice.

What does orthogonal add?

After each pick, candidates are adjusted so already-explained influence is not counted again under another variable's name.

What is the main risk?

Greediness: pairs or groups of variables that work only together may be skipped, and noisy training data can be overfit by the picks.

When should it not be used?

When the truth is genuinely dense, with many small influences adding up, or when combinations matter more than individual variables; then shrinking methods suit better.

AI Glossary / AI and machine learning / Advanced

Orthogonal Matching Pursuit

Definition

Orthogonal Matching Pursuit is a method for building a sparse model: it selects the few input variables that matter most, one at a time, each time picking the variable that best explains what is still unexplained.

What it means

Sparse modeling starts from a suspicion that, among hundreds of candidate variables, only a handful truly drive the outcome. The method greedily finds them: pick the single most explanatory variable, account for it, then ask which remaining variable best explains the residue, and repeat.

The orthogonal part is bookkeeping with a purpose. After each pick, every candidate is re-angled so its overlap with what is already chosen is removed, which stops the method from picking the same influence twice under different disguises.

The result is a model with few terms, which is easy to read, cheap to run and often more trustworthy on new data than a model that uses everything. The cost is greediness: picking one at a time can miss combinations that only shine together.

For a business, it answers the question every stakeholder asks: of everything we measure, which few things actually move this number?

How it works

List the candidate variables and the outcome to explain, and compute how much each candidate alone correlates with the outcome.

Pick the strongest candidate, fit its weight, and subtract its contribution from the outcome, leaving a residue.

Remove the already-chosen influence from every remaining candidate, so correlations are measured against what is left to explain.

Repeat: pick the candidate that best explains the residue, fit, subtract, re-angle, until adding more stops paying.

Validate the final small model on held-out data, since greedy selection can overfit a noisy training set.

Stop by a rule chosen in advance, like a fixed count or a validation threshold, so the shortlist does not grow to fit the noise.

Three practical examples

A retailer with three hundred candidate demand drivers ends up with a seven-variable model that explains weekly sales nearly as well as the full set and fits on one slide.
A sensor team compresses vibration signals by representing each as a small combination of reference patterns, cutting storage by an order of magnitude.
A marketing analyst uses the greedy picks as a shortlist for deeper causal testing, rather than trusting the full correlation table.

Think of it this way

Think of explaining a recipe's flavor by naming ingredients one at a time, each time naming the ingredient that explains the most of the taste still unexplained, and stopping when the last few additions change nothing a diner would notice.

Worked example

the shortlist that held up

Suppose an insurer wants to explain claim costs from two hundred recorded variables. The figures are illustrative.

Orthogonal Matching Pursuit selects nine variables that together explain ninety per cent of what the full two hundred explained on training data.

On held-out policies, the nine-variable model keeps nearly all of that performance, while the full model loses noticeably more, because the small model had less room to memorize noise.

The general lesson: a greedy shortlist is not the theoretically best subset, but it is fast, readable and often the best trade under time pressure.

Illustrative case study

A telecoms analytics group used the method to compress network-quality models that previously consumed hundreds of engineered signals. The sparse versions ran on far less hardware, could be explained to regulators variable by variable, and exposed two influential signals that turned out to be data-entry artifacts, which the full model had quietly relied on. The team kept the greedy method in their toolkit with a written caveat: when two variables only work as a pair, one-at-a-time selection can walk past both, so domain review of the shortlist is part of the process, not an optional extra.

Common misconceptions and mistakes

The greedy picks are the true causes. They are the strongest correlates found one at a time; causation needs separate evidence.
Orthogonal means the variables themselves are uncorrelated. It means the method re-angles candidates during selection to avoid double-counting influence.
Sparse is always better. Sparse is better for reading and often for generalization, but when many weak signals genuinely combine, a small model underfits.

Business use

Use sparse selection to turn an unreadable wide model into a shortlist people can review, challenge and govern.

Treat the selected variables as hypotheses for investment in better data, not as proven drivers.

Prefer the sparse model when the audience must understand the reasoning, like regulators, boards or customers.

Keep the full model's performance as a benchmark, so the cost of simplicity is visible in numbers.

Record why each shortlisted variable was accepted, so the next analyst inherits reasoning, not just a list.

Check your understanding

What does Orthogonal Matching Pursuit give a business, and what is its blind spot?

Answer: It gives a short, readable list of the variables that carry most of the explanatory weight, built quickly from a large candidate pool, which supports explanation, governance and cheap deployment. Its blind spot is greedy selection: influences that only appear in combination can be missed, so the shortlist deserves domain review before it drives decisions.

Related terms

Learn next

Be Productive Using AI covers how to question a model's chosen drivers before acting on them: https://aiforbusinessandlife.com/academy/be-productive-using-ai [C1]

Sources

[S24] scikit-learn documentation, Glossary of Common Terms and API Elements; [S11] Murphy, Probabilistic Machine Learning: An Introduction. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Orthogonal contrasts Next: Orthogonality condition