What is a local optimum?

A solution no single small change improves, which may still be far worse than the best solution available elsewhere in the space.

How do random restarts help?

Different starting points lead to different peaks, and keeping the best across runs raises the odds of a genuinely good answer.

Is hill climbing used in machine learning?

Yes, inside many fitting and tuning procedures, though modern variants add randomness and momentum to escape local traps.

AI Glossary / AI and machine learning / Beginner

Hill climbing

Definition

Hill climbing is a search method that improves a solution by repeatedly making the small change that helps most right now, stopping when no small change helps, like walking uphill in fog until every direction leads down.

What it means

Faced with a hard problem, schedule, route, layout, configuration, hill climbing starts anywhere and asks one question over and over: what single tweak makes this better? Take the best tweak, ask again. Stop when nothing nearby is better.

The method's strength is its simplicity: it needs no map of the whole problem, only a way to score the current solution and a notion of small change. That makes it cheap to try on almost anything.

The weakness is written into the metaphor: in fog, you cannot tell a local hilltop from the mountain. Hill climbing stops at the first peak where every step leads down, which may be far below the best available answer.

Practical versions fight the fog: restart from many random points and keep the best peak, or occasionally accept a downhill step to escape a local trap. Each refinement costs compute and buys altitude.

For a business, hill climbing is the mental model behind most tuning: adjust one thing, measure, keep what helps. It works, it is honest about its limits, and knowing the local-peak trap is what separates disciplined tuning from superstition.

How it works

Define the solution space, the scoring function and what counts as a small change, the neighborhood.

Start from any candidate solution and score it.

Generate the neighbors, score them, and move to the best one if it beats the current solution.

Repeat until no neighbor is better; the current solution is a local optimum.

Restart from different starting points, or add controlled randomness, and keep the best peak found across all runs.

Three practical examples

A delivery firm tweaking routes one swap at a time, keeping each swap that shortens total distance, until no single swap helps.
A marketing team tuning email send times hour by hour, settling at the hour where neighbors in both directions perform worse.
A data team adjusting model settings one at a time, accepting the risk that a pair of settings changed together might have helped more.

Think of it this way

Think of climbing a hill in thick fog with an altimeter. You cannot see the summit, but you can feel which way is up, so you keep stepping uphill. When every direction is down, you stop, and you may be on the mountain's peak or on a pimple beside it.

The limit: the fog never lifts by climbing harder. Only starting elsewhere, or occasionally stepping down to explore, tells you whether your hilltop was the real one, and even then there are no guarantees, just better odds.

Worked example

escaping a scheduling plateau

Suppose a clinic schedules staff by hill climbing: swap two shifts, keep the swap if coverage improves, repeat. The figures are illustrative.

The first run stalls with 11 per cent of slots under-covered, every single swap making things worse, a clear local peak.

Ten random restarts produce peaks ranging from 11 per cent down to 4 per cent, proving the first answer was a pimple, not the mountain.

The clinic adopts the 4 per cent schedule and the restart habit: any future tuning problem gets multiple starts as standard.

Illustrative case study

A fictional logistics firm tuned its loading configuration by single tweaks for two years, each quarter's small improvement celebrated, until a new hire ran a fresh search from scratch and beat the tuned configuration by 18 per cent on the first day.

Two years of hill climbing had polished one hilltop while the mountain stood unvisited, and every quarterly review had mistaken motion for progress.

The invented lesson: local improvement feels like winning. Restart the search before you frame the certificate.

Common misconceptions and mistakes

Hill climbing finds the best solution. It finds a locally best solution; global quality depends on the starting point and the landscape, with no certificate of optimality.
It is an obsolete toy. The method sits inside many production optimizers, usually with restarts and randomized escapes layered on top.
If no single change helps, nothing will. Combinations of changes can improve what no single change can; the method is blind to them by design.

Business use

Use hill climbing for quick tuning wins, but budget for restarts whenever the answer carries real money.

Treat any long-tuned configuration as a suspect local peak; an occasional fresh-start search is cheap insurance.

Change one thing at a time in business experiments for the same reason the method does: you learn what helped, even though you accept the local-peak risk.

Check your understanding

Why does hill climbing stop before it finds the best possible solution?

Answer: Because it can only see immediate neighbors, so it halts at the first peak where every small step goes downhill, and that peak may be far below the true summit.

Related terms

Optimization
Search
Gradient descent
local optimum
random restart
tuning

Learn next

Be Productive Using AI covers disciplined trial-and-measure habits for tuning anything, one change at a time: https://aiforbusinessandlife.com/academy/be-productive-using-ai. [C1]

Sources

[S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Hilbert space Next: Hindsight optimization