Is an empirical gradient always approximate?

In this local-probing sense it is an estimate, although special functions can make a finite-difference formula exact in exact arithmetic.

How many evaluations are needed?

A basic central-difference estimate in d coordinates uses two probes per coordinate; other constructions have different costs.

Does a zero estimate prove optimality?

No. It can reflect a stationary point, a plateau, inadequate perturbation, numerical effects or noise, and it does not establish global optimality.

AI Glossary / Mathematics and optimization / Intermediate

Empirical gradient

Definition

In the local-search terminology used by Artificial Intelligence: A Modern Approach, an empirical gradient is an estimate of how an objective changes, obtained by evaluating nearby settings rather than deriving its gradient analytically. Small increases and decreases in coordinates reveal promising local directions. This entry follows that source meaning, which differs from the separate statistical usage meaning the gradient of an empirical, sample-based loss.

What it means

Suppose a simulator returns a cost for each proposed configuration but does not expose a differentiable formula. You can still ask what happens when one setting is changed slightly. Repeating this for the coordinates provides local information that can guide a search. It does not create an exact derivative where none exists, nor does it reveal the entire objective landscape. The estimates describe the behavior around the tested point and at the chosen perturbation scale.

How it works

One common numerical construction approximates coordinate j of the gradient by [f(x + h ej) − f(x − h ej)]/(2h), where ej changes only that coordinate and h is a small positive increment. A forward difference instead compares f(x + h ej) with f(x). A search can move against the estimated gradient for minimization or with it for maximization, then evaluate the proposed point. Alternatively, a discretized hill-climbing method can directly compare the neighboring candidates. The perturbation size used to estimate change is distinct from the step size used to move the search point.

Three practical examples

A simulation study can estimate how operating cost changes when one continuous control setting is perturbed slightly.
A software engineer can compare a numerical gradient with an implemented analytic derivative on a small, deterministic test problem.
A policy-search experiment can compare nearby parameter settings using evaluated returns, while accounting for the variability of those evaluations rather than assuming every observed change reflects a true improvement.

Think of it this way

Think of testing a dimmer switch by moving it a little in either direction and observing how much the room brightness changes. The experiment estimates local responsiveness without knowing the electrical formula. The analogy is bounded: a multivariable objective can have interactions, noisy measurements and constraints, and changing a parameter is not always as harmless or inexpensive as turning a dial.

Worked example

Use the deterministic objective f(x, y) = (x − 2)² + 3(y + 1)² and start at (0, 0), where f = 7. With central-difference increment h = 0.1, the x probes give f(0.1, 0) = 6.61 and f(−0.1, 0) = 7.41. Their difference divided by 0.2 is −4. The y probes give f(0, 0.1) = 7.63 and f(0, −0.1) = 6.43, producing 6. Thus the estimated gradient is (−4, 6). For this quadratic and exact arithmetic, the central differences equal the analytic gradient at the starting point. A minimization step with learning rate 0.1 gives (0, 0) − 0.1(−4, 6) = (0.4, −0.6). Its objective is 1.6² + 3(0.4²) = 3.04, an improvement over 7. But using step size 1 gives (4, −6), where the cost is 4 + 75 = 79. A correct local gradient does not justify an arbitrarily large move. The equality between central difference and derivative here is a special quadratic property, not a promise for general functions.

Illustrative case study

In a fictional project, Linden Simulation estimates a local gradient for a two-setting equipment model. Analysts repeat evaluations to determine whether observed differences exceed simulation noise, then check the objective after each proposed move. Near a feasibility boundary, they revise the probing rule so it does not request invalid settings. They record evaluation budgets and unsuccessful steps as well as improvements. The case illustrates a controlled optimization experiment, not a claim that a fictional real-world system was improved or safely commissioned.

Common misconceptions and mistakes

Making h extremely small is not automatically better: subtraction of nearly equal computed values can amplify rounding effects, and measurement noise can dominate the difference. A large h can instead reflect curvature or behavior beyond the intended neighborhood. Finite differences and automatic differentiation are different procedures. A non-smooth objective may not have an ordinary gradient at the tested point. Also, a gradient of a training-set average can be computed analytically or by automatic differentiation; calling that an empirical gradient in another paper does not mean the paper used coordinate perturbations.

Business use

Use numerical probing when the objective can be evaluated and the evaluation budget supports it. Record coordinate scales, perturbation sizes, constraints and repeatability. Test more than one perturbation scale and check whether the resulting direction is stable enough for its intended use. For expensive or noisy evaluations, consider methods designed for that setting rather than assuming a basic finite-difference formula will be reliable. Keep estimation and search-step controls separate. Report a local improvement as local evidence, and avoid claiming a global optimum solely because nearby probes fail to improve the objective.

Check your understanding

Why did the worked search become worse with step size 1 despite using the correct gradient?

Answer: The gradient describes local change. A large move can overshoot the region where following that direction decreases the objective.

Related terms

Gradient
Hill climbing
Step size
Empirical risk
finite differences
numerical differentiation

Learn next

repeat the worked estimate using forward differences and compare it with (−4, 6). Then add a small artificial perturbation to each returned function value and examine how dividing by different h values affects the estimate. Keep the probing increment separate from the optimization step.

Sources

Sources: Russell and Norvig, Artificial Intelligence: A Modern Approach, publisher-hosted sample, Section 4.2, https://www.pearsonhighered.com/assets/samplechapter/0/1/3/6/0136042597.pdf ; SciPy, approx_fprime, https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.approx_fprime.html . The local-search usage is grounded in the publisher text; the quadratic calculations and fictional simulation case are original.

Previous: Empirical dynamic modeling Next: Empirical loss