Is the subgradient a scalar or a vector?

It is a scalar in one dimension and a vector in a multivariable problem.

Can a smooth function have subgradients?

A differentiable convex function has exactly its ordinary gradient as the subgradient.

Does zero in the subdifferential solve a constrained problem?

The constraint must be incorporated appropriately, for example through a normal-cone condition or an extended objective; the unconstrained statement alone is insufficient.

AI Glossary / Mathematics and optimization / Intermediate

Subgradient

Definition

A subgradient is a vector that defines a supporting linear lower bound to a function at a specified point. For a convex function, it extends the useful role of a gradient to points where an ordinary derivative may not exist.

What it means

A smooth curve has a single tangent slope at a differentiable point. A convex curve can also have a corner, such as the bottom of the absolute-value function. At that corner, several slopes can support the curve from below.

Each such slope is a subgradient in one dimension. In several dimensions, a subgradient is a vector defining a supporting affine plane. The set of all subgradients at a point is called the subdifferential, written ∂f(x).

This entry uses the convex-analysis definition. Other generalized derivatives used for nonconvex functions have different definitions and properties. A software package's chosen derivative convention at a corner should therefore be interpreted in the context of the mathematical operation being optimized.

How it works

A vector g is a subgradient of f at x when f(y) ≥ f(x) + gᵀ(y − x) for every y in the relevant domain. The expression on the right touches the function at x and never rises above it elsewhere.

For a differentiable convex function, the gradient is its unique subgradient. At a nondifferentiable point there may be many. For f(x) = |x|, the subgradient is 1 when x is positive, −1 when x is negative and any number in [−1, 1] at zero.

The zero vector belongs to ∂f(x) exactly when x is a global minimizer under this unconstrained definition. Substituting g = 0 makes the supporting inequality say f(y) ≥ f(x) everywhere.

A nonzero subgradient can still exist at a minimizer. Selecting one and stepping in its negative direction need not reduce the objective. This is a key difference between subgradient methods and the intuitive picture of always following a downhill slope.

Three practical examples

An optimization model uses an absolute-value penalty to encourage small or zero coefficients.
A linear classifier is trained with a hinge-style objective that has a corner at its margin boundary.
A resource-allocation model minimizes the maximum of several affine cost estimates, producing a convex but possibly nonsmooth objective.

Think of it this way

Think of placing a flat ruler beneath a convex bowl-shaped outline so that it touches without cutting through the outline. At a sharp corner, several ruler angles may fit.

The analogy describes a global lower support. A subgradient is not merely a line that seems close near one point; it must satisfy the defining inequality throughout the domain.

Worked example

Checking possible slopes at a corner

Take f(x) = |x| at x = 0. The subgradient inequality becomes |y| ≥ gy for every real y.

If y is positive, dividing by y gives g ≤ 1. If y is negative, dividing reverses the inequality and gives g ≥ −1. Both conditions hold exactly when −1 ≤ g ≤ 1. Thus ∂f(0) = [−1, 1].

Choose g = 0.4. At y = 3, the proposed lower bound is 1.2 while f(3) = 3. At y = −2, the bound is −0.8 while f(−2) = 2. These checks illustrate the inequality; the sign-based argument above proves it for all y.

Now try g = 2. At y = 1, the proposed bound is 2, above f(1) = 1, so 2 is not a subgradient. Finally, g = 0 is valid and certifies that zero minimizes the function. Yet g = 1 is also valid there: a step from zero to −0.1 would increase the function from 0 to 0.1.

Illustrative case study

A fictional forecasting team adds an absolute-value penalty to a model. Its derivative-checking script reports a mismatch at exactly zero because it expects a unique smooth derivative.

The team checks the objective's convex structure and documents which valid subgradient its optimizer selects at the corner. It uses the full optimality condition, including the smooth loss term, rather than treating the penalty's zero point as automatically optimal for the combined objective.

The review also tests points on both sides of the corner. The lesson is that nondifferentiability is a mathematical property to handle explicitly, not necessarily a calculation failure. An arbitrary numerical slope is not acceptable merely because an ordinary derivative is unavailable.

Common misconceptions and mistakes

A subgradient is an approximate gradient. It satisfies an exact inequality under its definition.
Every point has one subgradient. A subdifferential can contain many vectors, and existence needs appropriate conditions.
A nonzero subgradient proves a point is not optimal. Other subgradients at that point may include zero.
A negative subgradient always gives descent. At a corner it may increase the objective.
Checking two sample points proves a global support. The inequality must hold for every point in the domain.

Business use

Subgradients help explain optimization with absolute penalties, maximum costs and piecewise-linear losses. They make it possible to reason about objectives whose corners are intentional features of the model.

For implementation reviews, identify the exact objective and its convexity assumptions. Check how nonsmooth terms are combined and whether the solver uses subgradients, proximal operations or another suitable mechanism. Avoid interpreting a generic “gradient near zero” diagnostic as a complete optimality certificate when the objective is nonsmooth. The relevant condition may involve a whole set of possible supporting vectors.

Check your understanding

Why does choosing g = 1 at x = 0 not contradict the claim that zero is a minimizer of |x|?

Answer: The subdifferential contains both 1 and 0. Optimality requires that zero be included, not that every subgradient equal zero.

Related terms

Gradient
Convex function
Supporting hyperplane
subdifferential
absolute-value penalty
optimality condition

Learn next

Read subgradient descent and proximal operator. Derive the subdifferential of max(0, x) at zero, then compare a selected subgradient step with the behavior expected from a smooth objective.

Sources

Boyd, Subgradient Methods, Stanford EE364b notes: https://web.stanford.edu/class/ee364b/lectures/subgrad_method_notes.pdf. Definition and non-descent behavior reviewed.

Aalto Dictionary of Machine Learning, subgradient: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Supporting inequality. The slope proof and fictional derivative-review case are original examples.

Previous: Subgoal independence Next: Subgradient descent