Does convex mean U-shaped?

No. Affine functions and absolute-value functions are also examples.

Can several inputs minimize a convex function?

Yes. Strict convexity rules out multiple minimizers when a minimizer exists.

Does a convex loss make every neural-network training objective convex?

No. The way predictions depend on parameters must also be considered.

AI Glossary / Mathematics and optimization / Advanced

Convex function

Definition

A convex function is a function on a convex domain whose value at any weighted average of two inputs is no greater than the corresponding weighted average of its values. Formally, f(tx + (1 − t)y) ≤ tf(x) + (1 − t)f(y) for all domain points x and y and every t between zero and one. Geometrically, each connecting chord lies on or above the graph.

What it means

Convexity describes the whole function on its stated domain, not just the shape near one point. A bowl-shaped curve is a familiar example, but straight lines and functions with corners can also be convex. The domain matters because the weighted-average inputs must remain valid.
For minimization over a convex domain, any local minimum of a convex function is global. This removes the problem of being trapped in an inferior local minimum. It does not guarantee that a minimum is attained, that it is unique, or that any chosen numerical procedure will find it reliably.

How it works

One can establish convexity directly from the defining inequality or through appropriate mathematical tests. For a twice differentiable function on an open convex domain, a positive semidefinite Hessian everywhere characterizes convexity. In one dimension, this reduces to a nonnegative second derivative. Functions that are not differentiable require other arguments.
Strict convexity replaces the inequality with a strict one for distinct inputs and weights strictly between zero and one. A strictly convex function has at most one minimizer. “At most” matters: existence is a separate issue. Nonnegative weighted sums of convex functions are convex, provided they share an appropriate convex domain.

Three practical examples

A forecasting team fits a linear model using squared errors. Each residual is affine in the coefficients, making its square convex in those coefficients. This statement concerns the training objective, not whether the forecasts will be accurate on future data.
A planner uses absolute deviation from a target as a penalty. The function has a corner at the target but remains convex, showing that smoothness is not part of the definition.
An analyst adds nonnegative multiples of convex penalties to a convex objective. This preserves convexity. A negative weight does not enjoy that general guarantee, so the direction and sign of each added term matter.

Think of it this way

Imagine stretching a straight string between two points on a curve. For a convex function, the graph between those points never rises above that string. One successful pair is not enough: the condition must hold for every pair and every intermediate proportion within the domain.

Worked example

Take the original function f(x) = (x − 3)² + 2 on the real line. Select x = 0, y = 4 and t = 1/4. Their weighted average is 3. The function value there is f(3) = 2, whereas the weighted average of endpoint values is (1/4)f(0) + (3/4)f(4) = 11/4 + 9/4 = 5. Thus this comparison satisfies 2 ≤ 5.
A single comparison illustrates rather than proves convexity. For this quadratic, expansion gives tf(x) + (1 − t)f(y) − f(tx + (1 − t)y) = t(1 − t)(x − y)². This is nonnegative for every valid choice, and positive for distinct x and y with an interior weight. The function is therefore strictly convex.
Its unique minimizer is x = 3 with value two, because the squared term cannot be negative. Contrast the constant function g(x) = 2: it is convex but every real input minimizes it. Also contrast h(x) = exp(x): it is strictly convex but never attains its infimum of zero on the real line.

Illustrative case study

In a fictional calibration project, a team minimizes the preceding quadratic through gradient descent. Its derivative is 2(x − 3), so an update with step size η is x_new = x − 2η(x − 3).
Starting at zero with η = 1/4 gives 0, 1.5, 2.25 and 2.625, approaching three. With η = 1, the sequence instead alternates between zero and six. Both points have objective eleven, and the process never reaches the minimizer. With η = 5/4, the error is multiplied by −1.5 at each step and grows in magnitude.
The team concludes that the objective's favorable geometry does not excuse an unsuitable update rule. It checks the step-size policy separately from the proof of convexity. These are exact illustrative calculations, not reported performance from a deployed calibration system.

Common misconceptions and mistakes

A convex function need not have a unique minimizer, and even strict convexity does not guarantee that a minimizer exists. A straight line can be both convex and concave. A function with a corner can be convex, so demanding a second derivative would exclude valid examples.
Another mistake is confusing convexity in a prediction with convexity in model parameters. Squared error is convex in a scalar prediction. If that prediction is w² and the target is one, the parameter objective becomes (w² − 1)², whose second derivative is negative at zero. The nonlinear parameterization has changed the geometry.

Business use

Recognizing convex objectives helps teams choose methods with useful optimization guarantees and explain their limitations. Ask which variables the convexity claim concerns and whether constraints preserve a convex feasible region. Keep optimization quality separate from model validity: minimizing a poorly chosen error measure precisely can still produce an unhelpful decision. Numerical stopping rules, scaling and data quality also remain relevant even when convexity is established.

Check your understanding

What proves convexity of the worked quadratic for all inputs?

Answer: The chord gap t(1 − t)(x − y)² is always nonnegative.

Why does exp(x) contradict an unconditional existence claim?

Answer: It approaches zero as x decreases without ever reaching zero.

Why can gradient descent oscillate on a strictly convex function?

Answer: An unsuitable step size can prevent the updates from converging.

Related terms

Convex set
Hessian
Global minimum
Convex optimization
strict convexity
strong convexity
concave function
Jensen's inequality

Learn next

Compare convex, strictly convex and strongly convex functions. Then study subgradients for functions with corners and convergence conditions for gradient-based optimization methods.

Sources

Google Machine Learning Glossary, convex function: geometric interpretation and common machine-learning examples. https://developers.google.com/machine-learning/glossary#convex-function Stephen Boyd and Lieven Vandenberghe, Convex Optimization slides, chapter 3: defining inequalities, strict convexity and derivative-based conditions. https://web.stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf The quadratic proof, existence counterexamples and fictional calibration case are original teaching illustrations. Existence of a minimizer is stated separately from uniqueness.

Previous: Convex clustering Next: Convex hull