Does every fixed-point characterization use gradients?

No. Proximal and other operators can characterize problems involving nonsmooth functions or additional structure.

Can a characterization be correct while the algorithm fails?

Yes. Correct solutions can be fixed points of an iteration that does not converge from the chosen start.

Why does convexity matter in the gradient example?

It makes the zero-gradient condition sufficient for a global minimum, rather than only a stationary-point condition.

AI Glossary / Mathematics and optimization / Advanced

Fixed-point characterization

Definition

A fixed-point characterization expresses the solutions of a problem as exactly the points left unchanged by a suitable mapping. In optimization, it establishes a relation such as: x solves the optimization problem if and only if T(x) = x. The characterization connects a goal, such as minimizing a loss, with an operator. It is a mathematical equivalence, not yet a promise that repeatedly applying the operator will find a solution.

What it means

This distinction is useful because many algorithms are easier to describe as repeated updates than as direct solutions of an equation. A characterization explains why an unchanged state is relevant to the original problem. It should establish both directions: every intended solution is fixed, and every fixed point is an intended solution. If only one direction holds, the operator may also have unwanted fixed points. The required conditions depend on the problem, including whether the objective is convex, whether constraints apply and whether derivatives exist.

How it works

For an unconstrained differentiable convex function f on Euclidean space, a point minimizes f exactly when its gradient is zero. Choose any positive number η and define T(x) = x − η∇f(x). The equation T(x) = x is equivalent to η∇f(x) = 0, hence to ∇f(x) = 0. This gives the desired characterization. The positive step size can be freely chosen for this algebraic equivalence, but the behavior of the iteration depends strongly on it. For constrained problems, a different operator, such as a suitable projected-gradient map, incorporates the constraint instead of ignoring it.

Three practical examples

A solver designer can justify why fixed points of a gradient-step map solve an unconstrained convex training problem.
A constrained estimation method can characterize optimality through a projection-based update whose fixed points respect the feasible set.
A technical reviewer can inspect whether an algorithm's unchanged states correspond to global minima, stationary points or a weaker condition before interpreting its output.

Think of it this way

Think of translating a destination into a rule that recognizes arrival: the instruction leaves you where you are exactly when you have reached an acceptable endpoint. The analogy explains equivalence, but a recognition rule does not provide a reliable route. In mathematics, an operator can identify the right solutions even when its repeated application moves away from them.

Worked example

Consider f(x) = (x − 3)²/2. Its derivative is x − 3, and its unique minimizer is 3. The gradient-step operator is T_η(x) = (1 − η)x + 3η. For every η > 0, solving x = T_η(x) gives ηx = 3η, hence x = 3. With η = 0.5 and starting value 0, iteration gives 1.5, 2.25 and 2.625, approaching 3. With η = 3, it gives 9, −9 and 27, moving away. The fixed-point characterization remains valid in both cases. Now add the constraint x ≤ 1. The constrained minimizer becomes 1, where f'(1) = −2 rather than zero. The unconstrained gradient-step characterization is no longer the right test. Use projection onto C = (−∞, 1]: P_C(z) = min(z, 1), and define S(x) = P_C(x − 0.5(x − 3)). At x = 1, the expression before projection is 2, so S(1) = 1. Within C, no x below 1 is fixed: if projection does not activate, equality would require x = 3; if it activates, the output is 1. This explicitly verifies the constrained solution for this example.

Illustrative case study

In a fictional model review, Birch Planning introduces upper bounds on a fitted operational parameter. The original documentation treats a zero gradient as the stopping target. A reviewer notices that a valid constrained optimum can lie on the boundary with a nonzero gradient. The team rewrites the mathematical characterization using the feasible set and tests the update against a one-variable example. This prevents an incorrect interpretation of boundary behavior; it does not establish the quality of the larger planning model.

Common misconceptions and mistakes

Do not confuse an equivalent fixed-point equation with a convergent iteration. A step size that preserves the solution set can still make iteration unstable. In nonconvex optimization, fixed points of the plain gradient-step map characterize stationary points, which can include maxima and saddles. Ignoring constraints can exclude a valid boundary optimum. Choosing η = 0 makes every point fixed and destroys the useful characterization. Finally, an approximate fixed point is not automatically close to an optimizer without additional error bounds or conditioning assumptions.

Business use

Use a characterization to make solver claims precise. State the original objective and constraints, define the operator completely, and explain what its fixed points represent. Ask the technical team to distinguish algebraic equivalence from convergence assumptions and stopping diagnostics. Test boundary cases and nonunique solutions where relevant. When requirements change, such as adding a constraint or penalty, revisit the characterization before reusing an existing update rule. Business users mainly need to know whether a reported solution satisfies the intended problem, rather than merely satisfying an outdated mathematical condition.

Check your understanding

Why is x = 1 the relevant fixed point after adding x ≤ 1, despite its nonzero gradient?

Answer: The constraint changes optimality. The projected update leaves 1 unchanged because the unconstrained step points outside the feasible set and projection returns it to the boundary.

Related terms

Learn next

derive the fixed points of T_η for several positive step sizes and then study the corresponding iterations separately. Repeat the constrained example with a different upper bound. Explain which part of the derivation establishes equivalence and which part would be needed to establish convergence.

Sources

Sources: Jung and colleagues, The Aalto Dictionary of Machine Learning, fixed-point characterization and fixed-point equation, https://aaltodictionaryofml.github.io/ADictML_Main.pdf ; Parikh and Boyd, Proximal Algorithms, Sections 2.3 and 4.2, https://stanford.edu/~boyd/papers/pdf/prox_algs.pdf . The quadratic and constrained examples are original; the gradient is evaluated at the operator's input, and convergence is distinguished from characterization.

Previous: Fixed point Next: Fixed-point computation