Must the surrogate be minimized exactly?

Not always. Sufficient surrogate improvement can preserve the basic non-increase argument, but convergence claims may require additional conditions.

Does MM guarantee a global optimum?

No. Especially for nonconvex objectives, descent does not establish global optimality.

Is a surrogate the same as the true objective?

No. It is a constructed bound used to choose an update and is rebuilt or reconsidered at subsequent iterates.

AI Glossary / Mathematics and optimization / Intermediate

Majorize-minimize

Definition

Majorize-minimize, or MM, is an optimization principle that repeatedly minimizes an upper-bounding surrogate for the objective. At the current iterate, the surrogate must touch the original objective and remain at least as large throughout the relevant feasible domain. Improving this surrogate then gives a non-increasing sequence of objective values.

What it means

A difficult objective may be replaced temporarily by a simpler function that is easier to optimize. The replacement is carefully controlled: it must be an upper bound, not merely a nearby approximation. After taking a step, the method constructs a new surrogate around the new point.
The name describes a framework rather than one fixed algorithm. Different problems and surrogate constructions lead to different updates. The related minorize-maximize idea reverses the inequalities for a maximization problem.
The useful guarantee is usually descent of the stated objective. Descent alone does not prove that the iterates converge, that they converge quickly, or that the final point is globally optimal. Stronger conclusions require additional assumptions about the objective, surrogates, and update process.

How it works

Suppose the current point is θₜ and the objective is f. Construct g(θ|θₜ) such that g(θ|θₜ)≥f(θ) for every feasible θ and g(θₜ|θₜ)=f(θₜ). Choose a feasible next point that minimizes the surrogate, or at least does not increase it relative to the current point.
The descent argument is a short chain: f(θₜ₊₁)≤g(θₜ₊₁|θₜ)≤g(θₜ|θₜ)=f(θₜ). Each inequality has a distinct role: upper bounding, surrogate improvement, and contact at the current iterate.
An approximate surrogate solution can preserve this guarantee if it satisfies the required improvement condition. Merely running an inner optimizer for a few steps does not establish that condition automatically. Constraints must be respected, and the bound must apply wherever the proposed update is accepted.

Three practical examples

A numerical fitting method can replace a difficult objective with a sequence of simpler weighted quadratic problems when a valid upper bound is available.
An optimization researcher can design updates with a direct objective-descent argument using a problem-specific surrogate.
A constrained model can use surrogates that are easier to minimize over its feasible set than the original objective.

Think of it this way

Think of descending a difficult landscape using a temporary surface that lies above it and touches where you stand. Moving to a lower point on that surface cannot place you above your previous height on the original landscape. The analogy depends on the surface truly being an upper bound; a convenient sketch without that property gives no such assurance.

Worked example

Consider an original teaching example with f(x)=x² over the real line. At current value t, define g(x|t)=x²+(x−t)². The extra squared term is nonnegative, so g≥f everywhere, and g(t|t)=t²=f(t).
Expanding gives g(x|t)=2x²−2tx+t². Its derivative is 4x−2t, so the surrogate minimizer is x=t/2. Starting at t=4, the next iterate is 2. The original objective decreases from 16 to 4, while the surrogate at the new point is g(2|4)=4+4=8. The descent chain is 4≤8≤16.
Repeating gives iterates 4, 2, 1, 0.5 and objective values 16, 4, 1, 0.25. This example is deliberately simple: the original objective was already easy to minimize. Its purpose is to show the logic of the upper bound and contact condition, not to claim a computational advantage.
Now see why descent alone is insufficient. On [−1,1], let f(x)=(x²−1)² and start at zero. The surrogate g(x|0)=1+4x² touches f at zero and is an upper bound because g−f=6x²−x⁴≥0 on that domain. Its minimizer is still zero. Repeating this update makes no progress, although f(±1)=0 and f(0)=1. A valid non-increasing MM step can therefore remain at a poor stationary point unless further conditions or strategy address the issue.

Illustrative case study

In a fictional model-fitting project, engineers implement an iterative surrogate method for a difficult objective. The first prototype reports decreasing surrogate values, but the actual objective occasionally increases. Review reveals that the bound was only justified within a restricted region, while some updates left that region.
The team revises the feasible update rule and checks the original objective after each accepted step. This hypothetical example illustrates why implementation must match the mathematical bound. It does not claim that monitoring alone proves global convergence or that every observed increase has the same cause.

Common misconceptions and mistakes

Do not call every local approximation an MM surrogate. A tangent line to a convex function is generally a lower bound, which is the wrong direction for a minimization majorizer. Do not omit the contact condition at the current point.
Also distinguish non-increasing values from strict decrease. A method may stop improving at a non-optimal point. A surrogate can be valid but too loose to produce useful progress. Numerical errors, approximate inner solves, and inconsistent treatment of constraints can weaken the intended descent behavior unless the acceptance conditions are checked.

Business use

For a business application using an MM-based fitting procedure, the key questions concern the actual objective, computational cost, and quality of the fitted result. Record the stopping condition and evaluate the resulting model independently of the optimization trace.
An objective that decreases steadily can still be a poor representation of business needs. Conversely, a mathematically valid method may converge too slowly for operational limits. Compare useful prediction or decision quality alongside runtime and stability, and avoid treating a monotone training curve as a complete validation of the system.

Check your understanding

A surrogate lies above f but does not equal f at the current point. Does minimizing it automatically prove the next f value is no larger than the current f value?

Answer: No. Without contact, the final equality in the standard descent chain is missing.

Related terms

Optimization
Expectation-maximization
Convergence
Surrogate function
majorization
descent method
nonconvex optimization
iterative reweighting
feasible set

Learn next

Next, verify upper bounding and contact separately for a proposed surrogate. Then write the full descent chain for one update and identify which further assumptions would be needed for a convergence claim beyond non-increasing objective values.

Convergence

Sources

Primary terminology reference: Alexander Jung and contributors, The Aalto Dictionary of Machine Learning, majorize-minimize entry and its upper-bound/contact conditions: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Both scalar surrogate examples and their descent checks are original derivations.

Previous: Majority class Next: Makespan