Is a constraint the same as a penalty?

No. A penalty changes the objective; unless an appropriate equivalence is established, it can permit violations that a hard restriction forbids.

Must the best solution lie on a boundary?

No. It can be interior when the unconstrained optimum is feasible.

Can there be no solution?

Yes. Restrictions may conflict, or a finite minimum may fail to be attained.

AI Glossary / Mathematics and optimization / Advanced

Constrained minimization

Definition

Constrained minimization is the task of finding the smallest value of an objective function while satisfying specified restrictions on the decision variables. Restrictions can include variable bounds, equalities and inequalities. A solution must be feasible as well as good for the objective. In the supplied SciPy context, the term concerns numerical minimization with explicitly represented constraints.

What it means

A system rarely has unlimited freedom to improve one score. It may need to reduce prediction error while keeping coefficients nonnegative, or reduce a production penalty while respecting capacity. The constraints define what is allowed; the objective ranks the allowed alternatives.
This distinction prevents a misleading comparison between an attractive but impossible plan and a workable one. A lower objective value is not automatically a better answer if it breaks the rules. Conversely, satisfying every rule does not establish that a feasible answer is the best available.

How it works

Define the variables, the scalar objective and each restriction. Identify whether the restrictions are linear, nonlinear or simple bounds, and whether derivatives are available. Choose a solver that supports that structure. SciPy provides constrained methods through its minimization interface, including SLSQP and trust-constr; their accepted constraint representations and capabilities should be checked in the documentation.
After solving, inspect the termination information, recompute constraint residuals and evaluate the objective independently. Numerical tolerances matter: a computed equality may hold approximately rather than exactly. Many nonlinear methods seek local solutions. For a convex minimization problem, a local optimum is also global, but convexity must apply to the full problem, not merely one component.

Three practical examples

A forecasting team minimizes squared prediction errors subject to nonnegative model weights. The restriction encodes an intended interpretation, although it may increase the achievable training error compared with unrestricted fitting.
A packaging team minimizes deviations from two target ingredient quantities while limiting their combined amount. An unconstrained target can be physically impossible because it exceeds container capacity.
An operations team minimizes a scheduling penalty while requiring a specified total of staff hours. If assignments must be whole shifts, those discrete requirements need their own treatment; a continuous solution containing fractional shifts is not automatically usable.

Think of it this way

Think of choosing the lowest point on a landscape while staying inside a marked fence. The lowest point in the entire landscape may lie outside the fence. At the best permitted point, moving downhill may require crossing the boundary, which explains why the ordinary objective gradient need not vanish.

Worked example

Consider an original continuous allocation problem: minimize f(x, y) = (x − 4)² + (y − 3)², subject to x ≥ 0, y ≥ 0 and x + y ≤ 5. The unconstrained target (4, 3) has objective zero but uses seven units, so it is infeasible.
The constrained solution is (3, 2), with objective 1 + 1 = 2. To prove this beyond checking a few alternatives, write any feasible point as x = 3 + d and y = 2 + e. Feasibility implies d + e ≤ 0. Expanding the objective gives f = 2 − 2(d + e) + d² + e². Every term after the initial two is nonnegative. Equality requires d = e = 0, establishing the unique global minimum.
The capacity restriction is active because x + y = 5, while both nonnegativity restrictions have slack. The objective gradient at the solution is (−2, −2), not zero. With constraint function x + y − 5, multiplier 2 balances this gradient: (−2, −2) + 2(1, 1) = (0, 0).

Illustrative case study

In a fictional blending workshop, the preceding squared-deviation objective represents a simple preference for two target quantities, not measured financial profit. The team obtains the allocation (3, 2). Before production, an operator identifies an additional availability limit: x cannot exceed two units. The previous answer is now infeasible.
With x ≤ 2 included, the best allocation becomes (2, 3), and the objective rises to four. This follows because (x − 4)² is at least four for every x ≤ 2, while the second squared term cannot be negative. The proposed allocation attains that lower bound and satisfies capacity.
The workshop records why the result changed. It does not characterize the larger objective as a solver failure: an omitted operational requirement has reduced the available choices. These are analytical teaching calculations, not a reported software benchmark or deployment outcome.

Common misconceptions and mistakes

Solving without constraints and then clipping each variable does not generally solve a coupled constrained problem. In the worked example, neither variable alone exceeds five, yet their sum violates capacity.
An active constraint does not always have a positive multiplier; degeneracy can produce zero multipliers. A solver's failure to converge also does not by itself prove infeasibility. For comparison, requiring x ≥ 4 and y ≥ 3 together with x + y ≤ 5 is directly contradictory, regardless of which solver is used.

Business use

Constrained minimization makes operational limits explicit and reviewable. Teams should distinguish mandatory restrictions from preferences that belong in the objective, document units, and check proposed solutions against the original business rules. Relaxing a real restriction changes the decision problem. If a softer alternative is explored, report the permitted violation and its consequences rather than silently treating the relaxed answer as compliant.

Check your understanding

Why is (4, 3) unacceptable in the example?

Answer: Its total exceeds five.

Which original restriction is active at (3, 2)?

Answer: The combined capacity limit.

Does a nonzero objective gradient disprove constrained optimality?

Answer: No; feasible movement and the constraint geometry must also be considered.

Related terms

Constrained optimization
Objective function
Convex optimization
feasible region
active constraint
Lagrange multiplier
Karush–Kuhn–Tucker conditions
sequential quadratic programming

Learn next

Study feasibility and optimality residuals, then compare hard constraints with penalty formulations. Practice proving small convex solutions before interpreting numerical outputs from larger nonlinear problems.

Sources

SciPy documentation, Optimization tutorial, Constrained minimization: supported methods, constraint forms and worked implementation guidance. https://docs.scipy.org/doc/scipy/tutorial/optimize.html#constrained-minimization Stephen Boyd and Lieven Vandenberghe, Convex Optimization slides, chapters 4–5: local versus global optima and constrained optimality conditions. https://web.stanford.edu/~boyd/cvxbook/bv_cvxslides.pdf The allocation problem, proof and fictional workshop case are original illustrations.

Previous: Constrained least squares Next: Constrained mixture model