Is least squares related to quadratic programming?

Yes. A squared-error objective is quadratic in linear model parameters, and adding linear constraints yields a quadratic program.

Does convexity guarantee a feasible solution?

No. Constraints can still contradict each other, and some feasible convex problems may lack a finite attained optimum.

Can a quadratic program handle whole-number decisions?

Ordinary continuous QP does not enforce integrality. Adding integer restrictions changes the formulation and solver requirements.

AI Glossary / Graphs and symbolic AI / Intermediate

Quadratic programming

Definition

Quadratic programming is optimization with a quadratic objective function and linear equality or inequality constraints. The objective can include squared variables and products of pairs of variables. A convex quadratic program has a convex objective over its feasible region, making it an important class of structured optimization problems in machine learning, planning, and control.

What it means

A linear objective describes a constant rate of change, while a quadratic objective can represent increasing penalties for moving away from a target. For example, a squared deviation penalizes a large mismatch more strongly than several small ones of the same total absolute size.
The constraints in an ordinary quadratic program remain linear. If the constraints themselves are quadratic, the problem belongs to the broader class of quadratically constrained quadratic programs. “Programming” here means mathematical optimization, not writing software. Also, a quadratic objective is not automatically convex: its curvature determines which guarantees are available.

How it works

A common minimization form is one-half x-transpose P x plus q-transpose x, subject to linear inequalities and equalities. The vector x holds the decision variables, P describes quadratic curvature, and q describes linear contributions. A constant objective term can be omitted without changing the optimizer.
When P is positive semidefinite, the objective is convex. A suitable solver seeks a feasible point satisfying optimality conditions within numerical tolerances. The result may instead indicate infeasibility or unboundedness. Positive definite curvature gives strict convexity and therefore at most one optimizer on a convex feasible set, if an optimizer exists. Nonconvex quadratic programs require different care and can have local solutions that are not global optima.

Three practical examples

A planning prototype can allocate divisible capacity while penalizing squared deviations from preferred allocations.
A control model can choose future control inputs subject to linear limits while minimizing quadratic tracking and effort penalties.
A machine-learning formulation can use a quadratic objective for fitting or regularization, with additional linear constraints when the application requires them.

Think of it this way

Think of finding the lowest point on a bowl while staying inside a fenced region. A convex quadratic objective resembles a bowl, and linear constraints form straight boundaries. The analogy is limited because problems can have many dimensions, flat directions, or nonconvex curvature; not every quadratic surface is a simple bowl with one bottom.

Worked example

Consider an invented allocation of 10 divisible capacity units between activities X and Y. Their preferred allocations are 8 and 6, but only 10 units are available. Minimize f(x, y) = (x − 8)² + (y − 6)² subject to x + y = 10, x ≥ 0, and y ≥ 0.
Substitute y = 10 − x. The objective becomes (x − 8)² + (4 − x)² = 2x² − 24x + 80. Its derivative is 4x − 24, which is zero at x = 6. Then y = 4, and f = 4 + 4 = 8. The second derivative is 4, so this is the unique minimum along the feasible line segment.
Now add x ≤ 5. The unconstrained-on-the-line minimizer x = 6 is unavailable, so the best feasible allocation is x = 5, y = 5, with f = 9 + 1 = 10. Both solutions satisfy the total-capacity requirement; the additional limit raises the minimum penalty.
In the matrix convention, P = 2I and q = (−16, −12), with constant 100. Omitting that constant changes the reported objective value but not the allocation that minimizes it.

Illustrative case study

Fictional case: Harbor Studio uses a planning worksheet to balance editing capacity across two activities. A prototype optimizer recommends fractional hours, which are acceptable in the worksheet, but a later version switches the variables to counts of indivisible work packages.
The team recognizes that the original continuous quadratic program no longer captures the decision domain. It revises the formulation rather than simply rounding the solution and assuming feasibility. This fictional case illustrates the importance of variable meaning; it claims no real scheduling savings or measured productivity gain.

Common misconceptions and mistakes

Do not assume every quadratic problem is convex or every solver returns a global optimum. Check the objective's curvature and the supported problem class. Do not confuse a quadratic objective with quadratic constraints.
Rounding a continuous solution can violate equalities or alter optimality. Integer restrictions produce a different class of optimization problem. Numerical solver status and constraint residuals also matter: a displayed vector alone is not enough to establish a reliable solution. Finally, the optimizer minimizes the objective supplied to it, so poorly chosen weights or inconsistent units can produce mathematically correct but unsuitable decisions.

Business use

Quadratic programming is useful when deviation, smoothness, or effort costs are naturally expressed through squared terms and the operational limits are linear. The resulting model can make trade-offs explicit and support repeatable scenario analysis.
For a business pilot, define each variable's unit and whether it is divisible. Explain why the squared penalty reflects the decision objective and how weights compare different quantities. Check feasibility, solver status, and sensitivity to important constraints. Present the result as conditional on the model rather than as an objective answer to every business consideration that was left outside it.

Check your understanding

Why does adding x ≤ 5 in the worked example increase the best objective value from 8 to 10?

Answer: It removes the previous optimum (6, 4) from the feasible set. The best remaining point is (5, 5), whose squared deviations total 10. Restricting the feasible set cannot improve the minimum of the same objective.

Related terms

Convex optimization
Objective function
Linear constraints
Least squares
linear constraint
positive semidefinite matrix
optimality condition
feasible region
quadratic constraint
mixed-integer optimization

Learn next

Learn convexity and constrained least squares next. Replace the first penalty with 2(x − 8)² while keeping x + y = 10. Solve the one-variable quadratic and explain how giving the first deviation more weight changes the preferred allocation.

Constrained least squares

Sources

CVXPY documentation, Quadratic program, standard convex form and worked implementation: https://www.cvxpy.org/examples/basic/quadratic_program.html

Previous: Quadratic probing Next: Quadratic unconstrained binary optimization