Is squared error quadratic?

It is quadratic in predictions, and in coefficients when predictions are affine in those coefficients. A neural network's squared-error objective need not be quadratic in its weights.

Can a quadratic contain interactions?

Yes. A term such as uv couples two variables.

Does adding a constant change the minimizer?

No. It shifts all objective values equally while leaving gradients unchanged.

AI Glossary / Mathematics and optimization / Intermediate

Quadratic function

Definition

A quadratic function is a function built from constant, linear and second-degree terms. For a real vector x, it can be written f(x)=xᵀQx+bᵀx+c. Quadratic functions describe curved objectives whose second derivatives are constant.

What it means

In one variable, the familiar form is ax²+bx+c. With several variables, terms such as x₁² and x₁x₂ describe curvature along individual directions and interactions between directions.

Machine learning uses quadratic functions in squared-error fitting, regularization and local approximations to more complicated objectives. Their structure makes many calculations tractable: gradients are linear in the variables, and curvature is represented by one fixed matrix.

A quadratic is not automatically bowl-shaped. Depending on its coefficients, it can curve upward, curve downward, contain flat directions or form a saddle. The word identifies an algebraic structure, not the existence of a minimum.

Some optimization texts include affine functions as degenerate quadratics with Q=0. If strict polynomial degree matters, a quadratic has at least one nonzero second-degree term.

How it works

Only the symmetric part of Q affects xᵀQx. Replacing Q with (Q+Qᵀ)/2 therefore leaves the function unchanged. When Q is symmetric, the gradient is 2Qx+b and the Hessian is 2Q.

A widely used alternative convention writes f(x)=½xᵀHx+bᵀx+c with symmetric H. Then the gradient is Hx+b and the Hessian is H. Keeping track of the factor one-half prevents incorrect updates.

If H is positive definite, every nonzero direction has positive curvature. The unconstrained function has a unique minimum at the solution of Hx=−b. Positive semidefinite H gives convexity but can leave flat directions; the linear term still matters for whether a minimum exists. An indefinite H gives both upward and downward directions on the unrestricted space.

Constraints change the optimization problem. A function unbounded below over all real inputs may have a minimum on a nonempty compact feasible set. A local quadratic approximation also describes a different object from an objective that is exactly quadratic everywhere.

Three practical examples

A forecasting team fits linear coefficients by minimizing squared residuals, creating a quadratic objective in those coefficients.
An optimizer approximates a smooth nonlinear objective near its current point using the gradient and Hessian.
A scheduling analyst penalizes deviations from a target workload using squared differences, then adds capacity constraints.

Think of it this way

Think of a surface whose curvature is the same wherever you stand. A bowl, an upside-down bowl and a saddle give different versions of this idea.

A drawing is only an analogy for two variables. A model can have thousands of coordinates, and a constant Hessian is the precise statement. Calling every quadratic a bowl would hide important cases.

Worked example

Locating a two-variable minimum

Consider f(u,v)=u²+2uv+3v²−4u−8v+10. Its gradient is (2u+2v−4, 2u+6v−8), and its Hessian is the matrix with rows (2,2) and (2,6).

Setting the gradient to zero gives u+v=2 and u+3v=4. Subtraction yields v=1, then u=1. Substituting gives f(1,1)=4.

To verify this is a minimum, write a=u−1 and d=v−1. Direct expansion gives f(u,v)=4+a²+2ad+3d²=4+(a+d)²+2d². Both squared terms are nonnegative and vanish together only at a=d=0. The minimum is therefore unique.

If a constraint requires v≤0, the unrestricted solution is infeasible. For fixed v, minimizing over u gives u=2−v and f=6+2v²−4v. Over v≤0 this is minimized at v=0, giving u=2 and value 6. The function did not change; the permitted choices did.

Illustrative case study

A fictional distribution center uses a quadratic penalty to balance two staffing adjustments. Its analyst presents the unconstrained optimum as the recommended plan. Operations then points out that one adjustment is unavailable that week.

The team adds the availability constraint and resolves the problem. It reports the higher objective value as the cost within this simplified planning model of respecting the restriction, rather than calling the constrained solver inaccurate.

Before deploying the plan, staff check whether squared penalties reflect actual priorities. Mathematical convenience does not establish that a penalty represents overtime, service quality or employee preferences appropriately.

Common misconceptions and mistakes

Every quadratic has one minimum. Concave, indefinite and degenerate cases behave differently.
A zero gradient proves a minimum. Curvature or another argument is needed.
Positive semidefinite curvature guarantees a finite minimum regardless of the linear term. A flat direction with a nonzero linear slope can make the function unbounded below.
The Hessian equals Q under every notation. It equals 2Q in xᵀQx notation with symmetric Q.
A quadratic approximation is exact everywhere. Usually it is intended only near the expansion point.

Business use

Quadratic models are useful when a transparent, solvable representation of trade-offs is valuable. Ask which variables are optimized, how units affect squared penalties, and which constraints define feasibility.

When comparing solutions, distinguish the mathematical objective from the actual business outcome. A lower modeled penalty is informative only to the extent that the function captures relevant costs and constraints. Sensitivity analysis can show how a proposed decision changes when those assumptions move.

Check your understanding

For f(x)=−x², the derivative vanishes at zero. Is zero the minimum over the real line?

Answer: No. It is the maximum. Values decrease without bound as |x| increases, illustrating why stationary points need curvature and domain checks.

Related terms

Learn next

Read Hessian and positive definite matrix. Expand the completed-square form in the example, then change the feasible restriction from v≤0 to v≤2 and identify whether it still excludes the unconstrained minimum.

Hessian

Sources

Aalto Dictionary of Machine Learning, quadratic function: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Source for the general matrix form. Derivatives, completed-square verification, constrained calculation and fictional planning scenario are original teaching examples.

Previous: Quadratic classifier Next: Quadratic probing