Is every optimal controller an AI system?

No. Optimal control includes mathematical methods developed independently of machine learning, although it connects closely to planning and reinforcement learning.

Can two different controllers both be optimal?

Yes, if they solve different objectives or if one problem has multiple equally good solutions.

Does optimal mean fastest?

Only when the objective and constraints make speed the criterion being optimized. Other designs trade speed against effort or other outcomes.

AI Glossary / Graphs and symbolic AI / Intermediate

Optimal controllers

Definition

Optimal controllers select control actions that achieve the best value of a specified objective for a modeled system, subject to its constraints. The objective usually measures performance over time rather than only the immediate result. Optimality always refers to particular dynamics, available information, allowed actions, and costs; it does not mean that a controller is best for every situation.

What it means

A controller decides how to influence a changing system. For example, it can choose an actuator input from a measured or estimated state. An optimal controller makes that decision through an explicit performance criterion.

The criterion matters as much as the calculation. One design may value rapid correction, while another penalizes heavy actuator use. Both can be optimal for their own objectives and produce different behavior. A claim of optimal control is therefore incomplete without saying what is being optimized and under which assumptions.

How it works

Specify how actions change the state, which states and actions are allowed, and how outcomes accumulate cost. The planning horizon may be finite or extend indefinitely. An optimization method then finds a suitable action sequence or a policy that maps available information to actions.

Dynamic programming uses future cost as part of today's decision. Linear quadratic regulation is a well-known special case involving linear dynamics and a quadratic cost. Other systems require different solution methods or approximations.

A feedback policy updates its action from the current state. An optimized sequence executed without new observations is different: it may cease to suit the situation after an unexpected disturbance. Model predictive control repeatedly solves a planning problem using updated information, though its guarantees depend on the particular formulation.

Three practical examples

Temperature regulation: a controller balances deviation from a target against heating effort within a simplified process model.
Robot positioning: a controller trades rapid error reduction against the size of actuator inputs, while respecting the model and operational limits.
Computing services: a scheduling experiment treats backlog as a state and resource allocation as an action, comparing service delay with resource use over a stated horizon.

Think of it this way

Think of choosing how strongly to brake a rolling trolley so it reaches a marked stopping area smoothly. Braking harder can reduce position error but increase another cost, such as abruptness. An optimal controller needs a precise version of that trade-off. The analogy is only conceptual; real motion includes physical limits, measurement errors, and safety requirements absent from a simple mathematical example.

Worked example

Use a deliberately small one-step problem. Let x be an abstract deviation from a target. The next deviation is x plus u, where u is the chosen correction. Let the cost be the next deviation squared plus the correction squared:
J = (x + u) squared + u squared.

For x = 4, doing nothing gives a cost of 16. Fully canceling the deviation with u = minus 4 also gives 16, because the correction itself is penalized.

Rewrite the objective as 2 times (u + x/2) squared plus x squared divided by 2. The squared term cannot be negative, so the unconstrained minimum occurs at u = minus x/2. With x = 4, choose u = minus 2. The remaining deviation is 2, and the cost is 4 plus 4 = 8.

If the actuator is restricted to corrections between minus 1 and 1, minus 2 is infeasible. Within that interval, the closest value to the unconstrained optimum is minus 1, giving a cost of 9 plus 1 = 10. Constraints change the optimal action. This calculation proves only the stated one-step result; it does not establish a controller for an unlimited sequence of decisions.

Illustrative case study

Fictional case: Marlow Systems builds a simulator for regulating a small test chamber. Its team first proposes an objective that heavily rewards reaching the target temperature quickly.

Reviewers notice that the simulated controller uses large corrections. The team adds a justified penalty on control effort and represents the permitted input range explicitly. Engineers compare the resulting trajectories under several initial conditions and model variations.

They retain a clear distinction between the controller's mathematical objective and the wider acceptance criteria for a deployed product. The case reports a design process, not a measured energy saving or a claim that a simplified simulator proves real-world reliability.

Common misconceptions and mistakes

Do not confuse optimal with flawless. A controller can optimize an inaccurate model or an incomplete objective.

Do not compare two controllers' cost numbers when their cost definitions differ. A score of eight under one weighting cannot automatically be ranked against ten under another.

Avoid clipping an unconstrained action and assuming this preserves every guarantee of the original design. Explicitly constrained optimization may be a different problem. Also distinguish a local approximation from a global optimum, and distinguish finite-horizon performance from long-term stability. These are separate claims requiring appropriate evidence.

Business use

When evaluating a proposed control system, ask the supplier to state the objective, limits, information available to the controller, and conditions under which its performance claim holds. Request comparisons using the same operating scenarios and acceptance criteria.

For the worked example, a manager could understand the trade-off without solving a general control problem: stronger correction is not automatically preferable when effort carries a cost. The practical decision is whether those costs and restrictions reflect the intended operation. Testing should include model mismatch and disturbances relevant to that operation.

Check your understanding

In the example, why is a correction of minus 2 rejected when the input range is minus 1 to 1?

Answer: It is infeasible. The relevant optimum must satisfy the permitted input range, even though an unconstrained calculation prefers it.

Related terms

Dynamic programming
Feedback
control policy
cost function
linear quadratic regulator
model predictive control
state estimation
constraints

Learn next

change the correction penalty in the worked objective and derive the new one-step optimum. Then study how adding another time step changes the role of future cost. Compare this with a feedback policy before exploring linear quadratic regulation.

Feedback

Sources

Primary sources: Russ Tedrake, Underactuated Robotics, “Linear Quadratic Regulators”: https://underactuated.mit.edu/lqr.html and “Dynamic Programming”: https://underactuated.mit.edu/dp.html Relevant formulation passages were consulted. The one-step calculation and Marlow Systems case are original teaching illustrations.

Previous: Optimal brain damage Next: Optimal design