AI Glossary / AI and machine learning / Advanced

Machine learning control

Definition

Machine learning control is the use of machine-learning methods to develop or improve control laws that determine a system's actions from observations. In a specific research tradition, the term describes model-free searches for feedback laws using genetic programming.

What it means

A predictor estimates what may happen. A controller chooses an action intended to influence what happens next. For example, a controller may adjust an actuator based on sensor readings to maintain a desired operating condition.

Machine learning can contribute by identifying dynamics, learning a policy or searching directly for a useful control law. The phrase therefore has a broad engineering meaning, while the MLC work associated with Duriez, Brunton, Noack and collaborators emphasizes genetic-programming searches for feedback functions, including flow-control experiments.

A control system operates in a feedback loop: its actions change the environment that produces its next observations. This makes evaluation different from ordinary prediction on a fixed test set. Stability, constraints, delays and behavior outside familiar conditions can matter as much as average task performance.

How it works

The designer defines sensors, available actions, constraints and a performance objective. Candidate control laws map observations, and sometimes internal state, to actuator commands. Their performance is evaluated in simulation or a suitable experimental setting.

In genetic-programming-based MLC, candidate laws are represented as executable expressions. Selection, mutation and recombination generate new candidates from the search history. Each candidate is assessed with the control objective, which can combine performance with costs such as actuation effort.

Other learning-based control approaches use different mechanisms, so the method should always be named. A learned controller requires validation under disturbances, measurement noise, delays and operating changes. A high score in training does not prove closed-loop stability or safe constraint satisfaction. Those properties need analysis, testing or protective control structures appropriate to the application.

Three practical examples

1. A flow-control research experiment searches for a feedback law that adjusts actuation based on measured flow behavior.

2. A simulated energy-management task learns how to balance a tracking objective against the cost of actuator use.

3. An engineering team uses machine learning to identify a dynamics model, then employs that model within a separate control-design procedure.

Think of it this way

Think of learning how much to turn a tap while watching the water level, rather than only predicting the level. The limit is that a real control loop can have delays, instability and constraints that a simple everyday analogy does not capture.

Worked example

Suppose a fictional controller is evaluated using J = tracking error + 0.2 × actuation cost, with lower values preferred. Controller A has error 10 and actuation cost 20, giving J = 10 + 4 = 14. Controller B has error 12 and cost 5, giving J = 12 + 1 = 13.

B scores better under this objective even though its tracking error is larger. If the actuation penalty changes, the preference may change too.

This calculation compares stated objectives only. It does not establish that either controller is stable, respects actuator limits or performs acceptably under disturbances. Those are separate parts of controller evaluation.

Illustrative case study

A fictional laboratory searches for a controller in a simulator. The learned policy obtains a strong objective score by using rapid actuator changes that the real hardware cannot reproduce.

The team adds realistic actuator limits and delay models, then tests candidate behavior across a wider range of conditions. It compares with a conventional controller and documents the gap between simulation and physical operation.

The case illustrates that a learning algorithm optimizes the environment and objective it is given. Missing physical constraints can produce apparently successful behavior that is unsuitable for the actual system.

Common misconceptions and mistakes

• Accurate prediction automatically provides good control. Control requires choosing actions and evaluating their feedback effects.
• Model-free means no assumptions. The action space, objective and experimental setting still impose assumptions.
• A high reward proves stability. Reward and stability are different properties.

Business use

Machine-learning control can be relevant to specialized engineering tasks where dynamics are complex and conventional models are difficult to obtain. It requires collaboration between learning specialists and control engineers.

Define the operating envelope and constraints before evaluating candidates. Compare against established controllers and account for the cost of experiments. Test disturbances and delayed or missing observations, not only nominal conditions. Keep claims proportional to evidence: successful simulation, successful controlled experiment and dependable operational deployment are distinct milestones.

People also ask

Is machine learning control the same as reinforcement learning?

They overlap, but learning-based control includes other methods, and genetic-programming MLC is a specific approach.

Must the method learn a dynamics model?

No. Some approaches search directly for a control law, while others use learned models.

Why is feedback important?

Actions affect future observations, so the controller changes the conditions under which it will act next.

Check your understanding

Why can a controller that performs well in a simulator fail on hardware with actuator delays?

Answer: The learned law may rely on responses that the real actuator cannot deliver, changing the closed-loop behavior and potentially undermining performance or stability.

Related terms

• Reinforcement learning
• Genetic programming
• Stability
• feedback control
• control law
• system identification
• actuator
• model predictive control

Learn next

Read feedback control and system identification. Separate the prediction task from the action-selection task, then list the constraints and disturbances a controller must handle.

More in AI and machine learning

• M-theory (learning framework)
• Machine learning - Machine learning is the study and practice of creating systems that learn useful patterns from data or experience to perform a task.
• Machine learning as a service - Machine learning as a service is the delivery of ML capability over the internet as a metered product: instead of building models in-house, you call a...
• Machine Learning Pipeline - A machine learning pipeline is the ordered chain of steps that turns raw data into predictions: collection, cleaning, features, training, evaluation and...
• Machine learning system - A machine learning system is the whole working arrangement around a trained model: data, pipeline, model, serving, monitoring and the people and processes...
• Machine perception - Machine perception is the use of computational methods to interpret information from sensors, such as images, sound, depth or touch.

Sources

Parezanović and colleagues, Closed-Loop Control of an Experimental Mixing Layer Using Machine Learning Control: https://arxiv.org/abs/1408.3259. Duriez and colleagues, Attractor Control Using Machine Learning: https://arxiv.org/abs/1311.5250. Author-hosted MLC chapter: https://faculty.washington.edu/sbrunton/mlcbook/CH02_MLC.pdf. All objective values and laboratory scenarios are fictional.

Previous: Machine learning as a service Next: Machine learning model metadata