⚡ electrical engineering

A Data-Driven Algorithm for Model-Free Control Synthesis

This paper presents a model-free, data-driven algorithm that synthesizes optimal infinite-horizon LQR feedback and feedforward controllers for continuous-time systems using only finite input-output data, supported by theoretical justification and validated on a real-scale aircraft.

Original authors: Sean Bowerfind, Matthew R. Kirchner, Gary Hewer

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sean Bowerfind, Matthew R. Kirchner, Gary Hewer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to teach a robot to balance a broom on its hand. The traditional way to do this is to be a physicist first: you measure the broom's weight, calculate the air resistance, figure out the exact friction of the hand, and write a giant math equation describing exactly how the broom moves. Only then do you program the robot to balance it.

This paper says: "Why bother with all that math?"

Instead of trying to understand the physics of the broom, the authors propose a method where you just let the robot try balancing the broom, record what happens, and then use a computer algorithm to figure out the perfect balancing moves directly from that video footage. You skip the physics class entirely and go straight to the solution.

Here is a breakdown of their "Magic Trick" using simple analogies:

1. The Problem: The "Black Box"

In the real world (like flying a drone or driving a car), we often don't know the exact math behind how things move. The wind might be weird, the engine might be old, or the parts might be slightly bent.

Old Way: Try to guess the math (the "Model") first. If you guess wrong, your controller fails.
New Way (This Paper): Don't guess the math. Just watch the system move, record the data, and let the computer find the pattern.

2. The Core Idea: The "Energy Score"

The authors use a concept called LQR (Linear Quadratic Regulator). Think of this as a "Scoreboard" for how well the system is doing.

If the drone wobbles too much, the score goes down.
If the drone stays steady, the score goes up.
The goal is to find the set of rules (the controller) that keeps the score as high as possible forever.

Usually, to find these rules, you need to know the engine's specs. This paper says: "We don't need the specs. We just need to know that the Scoreboard has to follow a specific rule as time passes."

3. The "Magic Rule" (The Necessary Condition)

The authors discovered a mathematical "law of physics" for this Scoreboard. They proved that no matter what the system is, if it's doing a good job, the Scoreboard must change in a very specific way over time.

Think of it like a speedometer in a car.

If you know the car's engine and weight, you can calculate how fast it should go.
But this paper says: "We don't care about the engine. We just know that if you press the gas pedal (input) and the car speeds up (output), the speedometer reading must change in a specific relationship to the gas pedal."

The algorithm looks at a video of the system moving, checks if the "Scoreboard" is following this rule, and adjusts the controller until the rule is perfectly satisfied.

4. The Three Superpowers

The paper doesn't just stop at balancing; it adds three cool features:

Feature A: The "Follow the Leader" Mode (Reference Tracking)
Usually, controllers just try to keep things at zero (like keeping a drone hovering in one spot). But what if you want the drone to follow a specific path, like a race track?
- The Analogy: Instead of just balancing the broom, the robot learns to balance the broom while walking in a circle. The algorithm figures out how to add a "push" (feedforward gain) to make the system follow a command perfectly, without needing to know the physics of the circle.
Feature B: The "Hybrid Brain" (Mixed-Model)
Sometimes you know part of the system but not the rest. Maybe you know how the drone's motors work (because you bought them), but you don't know how the wind affects the specific shape of this drone.
- The Analogy: Imagine you know how a bicycle's gears work, but you don't know how the tires grip the road. The algorithm takes the "gear math" you know and combines it with the "road data" you collected, creating a perfect controller that uses the best of both worlds.
Feature C: Real-World Proof (The Flight Test)
Theory is great, but does it work in a storm?
- The authors took a small, cheap model airplane (a Freewing AL37). They didn't know the exact aerodynamics of this specific plane.
- They flew it manually in windy conditions, recorded the data, and fed it into their algorithm.
- The algorithm instantly designed a controller.
- They flew the plane again, this time letting the computer fly it. Result: The plane flew perfectly, tracking the pilot's commands even in the wind, proving the method works on real, messy, noisy hardware.

5. Why This Matters

Speed: You don't need to spend weeks building a physics model. You just need a few minutes of data.
Robustness: It works even if the data is noisy (like a shaky camera or a windy day).
Accessibility: You don't need to be a PhD in aerodynamics to build a great controller. You just need data.

In summary: This paper gives us a way to teach machines to be experts at their jobs by simply watching them work, rather than forcing them to study textbooks first. It turns "trial and error" into a precise, mathematical science.

1. Problem Statement

The Linear Quadratic Regulator (LQR) is a cornerstone of control theory, offering robustness and optimal performance for linear systems. However, classical LQR design requires complete knowledge of the system dynamics matrices ( $A$ and $B$ ). In many practical applications, particularly in aerospace (e.g., small Unmanned Aerial Vehicles or UAVs), obtaining accurate analytical models is difficult, expensive, or impossible due to:

Unknown Nonlinearities: Systems may be nonlinear, and linearization around an equilibrium point is difficult without a precise model.
Cost Constraints: High-fidelity modeling (CFD, wind tunnel testing) is often economically unfeasible for subscale or cost-sensitive platforms.
Data Availability: While system dynamics are unknown, collecting input-output data from real-world flights or simulations is often readily available.

Existing data-driven methods often rely on "on-policy" learning (requiring active policy updates during data collection, which is risky for real hardware) or assume discrete-time dynamics, which can introduce noise sensitivity. The authors aim to synthesize an optimal continuous-time, infinite-horizon LQR controller directly from finite-length, noisy input-output data without identifying the system model first.

2. Methodology

The proposed approach is a model-free, off-policy method that formulates the control synthesis as a constrained Nonlinear Programming (NLP) problem.

A. Theoretical Foundation

The core of the algorithm relies on a necessary condition derived from the Hamilton-Jacobi-Bellman (HJB) equation and the Algebraic Riccati Equation (ARE).

The authors prove that for an optimal LQR controller with gain $K$ and value function $V(x) = x^T P x$ , the time derivative of the value function along any arbitrary trajectory (not just the optimal one) must satisfy a specific dynamic constraint:
$\dot{V}(\hat{x}(t)) = \hat{x}(t)^T K^T R K \hat{x}(t) - \hat{x}(t)^T Q \hat{x}(t) + 2\hat{x}(t)^T K^T R \hat{u}(t)$
Crucially, this equation eliminates the need for the system matrix $A$ . It relates the state derivative, the value function gradient, and the control input directly.

B. Data-Driven Synthesis (NLP Formulation)

The algorithm discretizes the continuous-time necessary condition using finite differences (e.g., Euler method) over a collected data trajectory $\{u_k, y_k\}$ .

Implicit State Handling: Since measurements ( $y_k$ ) are noisy, the true states ( $x_k$ ) are treated as latent variables. The optimization minimizes the difference between observed outputs and latent states while enforcing the dynamic constraint.
Positive Semi-Definiteness: To ensure the solution $P$ is a valid Lyapunov function, the algorithm solves for a matrix $L$ such that $P = L^T L$ , rather than solving for $P$ directly.
Objective: Minimize the reconstruction error of the states subject to the dynamic constraint holding for all time steps in the dataset.

C. Extensions

The paper extends the base regulator problem to two advanced scenarios:

Reference Tracking: Simultaneously computes the feedback gain $K$ and a feedforward gain $F$ to track a reference command $r$ , eliminating steady-state error.
Mixed-Model Control: Handles systems where part of the dynamics is known (e.g., actuator dynamics) and part is unknown. The known dynamics are integrated into the NLP constraints, allowing the algorithm to solve for the optimal gain using partial model knowledge.

3. Key Contributions

Continuous-Time Model-Free Synthesis: Unlike many Reinforcement Learning (RL) approaches that assume discrete time, this method operates directly on continuous-time dynamics, making it suitable for real-world physical systems.
Robustness to Noise: By treating the state trajectory as an optimization variable constrained by the data, the method is inherently robust to measurement noise, a common issue in real-world flight data.
Unified Framework: The paper provides a single mathematical framework that handles:
- Pure model-free regulation.
- Model-free reference tracking.
- Mixed-model (partial knowledge) control.
Real-World Validation: The method is not just theoretical; it is validated on a physical subscale UAV in windy conditions, demonstrating its practical viability.

4. Results

The authors validated the method through two primary case studies:

A. Simulation (Known Linear System)

Setup: Used a linearized model of a McDonnell-Douglas A-4D Skyhawk.
Comparison: The model-free controller ( $\hat{K}, \hat{F}$ ) was compared against the "ground truth" LQR solution ( $K^*, F^*$ ) derived from known $A$ and $B$ matrices.
Outcome: The model-free gains were nearly identical to the ground truth. Closed-loop simulations showed the model-free controller tracked reference commands with performance indistinguishable from the classical LQR solution.
Mixed-Model Test: When actuator dynamics were added as "known" and the rest as "unknown," the mixed-model NLP successfully recovered the optimal controller, validating the partial-knowledge extension.

B. Real-World Flight Test

Platform: A 1/19th scale Freewing AL37 (Boeing 737-inspired) UAV with unknown, nonlinear dynamics.
Process:
1. Data Collection (TF1): A human pilot flew the aircraft in a racetrack pattern with no stabilization, recording IMU data (roll angle, roll rate) and pilot inputs.
2. Synthesis: The NLP was solved using this raw, noisy data to compute a roll controller gain $\hat{K}$ .
3. Validation (TF2): The computed controller was deployed on the aircraft.
Outcome: The model-free controller successfully stabilized the roll angle and tracked pilot commands during a figure-eight flight pattern in windy conditions. The aircraft achieved the desired trajectory with negligible steady-state error, proving the algorithm's robustness to real-world noise and unmodeled aerodynamics.

5. Significance

This paper represents a significant step forward in data-driven control for aerospace applications.

Bridging the Gap: It bridges the gap between theoretical optimal control and practical implementation on systems where modeling is too costly or complex.
Safety and Feasibility: By using an off-policy approach (data collected via a human pilot or simple controller), it avoids the safety risks associated with "on-policy" adaptive learning algorithms that might destabilize a real aircraft during the learning phase.
Practical Utility: The successful flight test demonstrates that high-performance optimal controllers can be synthesized directly from sensor data, enabling rapid deployment on new or modified UAV platforms without the need for extensive wind tunnel testing or computational fluid dynamics modeling.

In summary, the authors present a rigorous, mathematically grounded, and practically validated algorithm that allows engineers to design optimal LQR controllers using only input-output data, bypassing the traditional and often prohibitive step of system identification.