🔬 physics

Heuristic Learning for Active Flow Control Using Coding Agents

This paper introduces a heuristic learning framework using coding agents to directly search for explicit, interpretable feedback laws for active flow control, demonstrating that this approach matches or outperforms state-of-the-art deep reinforcement learning across 13 benchmarks while offering greater transparency and physical insight.

Original authors: Paul Garnier, Jonathan Viquerat, Elie Hachem

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Paul Garnier, Jonathan Viquerat, Elie Hachem

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to steer a very tricky, invisible boat through a stormy river. The river is full of hidden whirlpools and sudden waves, and you can only see a tiny slice of the water ahead of you. Your goal is to keep the boat moving smoothly without crashing or wasting fuel.

For a long time, scientists have tried to solve this using Deep Reinforcement Learning (DRL). Think of DRL as a super-smart but mysterious apprentice. You let this apprentice crash the boat thousands of times in a computer simulation, learning from every mistake. Eventually, the apprentice gets really good at steering. But here's the catch: the apprentice's brain is a giant, tangled web of numbers (a neural network). Even the scientists can't easily look inside and say, "Ah, I see! You turn the rudder left because the water is moving fast here." It's a "black box" that works, but nobody knows exactly how it thinks. Plus, it takes a massive amount of computer power and time to train this apprentice.

In this paper, the authors, Paul Garnier and his team, tried a totally different approach. Instead of training a mysterious apprentice, they hired a Coding Agent—think of it as a brilliant, tireless robot programmer.

The New Strategy: Writing the Rules, Not Learning the Feelings
Instead of letting the AI tweak millions of invisible numbers, the researchers asked the Coding Agent to write actual, readable computer code that acts as the steering rule. It's like asking the agent to write a simple recipe: "If the water looks wavy on the left, wait 11 seconds, then push the rudder slightly right."

The agent tries a recipe, runs a simulation to see if the boat crashes, and if it does, the agent rewrites the recipe. It keeps doing this, reading its own "diary" of what worked and what failed, until it finds a set of instructions that works perfectly.

What They Found (The Good News)
The team tested this new "robot programmer" method on 13 different fluid dynamics challenges, ranging from simple 2D flows to complex 3D turbulent channels. They gave the Coding Agent the exact same amount of computer time and the exact same rules as the best DRL apprentices.

The results were surprising:

In 10 out of the 13 challenges, the robot programmer found a steering rule that was just as good as, or even better than, the best DRL apprentice.
In one specific 3D turbulent channel test, the new method reduced drag (air/water resistance) by nearly 40%, beating the DRL method which only got about 30%.
The best robot programmer (called "Codex") found solutions that were compact, readable, and easy for humans to understand. You can actually look at the code and see the logic: "Oh, it's using a delay to wait for the wave to pass before acting!"

What They Ruled Out (The "No-Go" Zones)
The paper is very strict about what this method is not.

It is NOT a magic trick that works without limits. The researchers explicitly ruled out letting the AI cheat by looking at "hidden" parts of the simulation or changing the rules of the game. The robot programmer had to play by the exact same strict rules as the DRL apprentice.
It is NOT always better. In 3 of the 13 cases, the robot programmer didn't quite beat the DRL apprentice. The paper suggests that while this new method is a strong competitor, it hasn't completely solved every single problem yet.
It is NOT about having more information. The team tried giving the robot programmer a super-powerful view of the entire river (full mesh fields) instead of just a few sensors. Surprisingly, this didn't help; in fact, it sometimes made the search harder. The paper suggests that having too much data can confuse the search for a simple, clear rule.

How Sure Are They?
The authors are confident in their numbers because they ran these experiments in a controlled, simulated environment. They didn't just guess; they measured the results.

They proved that the robot programmer can find rules that match or exceed the performance of the best DRL methods in most cases.
They showed that these rules transfer well. For example, a rule they wrote for a river with 5 "jets" (actuators) could be tweaked slightly to work perfectly on a river with 10 jets, without needing to start from scratch.
They found that these rules are robust. Even when they reduced the number of sensors (making the view very blurry), the robot programmer still found good solutions, whereas the DRL apprentice struggled significantly.

The Big Picture
The paper suggests that we might not need to rely solely on those giant, mysterious neural networks for controlling fluid flows. By using modern coding agents to search for simple, human-readable rules, we can get the same (or better) performance while finally understanding why the controller is making its decisions. It's like swapping a magic wand that works for a clear, step-by-step instruction manual that anyone can read, tweak, and trust.

The authors conclude that this "heuristic learning" is a credible and exciting alternative to traditional methods, combining the power of AI with the clarity of human logic. They even made their code and prompts available for others to try, proving that this isn't just a secret trick, but a new way of thinking about how to control the invisible forces of nature.

Technical Summary: Heuristic Learning for Active Flow Control Using Coding Agents

Problem Statement

Active flow control presents a significant challenge in computational fluid dynamics (CFD) due to nonlinear dynamics, partial observability, and the high computational cost of numerical simulations. While Deep Reinforcement Learning (DRL) has become a dominant framework for these problems, it suffers from two primary limitations:

Sample Inefficiency: Training typically requires vast numbers of simulator interactions (rollouts), each involving expensive CFD solutions.
Lack of Interpretability: The resulting controllers are neural networks with opaque decision processes, making them difficult to analyze, modify, or transfer to related configurations.

The authors pose a complementary scientific question: Can explicit, human-readable control laws be discovered automatically that remain competitive with state-of-the-art DRL, while offering interpretability and direct inspectability?

Methodology: Heuristic Learning via Coding Agents

The paper proposes a paradigm shift from optimizing neural network parameters to searching directly in the space of explicit executable programs. This is achieved using modern coding agents (Large Language Models with code-generation capabilities) within a constrained heuristic-learning protocol.

Core Framework

The process treats the controller as an executable program $c$ with explicit internal memory $m_t$ . The agent iteratively proposes, evaluates, and revises these programs based solely on observed performance, without accessing hidden simulator states.

The Optimization Loop:

Interface Inspection: The agent analyzes the public benchmark interface (observation structure, action bounds, reward definitions).
Policy Proposal: The agent writes a compact, executable controller (e.g., feedback laws, delayed filters, finite-state machines, spatial projections).
Evaluation: The controller is tested via deterministic rollouts in the simulator. Scores, diagnostics, and action traces are recorded.
Revision: Based on the "search ledger" (accumulated evidence of successes, failures, and physical metrics), the agent refines the controller's structure, gains, delays, or logic.
Validation & Simplification: Surviving controllers undergo ablation studies and validation on unseen seeds or longer horizons.

Strict Constraints

To ensure a fair comparison with DRL, the framework enforces three explicit constraints:

Identical Budget: The heuristic search receives the exact same total simulation budget (environment interactions) as the corresponding DRL baseline. Early termination of ineffective candidates still counts against the budget.
Public Interface Only: The agent is prohibited from modifying the simulator, accessing hidden variables, or bypassing the official control interface.
Explicit Validity: Controllers must rely exclusively on public observations and must not load opaque pre-trained neural networks. They must be fully inspectable and executable without training.

Key Contributions

The paper makes four primary contributions:

Reproducible Protocol: Introduction of a heuristic-learning protocol where coding agents autonomously search for explicit feedback controllers under strict public-interface constraints.
Comprehensive Evaluation: Evaluation of this protocol across 13 fluid-dynamics control benchmarks (spanning 1D, 2D, and 3D problems) using multiple coding agents (Codex, Claude, Fable, Gemini) and comparison against the strongest available DRL baselines (PPO, SAC, TD3).
Performance & Interpretability: Demonstration that automatically discovered heuristic controllers match or outperform the best DRL policies in 10 of 13 environments while remaining compact and physically interpretable.
Robustness Analysis: Investigation of the approach's generalization capabilities through transfer learning, variable-dimensional action spaces, and sparse observation settings.

Key Results

Performance Comparison

Benchmark Success: The heuristic approach reached or exceeded the best DRL performance in 10 out of 13 environments.
Specific Improvements:
- In the 3D Turbulent Channel Flow benchmark, the discovered heuristic improved the normalized score by approximately 50% compared to the best DRL baseline, corresponding to a drag reduction of nearly 40% (vs. ~30% for DRL).
- In the 2D Cylinder Jet benchmark, while the heuristic was slightly below the best DRL score on the original training horizon, it demonstrated superior long-horizon stability. When evaluated over 400 time steps (vs. the original 80), the DRL policy (PPO) lost stability, whereas the heuristic and SAC remained stable, suggesting the heuristic captured a more generalizable feedback strategy.

Agent Comparison

Model Performance: Among the tested agents, Codex (GPT-5.5) performed best, followed by Fable 5, Claude Opus 4.7, and Gemini 3 Pro. Codex achieved the highest scores in 5 of the 7 BEACON environments tested.
Cost Efficiency: While Codex had the highest absolute token cost, Fable 5 offered the most favorable performance-to-cost ratio when normalized by benchmark score.

Robustness and Transfer

Transfer Learning: Controllers transferred from simpler to more complex configurations (e.g., Reynolds number changes, increased actuator counts) required only minor parameter adjustments (e.g., changing delay or gain values) to recover or exceed original performance. This contrasts with DRL, which often requires complete retraining.
Sparse Observations: The heuristic search remained robust even when the number of sensors was reduced from 192 to 12, outperforming TD3 in sparse regimes and converging faster.
Observation Dimensionality: Interestingly, providing full-field mesh observations did not automatically improve performance; in some cases, the search space enlargement hindered the discovery of optimal explicit controllers compared to using limited, physically meaningful probes.

Policy Anatomy and Physical Insight

The paper analyzes the structure of the discovered controllers to demonstrate their physical interpretability:

Shkadov (Falling Film): The agent discovered a delayed shape feedback mechanism. It identified that successful control requires extracting local wave geometry features (mean, slope, curvature) and applying actuation after a specific transport delay to synchronize upstream sensing with downstream wave propagation.
Turbulent Channel Flow: The controller utilized normalized local wall feedback. It learned to normalize wall velocity fluctuations relative to the mean and standard deviation, then apply a distributed feedback signal. Ablation studies confirmed that relative fluctuations, not absolute values, were the critical control variables.
Comparison with PPO: On the Cylinder Jet benchmark, the heuristic and PPO policies showed high agreement in instantaneous control decisions (mean absolute difference of 0.2852) during the quasi-stationary regime, suggesting they learned compatible feedback mechanisms despite different representation spaces.

Significance and Claims

The authors claim that heuristic learning through coding agents constitutes a credible and complementary alternative to conventional reinforcement learning for active flow control.

Interpretability: Unlike neural networks, the resulting controllers are explicit programs that fluid dynamicists can inspect, modify, and transfer.
Competitive Performance: The method does not sacrifice performance for interpretability, matching or beating DRL in the majority of tested scenarios.
Physical Insight: The search process naturally recovers physically meaningful mechanisms (e.g., transport delays, local feedback normalization) without human prescription.
Protocol Importance: The paper emphasizes that rigorous experimental protocols—specifically constraining agents to public interfaces and identical budgets—are essential to prevent agents from exploiting simulator loopholes rather than discovering valid control strategies.

The work suggests a future where controller design combines the optimization power of modern AI with the interpretability and physical insight traditionally sought in computational fluid dynamics.