🔬 physics

Gradient-free learning of a closed-loop wall controller for turbulent drag reduction

This paper presents the first application of Evolution Strategy to turbulent flow control, demonstrating that a gradient-free recurrent controller trained directly on a large domain at $Re_\tau \simeq 180$ achieves a 26% skin-friction drag reduction by reorganizing near-wall turbulence through streamwise velocity correlations, outperforming both previous gradient-based multi-agent reinforcement learning and classic opposition control.

Original authors: Giorgio Maria Cavallazzi, Miguel Pérez Cuadrado, Alfredo Pinelli

Published 2026-07-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Giorgio Maria Cavallazzi, Miguel Pérez Cuadrado, Alfredo Pinelli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a river of air rushing through a long, smooth tunnel. Even though the walls look smooth, the air right next to them is a chaotic mess of tiny, swirling tornadoes. These invisible whirlpools rub against the walls, creating "skin friction"—a kind of sticky drag that makes it hard to push the air through. If you could calm these whirlpools, the air would glide much more easily, saving a massive amount of energy.

For a long time, scientists have tried to teach computers how to "push back" against these whirlpools to reduce drag. One popular method, called Opposition Control, is like a reflex. If a sensor detects air moving up toward the wall, the computer instantly blows air down to cancel it out. It's a simple, physical rule: "Do the opposite of what you feel." This method works pretty well, cutting drag by about 22.5%.

But recently, researchers tried something fancier: they used a "learning" computer (a type of artificial intelligence) to figure out the best way to push back, without giving it any rules. They trained this AI on a tiny, miniature version of the tunnel. The AI learned a strategy that looked great in the small box, but when they tried to use it in a big, real-sized tunnel, it fell apart. The AI got confused, started pushing too hard in a jerky, on-off way (like a light switch being slammed), and the drag reduction dropped to just 17.4%. It turns out, the tiny training box tricked the AI into thinking the whole world was small.

The Big Discovery
In this new study, the researchers tried a different approach. Instead of teaching the AI by showing it mistakes and correcting them step-by-step (which is how the previous method worked), they used a technique called Evolution Strategy.

Think of this like breeding champion racehorses. Instead of coaching one horse, you start with a whole stable of slightly different horses. You let them all run a full race (a complete simulation of the airflow) and see which ones finish fastest. You don't tell them how to run; you just pick the winners, mix their traits, and create a new generation of horses that are a bit better. You repeat this over and over.

Using this "breeding" method, the researchers trained their controller directly on the large, full-sized tunnel (where the friction Reynolds number is Re𝜏≃180). They didn't use a tiny box as a practice ground. They let the computer evolve a strategy that worked specifically for the big, messy reality.

The Results
The result was a new controller that reduced drag by 26.1%.

This is better than the old "Opposition Control" reflex (22.5%).
It is much better than the AI trained on the tiny box (17.4%).
It is the first time this specific "breeding" method has been used to control turbulent flow in a simulation.

How It Works (The Twist)
Here is the surprising part: even though the new controller and the old "Opposition Control" both reduced drag by similar amounts, they did it in completely different ways.

The Old Way (Opposition Control): The computer watched the air moving up and down (wall-normal velocity) and pushed back against that motion. It was like trying to stop a wave by pushing down on the water.
The New Way (Evolution Strategy): The computer ignored the up-and-down motion mostly. Instead, it learned to track the long, streaky lines of fast and slow air moving along the tunnel (streamwise velocity). It acted like a gardener trimming long, unruly vines.

The simulations show that the new controller breaks those long, lazy streaks of air into shorter, choppy fragments. The old controller just tried to flatten them. Both methods calm the turbulence enough to reduce drag, but they rearrange the chaos in the "buffer layer" (the zone right next to the wall) in totally different patterns.

Why This Matters
The researchers are careful to point out that this is a simulation, not a real wind tunnel test yet. However, the simulation shows that the new controller is smooth and steady; it doesn't get stuck in a jerky, "on-off" panic like the old AI did.

The main takeaway is that you don't have to follow the same physical rules as the old methods to get good results. By letting the computer evolve a strategy directly on the big, difficult problem, it found a secret shortcut: instead of fighting the up-and-down motion, it learned to dance with the side-to-side streaks. It's a reminder that sometimes, the smartest way to solve a messy problem isn't to fight it head-on, but to find a completely different rhythm that the problem didn't expect.

Technical Summary: Gradient-free learning of a closed-loop wall controller for turbulent drag reduction

Problem Statement
Closed-loop wall control for turbulent drag reduction, particularly when employing multi-agent reinforcement learning (MARL), faces significant limitations when transitioning from training to application. Current gradient-based policies are typically trained on small, periodic minimal flow units (MFUs). When transferred to larger domains, these policies often exhibit reduced performance. Furthermore, such policies are prone to "saturated bang-bang" actuations that collapse into standing streamwise waves. These degenerate behaviors are artifacts where the control scale is dictated by the computational box size rather than the physics of the near-wall cycle. Additionally, the zero-net-mass constraint required for wall actuation corrupts per-patch credit assignment in gradient-based methods, necessitating a central critic that struggles to resolve joint action spaces in larger domains.

Methodology
The authors propose the application of Evolution Strategy (ES) to optimize a recurrent closed-loop controller directly on a large, fully turbulent channel at a friction Reynolds number of $Re_\tau \approx 180$ .

Flow Configuration: Direct Numerical Simulations (DNS) are performed in a plane channel with dimensions $(L_x^+, L_y^+, L_z^+) \approx (1922, 576, 360)$ . Control is applied via wall-normal blowing and suction ( $w_w$ ) with an instantaneous zero-net-mass constraint enforced via a projection layer. Sensing occurs at a detection plane $z_d^+ \approx 14$ .
Controller Architecture: A recurrent neural network (RNN) maps the velocity field at the detection plane to a wall actuation field. The architecture is identical to a previously studied GRU-MARL controller but is optimized differently.
Optimization Strategy: Unlike gradient-based methods, the ES treats the simulation and network as a black box. It optimizes the policy parameters ( $\theta$ ) by evaluating a population of perturbed parameter vectors over complete flow episodes. The fitness function is an energy-aware criterion: $F(\theta) = (\varepsilon_0 - \varepsilon(\theta)) / \varepsilon_0$ , where $\varepsilon$ is the total dissipation including pumping power and wall power.
Update Mechanism: The population mean is updated based on the relative ranking of fitness values within the generation, using mirrored Gaussian perturbations. This approach eliminates the need for backpropagation through the DNS solver or a central critic, allowing parallel evaluation of independent episodes on the full domain.

Key Contributions and Results

First ES Application to Turbulent Flow: To the authors' knowledge, this is the first application of an evolution strategy to the control of a turbulent flow.
Superior Drag Reduction: The ES-optimized controller achieves a skin-friction drag reduction of approximately 26.1%. This exceeds the performance of:
- The same architecture trained via gradient-based MARL on a minimal box and transferred to the large domain (17.4%).
- Classic opposition control (OC) (22.5%).
  The results demonstrate that the drag reduction lost when transferring a gradient-based policy from a minimal box to a large domain is recovered when the policy is optimized directly on the large domain.
Distinct Control Mechanisms: Despite achieving comparable drag reduction, the ES controller and Opposition Control (OC) operate through fundamentally different physical mechanisms:
- Reynolds Stress Reorganization: Both controllers reduce the weighted Reynolds shear stress, but they follow separate trajectories through the buffer layer, resulting in different anisotropy invariants and stress profiles.
- Actuation Correlation: While OC acts against the wall-normal velocity fluctuations ( $w'$ ), the ES actuation correlates predominantly with streamwise velocity fluctuations ( $u'$ ). The ES actuation aligns with streamwise low- and high-speed streaks rather than the sweeps and ejections targeted by OC.
- Streak Modification: OC primarily lowers the amplitude of near-wall streaks. In contrast, the ES controller breaks long streamwise streaks into shorter fragments, redistributing streamwise energy to smaller scales.
Avoidance of Degeneracy: The ES controller produces smooth, unsaturated actuation that tracks the evolving flow, avoiding the "bang-bang" saturation and standing wave artifacts observed in some gradient-based policies.

Significance
The paper claims that the ES framework successfully addresses the credit assignment issues inherent in zero-net-mass constrained multi-agent control by evaluating complete episodes rather than individual patches. By optimizing directly on the large evaluation domain, the controller avoids the domain-size mismatch that plagues transfer learning from minimal boxes. The study demonstrates that high-performance drag reduction can be achieved without relying on specific physical priors (like opposition control), and that different control strategies can achieve similar drag reduction metrics through distinct reorganizations of the near-wall turbulence cycle. The smoothness and interpretability of the ES controller suggest it reflects the controlled turbulence physics rather than numerical artifacts of the training domain.

More like this