Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reduction

This paper proposes an explainable multi-agent deep reinforcement learning framework that leverages SHAP-guided rewards to discover a highly energy-efficient control strategy for turbulent drag reduction, achieving a 34.44% drag reduction and 34.01% net energy saving with minimal actuation cost by activating pressure-gated controls in sync with near-wall turbulent structures.

The Gist

The Big Picture: Taming the "Turbulent Traffic"

Imagine a highway where cars (air or water molecules) are driving smoothly in lanes. But near the road surface (the "wall"), the traffic gets chaotic. Cars swerve, crash into each other, and create a messy, swirling traffic jam. This chaos creates drag—a force that slows everything down and wastes energy.

In the world of engineering, this is called turbulent drag. It accounts for about one-third of all the energy the world uses for transport (like ships and planes). The goal of this research is to teach a computer how to "traffic control" this chaos to make it smoother, using less energy than it costs to run the control system itself.

The Problem: The "Brute Force" Approach

For a long time, scientists tried to fix this by using a strategy called Opposition Control.

• The Analogy: Imagine a traffic cop standing on the side of the road. Whenever a car swerves left, the cop yells "Go right!" and pushes it back.
• The Flaw: This works okay, but it's exhausting. The cop has to shout constantly, using a lot of energy. Sometimes, the energy the cop spends shouting is almost as much as the fuel saved by the cars moving smoother.

Then, scientists tried Deep Reinforcement Learning (DRL). This is like hiring a super-smart AI traffic cop that learns by trial and error.

• The Success: The AI learned to stop the swerving cars much better than the human cop, reducing drag significantly.
• The New Problem: The AI was a "black box." It knew how to stop the cars, but we didn't know why. Also, the AI was still shouting (using energy) constantly, which ate up the savings.

The Solution: The "Sherlock Holmes" AI

The authors of this paper combined two things:

1. Multi-Agent DRL: Many tiny AI agents working together (one for every inch of the road).
2. Explainable AI (XDL): A tool called SHAP that acts like a magnifying glass, showing the AI exactly which parts of the flow are causing the most trouble.

Instead of just telling the AI "Stop the drag," they gave the AI a new instruction: "Look at the clues that tell us where the drag is coming from, and only act on those specific clues."

They tested three different "clue books" (reward strategies) for the AI:

1. The Velocity Book: Look at how fast the air is moving. (This was the old method).
2. The Friction Book: Look specifically at the "rubbing" force (skin friction) on the wall.
3. The Pressure Book: Look at the "pushing" force (pressure fluctuations) on the wall.

The Winning Strategy: The "Silent Gatekeeper"

The researchers found that the best strategy was a combination of the Friction and Pressure books.

Here is what happened when they used this new strategy:

• The Old AI (Brute Force): It was like a frantic security guard running back and forth, pushing people left and right constantly. It used a lot of energy (5.90% of the total energy budget).
• The New AI (SHAP cf + pw): It became a Silent Gatekeeper.
  • The Discovery: The AI learned that it didn't need to push constantly. It only needed to act when the "pressure" on the wall was near zero.
  • The Metaphor: Imagine a bouncer at a club. Instead of yelling at everyone all night, the bouncer only steps in when the music stops (near-zero pressure) to gently guide a few people.
  • The Result: The AI stopped acting constantly. It waited for the perfect moment to make a tiny, precise adjustment.

The Results: Smarter, Not Harder

The new method achieved amazing results compared to the old methods:

• Drag Reduction: It reduced the "traffic jam" (drag) by 34.4%. This is better than the old AI and much better than the human cop.
• Energy Savings: Because the AI stopped shouting constantly, it used only 0.43% of the energy budget to do its job.
• Net Gain: The "Net Energy Saving" (the actual fuel saved after paying the AI's energy bill) jumped by nearly 50% compared to the old AI.

Why It Works: The "Ghost" Timing

The paper explains that the near-wall turbulence has a natural "heartbeat" or rhythm. The old AI tried to fight this rhythm by acting every single second, which was wasteful.

The new AI, guided by the "Pressure and Friction" clues, learned to sync with the heartbeat.

• The Analogy: Imagine trying to stop a swinging pendulum. If you push it every time it moves, you waste energy. But if you wait until it reaches the very top of its swing (where it pauses for a split second) and give it a tiny nudge, it stops with almost no effort.
• The new AI learned to wait for that "pause" (near-zero pressure) and act on the same timescale as the turbulence itself.

Summary

The paper shows that by teaching an AI to look at the right clues (friction and pressure) rather than just the speed, we can create a control system that is:

1. More effective at stopping drag.
2. Much cheaper to run (using 14 times less energy than previous AI methods).
3. Smarter about when to act, waiting for the perfect moment rather than acting constantly.

It's the difference between a frantic guard shouting all night and a calm, observant expert who knows exactly when to step in to save the day.

Technical Summary

Technical Summary: Explainable Deep Reinforcement Learning for Turbulent Drag Reduction

Problem Statement
Skin-friction drag in wall-bounded turbulent flows constitutes approximately one-third of global transport energy consumption. While active flow control strategies, such as opposition control, target the near-wall self-sustaining cycle to disrupt drag-generating structures, they face two primary limitations: performance degradation at higher Reynolds numbers and high energetic costs. Specifically, the power required for actuation can negate the energy saved by drag reduction, often resulting in negligible or negative net energy saving (NES). Although Deep Reinforcement Learning (DRL) has demonstrated superior drag reduction capabilities compared to classical methods, standard DRL policies often remain "opaque," failing to identify which flow structures drive the control, and frequently incur high actuation costs that compromise energy efficiency.

Methodology
The authors propose a framework combining Multi-Agent Deep Reinforcement Learning (MARL) with Explainable Deep Learning (XDL) to address these limitations. The core innovation lies in using SHapley Additive exPlanations (SHAP) not merely for post-hoc analysis, but as the direct reward signal for the control policy.

• Framework: The study utilizes a Multi-Agent DRL setup where 256 agents (in the training domain) control wall-normal blowing and suction. The agents use the Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm.
• Explainable Reward Mechanism: Instead of rewarding agents directly for minimizing wall-shear stress (the standard approach), the authors train auxiliary U-nets to predict specific flow quantities. SHAP values are computed to determine the contribution of local flow states to these predictions. The reward is defined as the negative domain-averaged magnitude of the SHAP attribution vector field. By minimizing this magnitude, the policy suppresses the coherent structures deemed most relevant to the prediction target.
• Configurations: Five strategies are compared:
  1. Opposition Control (OPP): A classical baseline.
  2. WSE: Direct minimization of wall-shear stress (standard DRL).
  3. SHAP vel: SHAP attributions derived from a U-net predicting the future velocity field (reproducing prior work).
  4. SHAP cf: SHAP attributions derived from a U-net predicting the skin-friction coefficient ($c_f$).
  5. SHAP cf + pw: A combined approach using SHAP attributions from two U-nets predicting the skin-friction coefficient and wall-pressure fluctuations ($p_w$), respectively. The attribution surrogates are merged via parameter-space interpolation.
• Simulation Setup: Training occurs in a Small Channel Configuration (SCC) with $Re_\tau = 180$, while policy inference is tested on 50 unseen initial conditions in a Large Channel Configuration (LCC).

Key Results
The combined SHAP cf + pw strategy achieved the best overall performance, outperforming all other methods in both drag reduction and energy efficiency:

• Performance Metrics: The SHAP cf + pw policy achieved a Drag Reduction (DR) of 34.44% and a Net Energy Saving (NES) of 34.01%.
• Comparison to Baselines:
  • Compared to the direct wall-shear stress baseline (WSE), the proposed strategy improved DR by 49.41% and NES by 48.52%, while simultaneously reducing the normalized actuation cost from 5.90% to 0.43%.
  • Compared to Opposition Control, DR increased by 49.41% and NES by 48.52%.
• Actuation Characteristics: Analysis of the control signals revealed a distinct "pressure-gated" mechanism. Unlike the WSE and SHAP vel policies, which actuate in large, high-amplitude patches across the full range of wall pressures, the SHAP cf + pw policy acts predominantly at near-zero wall pressure ($p_w \approx 0$) with low amplitude.
• Temporal Dynamics: The actuation signal of the SHAP cf + pw policy exhibits a smooth temporal autocorrelation with an integral time scale ($\tau^+_{int} \approx 5.1$), which is roughly three times longer than other DRL policies and comparable to the lifetime of near-wall quasi-streamwise vortices. This suggests the controller operates on the timescale of the turbulent structures rather than reacting instantaneously at every control step.

Significance and Claims
The paper claims that aligning the SHAP attribution target with the specific control objective (skin friction) and augmenting it with wall-pressure fluctuations reconciles the trade-off between high drag reduction and low actuation cost.

• Emergent Efficiency: The energy-efficient "pressure-gated" behavior was not explicitly programmed into the reward function but emerged naturally from the choice of the attribution target (predicting $c_f$ and $p_w$). This identifies the attribution target as a critical, previously unexploited design choice in XDRL-guided control.
• Transferability: The authors posit that this principle—aligning the target variable with the control objective—offers a transferable strategy that could be tested at higher Reynolds numbers and different geometries.
• Mechanism: The results suggest that the most energetically efficient policy targets the regeneration cycle of the near-wall turbulence (by acting on the timescale of the structures and gating by pressure) rather than simply suppressing the instantaneous footprint of the flow.

The study concludes that by leveraging explainable AI to guide the reward signal, it is possible to discover control policies that match the energy efficiency of classical opposition control while retaining the superior drag reduction capabilities of deep reinforcement learning.