High-lift Wing Separation Control via Bayesian Optimization and Deep Reinforcement Learning

This study demonstrates that while open-loop Bayesian optimization successfully improved aerodynamic efficiency by 10.9% through steady jet control on a 30P30N high-lift wing, closed-loop deep reinforcement learning yielded negligible gains due to a penalty-dominated reward function that constrained exploration.

The Gist

Imagine you are trying to fly a heavy airplane. To take off and land safely, the wings need to generate a lot of lift. To do this, engineers use "high-lift" wings, which are like wings with extra flaps and slats (small movable pieces) that pop out to change the wing's shape.

However, at steep angles (like when a plane is climbing steeply or landing), the air flowing over these extra pieces can get messy and separate from the surface, causing the plane to "stall" (lose lift). This is like trying to run through a thick crowd; if you move too fast or at the wrong angle, people bump into you, and you can't move forward efficiently.

This paper is a study by a team of researchers who wanted to fix this "messy air" problem using two different smart strategies. They used a super-advanced computer simulation (like a virtual wind tunnel) to test their ideas on a specific wing design called the "30P30N."

Here is how they tried to solve the problem, explained simply:

The Tool: "Synthetic Jets"

Instead of using big mechanical flaps, the researchers used tiny, invisible "breaths" of air. Imagine blowing a steady stream of air through tiny holes on the wing surface. These are called synthetic jets. They don't add extra air to the system (they just move it around), but they can smooth out the messy airflow, keeping the air glued to the wing so the plane doesn't stall.

Strategy 1: The "Smart Searcher" (Bayesian Optimization)

The first method is like a very organized treasure hunter.

• How it works: The computer tries different combinations of blowing air from the front, middle, and back of the wing. It doesn't just guess randomly; it uses a mathematical map to learn from each attempt. If a certain setting works well, it looks nearby for even better settings.
• The Result: This method was very successful. It found a specific, steady "breathing" pattern that made the wing 11% more efficient.
• What happened: It mostly worked by sucking air in at the front part of the wing (the slat), which smoothed out the flow and reduced drag (air resistance). It was like finding the perfect rhythm to walk through that crowded room without bumping into anyone.

Strategy 2: The "Video Game Player" (Deep Reinforcement Learning)

The second method is like training a video game character (an AI agent) to play a flight simulator.

• How it works: This AI gets real-time updates from sensors on the wing (like a player seeing the screen). It tries to adjust the air "breaths" instantly based on what the air is doing right now. The goal is to learn a complex, changing dance of air control that a human couldn't figure out.
• The Result: This method struggled. Even though the AI had access to instant data, it didn't improve the wing's performance much.
• Why it failed: The researchers realized the "score" they gave the AI was too strict. The AI was so afraid of making a mistake (like losing a tiny bit of lift) that it was afraid to try anything new. It got stuck in a safe, boring loop where it barely improved anything. It's like a student who is so afraid of getting a question wrong that they never raise their hand to try a harder answer.

The Big Lesson

The study found that:

1. The "Smart Searcher" (Bayesian Optimization) worked great. It found a simple, steady solution that made the wing fly much better with very few computer tests.
2. The "Video Game Player" (Deep Reinforcement Learning) didn't work well in this specific case. The computer was too expensive to run (it took two weeks of supercomputer time for one training session), and the AI's "rules" were too strict, preventing it from learning the best moves.

In short: For this specific wing problem, a methodical, steady search for the best setting worked better than a high-tech AI trying to react instantly. The researchers concluded that if we want to use these "video game" AI methods in the future, we need to give them better rules (rewards) and faster computers so they can actually learn to fly better.

Technical Summary

Technical Summary: High-lift Wing Separation Control via Bayesian Optimization and Deep Reinforcement Learning

Problem Statement
This study addresses the challenge of controlling flow separation on a 30P30N high-lift wing configuration, a benchmark geometry for multi-element wings used during take-off and landing. Specifically, the research targets the mitigation of stall at a high angle of attack ($\alpha = 23^\circ$) and a Reynolds number of $Re_c = 450,000$. At these conditions, the flow is dominated by shear-layer separation between the main and flap elements, which degrades aerodynamic performance. The objective is to apply Active Flow Control (AFC) using synthetic jets on the slat, main, and flap elements to delay separation and improve aerodynamic efficiency ($E = C_l/C_d$). The study compares two distinct optimization strategies: open-loop Bayesian Optimization (BO) and closed-loop Deep Reinforcement Learning (DRL).

Methodology
The research employs wall-resolved Large-Eddy Simulations (LES) using the in-house GPU-accelerated CFD solver SOD2D, which solves the filtered incompressible Navier-Stokes equations using a spectral element method. The computational domain spans $20c \times 20c \times 0.1c$ in the streamwise, cross-stream, and spanwise directions, respectively.

• Actuation: Synthetic jets are placed on the slat ($x/c = -0.050$), main element ($x/c = 0.725$), and flap ($x/c = 1.0$). The jets operate with zero net mass flux; the flap jet velocity is constrained to balance the mass flow of the slat and main jets.
• Bayesian Optimization (BO): This open-loop approach treats the jet velocities as fixed parameters. A Gaussian Process (GP) surrogate model approximates the relationship between control inputs and aerodynamic outputs. The objective function maximizes efficiency while imposing strict penalties for any reduction in lift or increase in drag relative to the baseline. The optimization iteratively samples the parameter space to find a steady-state optimal configuration.
• Deep Reinforcement Learning (DRL): This closed-loop approach utilizes a Multi-Agent Reinforcement Learning (MARL) framework. The agent, implemented as a neural network, receives instantaneous pressure data from 78 sensors per pseudo-environment (spanning three 3D domains) and outputs time-dependent jet velocities. The agent is trained using the Proximal Policy Optimization (PPO) algorithm. The reward function mirrors the BO objective but is calculated locally for each pseudo-environment and aggregated. The training environment runs 10 concurrent CFD simulations to accelerate data collection.

Key Contributions and Results

1. Baseline Validation: The uncontrolled LES setup was validated against literature data (Montalà et al.) obtained at a slightly higher Reynolds number ($750,000$). Two mesh resolutions were used: a coarse mesh ($p=2$) for optimization training and a fine mesh ($p=4$) for final evaluation. The fine mesh accurately captured wall-resolved statistics ($\Delta y^+ < 2$), while the coarse mesh, despite under-predicting drag due to lower resolution, successfully captured the lift coefficient and flow topology trends, validating its use for the optimization loop.

2. Bayesian Optimization Success: The BO framework successfully identified a steady jet configuration that improved aerodynamic efficiency by +10.9% on the fine mesh compared to the baseline. This improvement was achieved primarily through a 9.7% reduction in drag while maintaining lift ($+0.15\%$). Flow analysis revealed that suction on the slat thinned the boundary layer and eliminated the slat wake, driving the drag reduction. Although drag on the main and flap elements increased slightly due to a larger recirculation zone, the net effect was positive. The BO method required only 19 CFD evaluations to converge.

3. DRL Performance and Limitations: In contrast, the DRL agent, despite leveraging instantaneous flow information and a MARL framework for 3D actuation, achieved only marginal improvements. While the agent slightly reduced drag and maintained lift, the aerodynamic efficiency gain was negligible. Analysis of the training process indicated that the reward function was dominated by penalty terms (specifically the lift penalty). Because the agent could not fully eliminate the lift penalty (lift never exceeded the baseline), the efficiency term remained stagnant. The authors note that the "hard" constraints in the reward function likely restricted the agent's exploration of the control space, trapping it in a local optimum.

4. Computational Challenges: The study highlights the significant computational cost of high-Reynolds-number DRL. Training 990 MARL episodes required approximately two weeks on 30 nodes of the MN5 supercomputer (approx. 48,000 GPU-hours). This cost, driven by the small time steps required for LES at $Re_c = 450,000$, severely limited the ability to tune hyperparameters or test alternative reward formulations.

Significance and Claims
The paper claims that while BO is a robust and efficient tool for identifying steady, interpretable AFC strategies with few evaluations, DRL faces significant hurdles in high-Reynolds-number flow control. The primary significance of the work lies in demonstrating that:

• BO can effectively optimize steady synthetic jet actuation to mitigate stall and improve efficiency on complex 3D high-lift wings.
• For DRL, the design of the reward function is critical; penalty-dominated rewards can constrain exploration and prevent the discovery of performance-enhancing policies.
• Computational cost remains a major bottleneck for DRL-based flow control at high Reynolds numbers. The authors suggest that future progress depends on acceleration strategies such as surrogate modeling, transfer learning from lower Reynolds numbers, or increased parallelization, rather than simply scaling up current training protocols.

The study does not claim that DRL is superior to BO for this specific application; rather, it presents a comparative analysis showing that, under the current computational and reward constraints, BO outperformed the closed-loop DRL approach in terms of efficiency gains.