Acoustics-based Active Control of Unsteady Flow Dynamics using Reinforcement Learning Driven Synthetic Jets

This paper presents a deep reinforcement learning framework that utilizes far-field acoustic measurements as the primary feedback signal to drive synthetic jet actuation, successfully suppressing unsteady wake dynamics behind a circular cylinder and achieving significant reductions in radiated noise and drag without relying on traditional velocity or pressure sensors.

The Gist

Imagine you are standing next to a flagpole on a windy day. The wind doesn't just blow past the pole; it creates a rhythmic "flapping" sound and causes the pole to shake. In physics, this is called a "wake," where the air swirls into spinning vortices (like tiny tornadoes) that create drag (slowing things down) and noise.

For decades, engineers have tried to stop this shaking and noise. Usually, they do this by installing sensors that measure the wind speed or pressure right next to the pole to tell a computer how to fix it.

This paper introduces a new, clever idea: What if we just listened to the noise instead of measuring the wind?

Here is a simple breakdown of how the researchers did it, using everyday analogies:

1. The Problem: The Shaky Pole

The researchers simulated wind blowing past a round cylinder (like a pipe or a flagpole). As the wind hits it, it creates a "vortex street"—a line of spinning air bubbles that detach from the top and bottom. This causes two bad things:

• Drag: The object gets pushed back harder.
• Noise: The spinning air creates a humming sound (like a whistle).

2. The Solution: The "Smart Ear" and the "Artificial Lungs"

Instead of using complex wind sensors, the team used a Deep Reinforcement Learning (DRL) agent. Think of this agent as a super-smart student who is learning to play a video game.

• The "Ears" (Feedback): Instead of looking at the wind, the agent "listens" to the sound pressure (noise) created by the swirling air using a virtual microphone array placed downstream.
• The "Lungs" (Actuation): The cylinder has two tiny "mouths" (synthetic jets) on its top and bottom. These can blow air out or suck air in, acting like artificial lungs that can puff or inhale to change the wind's path.

3. The Learning Process: Trial and Error

The AI agent didn't know the rules of physics at first. It had to learn by doing, similar to how a baby learns to walk by falling and trying again.

• The Goal: The agent's only instruction was: "Make the noise quieter."
• The Strategy: The agent would puff air from the top or bottom jets. If the noise got quieter, it got a "reward" (like a high score in a game). If the noise got louder, it got a penalty.
• The Discovery: Through thousands of tries, the AI figured out exactly when and how hard to puff the air to cancel out the spinning vortices before they could grow loud and cause shaking.

4. The Results: Quieter and Smoother

The paper reports that this "listening" approach worked surprisingly well. By simply reacting to the sound:

• Noise Reduction: The "humming" of the wind dropped by about 9.5%.
• Drag Reduction: The force pushing back against the cylinder dropped by 23.8%.
• Stability: The violent shaking (oscillations) of the wake was significantly calmed down.

The Big Takeaway

The paper claims that you don't need to see the wind to control it; you just need to hear it. By using sound as the primary signal, the AI learned to "tune" the airflow like a musician tuning an instrument, turning a chaotic, noisy, and drag-heavy flow into a smooth, quiet, and efficient one.

In short: They taught a computer to "listen" to the wind's complaints and "blow" just the right amount of air to make it stop complaining, resulting in a quieter and more efficient flow.

Technical Summary

Technical Summary: Acoustic Flow Control using Reinforcement Learning

Problem Statement
Active Flow Control (AFC) has traditionally relied on state variables such as velocity, pressure, or vorticity as feedback signals to regulate unsteady flow dynamics. However, these methods often require complex state-space approximations and transfer function calculations that are case-dependent and non-generalizable. Furthermore, existing Deep Reinforcement Learning (DRL) studies in fluid dynamics predominantly utilize velocity-field observations or force coefficients as feedback. This paper addresses the gap in utilizing acoustic observables as the primary control signal. The authors posit that since vortex shedding is the source of flow-generated noise (as described by Lighthill's Acoustic Analogy), the acoustic signature of the wake contains sufficient information to control the flow. The specific problem investigated is the suppression of unsteady wake dynamics and the mitigation of flow-induced disturbances behind a circular cylinder at a Reynolds number of 100, using sound as the sole feedback mechanism.

Methodology
The study employs a Deep Q-Network (DQN) framework integrated with synthetic jet actuation on a circular cylinder.

• Simulation Setup: The computational domain is based on the DFG 2D-3 benchmark (flow past a cylinder) using the DOLFINx solver. The flow is incompressible with a Reynolds number ($Re$) of 100.
• Actuation: Two synthetic jets are positioned on the top and bottom surfaces of the cylinder (perpendicular to the flow). These jets operate via blowing and suction, controlled independently by the DRL agent.
• Feedback Formulation (The Novelty): Instead of measuring velocity or vorticity directly, the system uses Sound Pressure Levels (SPL) derived from hydrodynamic pressure fluctuations. A dense array of 2,500 sensors (2,000 horizontal, 500 vertical) surrounds the cylinder to capture pressure fields. The effective SPL ($SPL_{eff}$) is calculated as a root-mean-square value of the pressure fluctuations relative to the average pressure. This SPL serves as the state input for the DQN agent.
• DRL Architecture: The agent utilizes a neural network with four fully connected layers (50 nodes each) and ReLU activation. The input is a 4-dimensional vector comprising the SPLs from the upper and lower sensor arrays and the current velocities of the two jets. The output is the control action (adjustment of jet velocities).
• Training Strategy: The simulation is divided into two stages:
  1. Exploration: The agent explores a wide range of jet velocity increments ($\pm0.01$ to $\pm0.5$) to identify optimal control patterns.
  2. Testing: The agent operates with a fixed set of discrete jet velocity values to verify stability and performance.
• Reward Function: The reward is strictly tied to the reduction of SPL. High rewards are given for SPL values below 66 dB, while penalties are applied for values exceeding 74.5 dB.

Key Contributions

1. Acoustic-Driven Control Framework: This work establishes a direct link between far-field acoustic emissions and near-field actuation, demonstrating that acoustic sensing alone can provide sufficient information for effective closed-loop flow control without explicit velocity or vorticity sensing.
2. DRL for Acoustic Regulation: It introduces a DQN-based strategy specifically tailored to minimize Sound Pressure Levels generated by vortex shedding, treating noise reduction as the primary objective for flow stabilization.
3. Quantitative Performance: The study provides quantitative evidence that acoustic feedback can simultaneously reduce radiated noise and aerodynamic drag, outperforming baseline uncontrolled flows and showing competitive results against velocity-based DRL configurations.

Results
The simulations ran for 20 seconds (10,000 time steps) at a Reynolds number of 100.

• Noise Reduction: The control strategy successfully reduced the radiated noise. The baseline SPL was approximately 73.5 dB.
  • Test Case 1 (lower jet velocities) reduced the SPL to 69.0 dB (6.1% reduction).
  • Test Case 2 (higher jet velocities) reduced the SPL to 66.5 dB (9.5% reduction).
• Drag Reduction: The method also significantly mitigated the drag coefficient ($C_D$).
  • The baseline mean $C_D$ was 3.15.
  • Test Case 1 reduced $C_D$ to 2.65 (15.9% reduction).
  • Test Case 2 reduced $C_D$ to 2.40 (23.8% reduction).
• Wake Stabilization: The control led to a marked attenuation of wake oscillations. The standard deviation of the lift coefficient and the time-averaged fluctuation of flow speed in the downstream region were significantly reduced, indicating the suppression of coherent wake structures.
• Validation: Mesh independence and time-step sensitivity analyses confirmed the reliability of the results. Statistical convergence was achieved after approximately 20 vortex shedding cycles.

Significance and Claims
The paper claims that this research demonstrates the feasibility of using acoustically informed reinforcement learning for adaptive wake stabilization under nonlinear flow conditions. The authors assert that their findings highlight the potential of acoustic sensing as a non-intrusive feedback modality for coupled aerodynamic and aeroacoustic optimization in bluff-body flows.

The study suggests that controlling the "source" of noise (vorticity) via acoustic feedback is a logical and effective approach for flow control problems. While the authors acknowledge that the current framework does not seek globally optimal control in the strict mathematical sense, they conclude that the DRL-guided strategy consistently identifies dynamically effective actuation regimes. The work serves as a stepping stone toward integrating machine learning techniques to enhance the efficiency of active flow control systems, with potential implications for reducing flow-induced vibrations and drag in engineering applications such as marine structures and aircraft. The authors explicitly state that the codes are available to facilitate further research in this domain.