🤖 machine learning

Adversarial Sensor Errors for Safe and Robust Wind Turbine Fleet Control

This paper proposes an adversarial training framework for wind turbine fleet control that utilizes an "Arms Race" approach to co-train a central controller and an adversary, successfully reducing worst-case performance degradation from a 39% power loss to a 7.9% power gain compared to baseline strategies.

Original authors: Julian Quick, Marcus Binder Nilsen, Andreas Bechmann, Tran Nguyen Le, Pierre-Elouan Mikael Rethore

Published 2026-04-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Julian Quick, Marcus Binder Nilsen, Andreas Bechmann, Tran Nguyen Le, Pierre-Elouan Mikael Rethore

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a wind farm not as a collection of individual machines, but as a highly choreographed dance troupe.

In a traditional setup, each dancer (turbine) tries to spin as fast as possible on their own. But in a wind farm, the "wind" one dancer feels is actually the messy wake left behind by the dancer in front of them. If they all just spin wildly, they end up stepping on each other's toes, and the whole group produces less energy.

To fix this, scientists are teaching these turbines to dance together using a central "choreographer" (a central controller). This choreographer tells the turbines exactly how to turn (yaw) to catch the wind perfectly while avoiding the turbulence of their neighbors.

The Problem: The "Fake News" Sensor

Here's the catch: The choreographer relies on sensors to know where the wind is blowing.

Real-world issue: Sensors sometimes glitch or drift (like a compass spinning in a storm).
The scary scenario: A hacker could hack into the system and feed the choreographer fake data, telling the turbines the wind is coming from the North when it's actually coming from the East. If the turbines turn to face the fake wind, they stop producing power or even damage themselves.

The researchers wanted to know: How do we train a choreographer that won't be fooled by a liar?

The Solution: The "Arms Race" Training Camp

Instead of just teaching the controller to handle random sensor glitches (like static on a radio), the researchers decided to train it against a professional liar.

They set up a digital training camp with three different ways to practice:

The "Noise" Method (The Old Way): They just added random static to the sensors. It's like practicing dance while wearing blindfolds that wiggle randomly. It helps, but it's not very realistic.
The "Self-Play" Method: The controller and the liar train at the same time, constantly trying to outsmart each other in a single loop.
The "Arms Race" Method (The Winner): This is the paper's big discovery. Imagine a series of boxing matches.
- Round 1: The Controller fights a weak Liar. The Controller wins.
- Round 2: The Controller fights a Liar that learned from the previous match. The Liar gets smarter.
- Round 3: The Controller fights an even smarter Liar.
- The Twist: The Controller only trains against the current Liar. It doesn't look back at the old ones. It's a relentless, escalating battle where both sides get stronger and smarter with every generation.

The Results: Who Won the Dance?

The researchers put these trained controllers to the test against the most dangerous liars they could create.

The "Noise" Controller: When faced with a sophisticated hacker, this controller panicked. It thought the wind was coming from everywhere at once and spun the turbines in the wrong direction. Result: It lost 39% of its power compared to doing nothing. It was worse than having no controller at all!
The "Arms Race" Controller: This controller had been through the wringer. It had fought liars that were getting smarter every day. When the final hacker tried to trick it, the controller didn't panic. It realized, "Wait, this data looks suspicious," and adjusted its moves. Result: It actually gained 7.9% more power than the baseline, even under attack!

The Takeaway

Think of it like training a security guard.

If you only train them to spot people wearing red hats (random noise), they will fail when a hacker wears a blue hat or a mask.
But if you train them by having a master thief try to sneak past them every single day, the guard learns to spot any trick, not just the red hats.

In simple terms: By forcing the wind farm controller to fight against a constantly evolving, intelligent "enemy" (the adversarial agent), the researchers created a system that is incredibly tough. Even if a hacker tries to trick the sensors, the wind farm keeps dancing perfectly and generating power.

This is a huge step forward because as wind farms get bigger and more connected, they become bigger targets for hackers. This "Arms Race" training ensures that even in a worst-case scenario, the wind farm doesn't just survive—it keeps thriving.

1. Problem Statement

Wind farm fleet control aims to maximize total power production by coordinating individual turbines, accounting for wake effects where upstream turbines impact downstream ones. While Reinforcement Learning (RL) offers promising data-driven control strategies, these systems face two critical vulnerabilities:

Sensor Uncertainty: Real-world sensors have inherent noise and biases. Standard procedural noise models (e.g., independent Gaussian noise with fixed bias) may fail to capture complex, time-correlated, or worst-case error realizations.
Cybersecurity Threats: Malicious actors could hack telemetry systems to alter sensor readings (wind speed, direction, power), potentially degrading fleet performance or causing shutdowns.

The core research question is: Does training a wind farm controller (the "protagonist") through interaction with an adversarial agent (the "adversary") yield greater robustness than training with standard procedurally generated noise?

2. Methodology

2.1. Simulation Environment

Physics Model: The study uses the Dynamic Wake Meandering (DWM) model implemented in Dynamiks software.
Setup: A two-turbine wind farm (Vestas V80) aligned East-West, spaced 7 rotor diameters apart.
Conditions: Inflow wind speeds of 6–7 m/s and directions of 267–273°. Simulations run at 5-second time steps; control actions update every 10 seconds. Turbulence is excluded to isolate large-scale wake dynamics.

2.2. Agent Definitions

Protagonist (Controller): An RL agent trained to maximize total farm power relative to a baseline greedy strategy. Actions are changes in turbine yaw offsets.
- Observation: Noisy sensor data ( $\hat{x} = x_{true} + \epsilon$ ).
- Reward: Relative power gain over a baseline no-steering case.
Adversary: An RL agent trained to inject measurement errors ( $\epsilon$ $ϵ$ ) into the protagonist's sensors to minimize farm power.
- Constraints: Errors must be physically plausible (bounded bias and rate of change). The adversary observes the true state but manipulates the protagonist's observed state.
- Objective: Zero-sum game; the adversary's reward is the negative of the protagonist's reward.
Expert Baseline: A PyWake agent using a deterministic optimization tool based on sensed wind direction. It serves as a conservative benchmark; adversarial errors can degrade it to zero-yaw behavior but cannot force physically impossible counter-productive actions (e.g., 360° turns) due to safety constraints.

2.3. Training Paradigms

The paper compares three co-training approaches for the protagonist and adversary:

Arms Race: A sequential iteration where Protagonist $N$ trains exclusively against Adversary $N-1$ . Prior opponents are discarded.
Synthetic Self-Play (SSP): Protagonist $N$ trains against a "zoo" of all prior adversaries ($0$ to $N-1$ ) plus procedural noise, sampled uniformly.
Self-Play: A concurrent optimization where the protagonist and adversary co-evolve simultaneously in a single competitive loop (using the PettingZoo library).

2.4. Algorithm & Hyperparameters

Algorithm: Proximal Policy Optimization (PPO).
Network Architecture: Multi-Layer Perceptrons (MLP) with two hidden layers (128 units) and tanh activation.
Noise Models:
- Procedural: Gaussian noise + persistent episodic bias.
- Adversarial: Time-dependent error injection with bounded changes (max 1/10th of allowable bias per step).

3. Key Contributions

First Application of Adversarial RL to Wind Farm Sensors: This work is the first to apply adversarial agent training specifically to enhance wind farm sensor robustness against both random errors and malicious attacks.
Comparative Framework: It establishes a rigorous comparison between "Arms Race," "Synthetic Self-Play," and "Self-Play" training paradigms in the context of wind energy.
Open-Source Implementation: The framework utilizes the open-source WindGym package, making the methodology reproducible.
Addressing Catastrophic Forgetting: The study explicitly investigates how different training methods handle "catastrophic forgetting" (loss of performance in nominal/clean conditions while training for robustness).

4. Results

4.1. Training Performance & Convergence

Arms Race: Showed signs of instability and "catastrophic forgetting." Protagonists trained in later iterations often performed poorly in clean environments, having "forgotten" how to operate without adversarial noise. However, the adversaries became increasingly potent.
Synthetic Self-Play (SSP): Produced the highest performance in clean and procedural noise environments initially but suffered from forgetting after iteration 8.
Self-Play: Converged to a state where the adversary consistently dominated the protagonist, indicating the agents did not find a stable equilibrium (Nash equilibrium) within the training window.

4.2. Robustness Analysis (The "Gauntlet" Test)

The most critical finding comes from cross-evaluating the best agents from each method against a wide range of adversaries:

Arms Race Superiority: Despite training instabilities, the Arms Race approach produced the most robust protagonist.
- Worst-Case Performance: The Arms Race controller maintained a +7.9% power gain over the baseline even under the worst-case adversarial attack.
- Comparison: In contrast, the procedurally trained controller suffered a -39% power loss (performing worse than doing nothing) under similar attacks.
- Adversary Strength: The Arms Race also generated the most formidable adversary, capable of confounding expert controllers significantly.
Behavioral Analysis:
- Procedural Controller: When attacked, it was easily confounded into requesting extreme yaw offsets (potentially damaging the turbine) based on spoofed zero-power readings.
- Arms Race Controller: Demonstrated resilience, correctly interpreting the flow dynamics despite sensor spoofing, resulting in a 15.76% power gain over the no-steering baseline during specific attack scenarios.

5. Significance and Conclusion

This paper demonstrates that adversarial training is superior to procedural noise training for developing safe and robust wind farm controllers.

Security Implications: As wind farms move toward centralized control, the risk of cyberattacks increases. This framework provides a method to "harden" controllers against such threats before deployment.
Operational Safety: By training against worst-case sensor errors, the resulting controllers avoid catastrophic failures (like extreme yaw misalignment) that could occur if trained only on nominal or simple noise data.
Future Work: The authors note that while the Arms Race approach yielded the best results in this small-scale (2-turbine) study, further investigation is needed to scale these methods to larger farms and to address the computational costs of extensive comparative analysis.

Final Takeaway: Training wind farm controllers in an "arms race" against an adaptive adversary creates a controller that is significantly more resilient to sensor manipulation and uncertainty than those trained with standard noise models, ensuring higher efficiency and safety in real-world operations.