⚡ electrical engineering

The Wisdom of the Crowd: High-Fidelity Classification of Cyber-Attacks and Faults in Power Systems Using Ensemble and Machine Learning

This paper demonstrates that while offline accuracy is an unreliable indicator of field readiness, a multi-layer perceptron (MLP) outperforms ensemble methods in real-time, sub-cycle cyber-attack and fault classification for power systems by maintaining high coverage without the frequent abstentions observed in ensemble models.

Original authors: Emad Abukhousa, Syed Sohail Feroz Syed Afroz, Fahad Alsaeed, Abdulaziz Qwbaiban, Saman Zonouz, A. P. Sakis Meliopoulos

Published 2026-05-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Emad Abukhousa, Syed Sohail Feroz Syed Afroz, Fahad Alsaeed, Abdulaziz Qwbaiban, Saman Zonouz, A. P. Sakis Meliopoulos

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the power grid as a massive, complex orchestra. In the past, the instruments were heavy, predictable machines (like traditional power plants). But today, the orchestra is mixing in new, tricky instruments like solar panels and batteries (called "Inverter-Based Resources" or IBRs). These new instruments are great, but they behave differently and can sometimes sound like a mistake when they are actually working fine, or they can be tricked by a saboteur playing a fake note.

This paper is about building a super-smart "conductor" (a computer program) that can instantly tell the difference between:

A real problem: A broken wire or a short circuit (a "fault").
A hacker attack: Someone trying to trick the system with fake data (a "cyber-attack").
Just noise: Normal startup sounds that might look like a problem but aren't.

The Experiment: A High-Stakes Simulation

The researchers didn't just guess; they built a ultra-realistic digital twin of a power grid. They ran this simulation at a speed of 4,800 times per second (4.8 kHz), which is like watching a hummingbird's wings in slow motion. This allowed them to capture every tiny vibration and glitch.

They created a "training camp" with 12 different types of AI students (Machine Learning models). Some were "Ensemble" models (think of them as a committee of experts voting on a decision), and others were "Neural Networks" (think of them as a single, highly adaptable brain).

They fed these students a massive library of data containing:

Normal days: Just the grid humming along.
Disasters: Broken wires, short circuits, and ground faults.
Sabotage: Hackers pretending to be sensors, messing with GPS signals, or faking voltage readings.

The Big Surprise: The "Exam" vs. The "Live Show"

Here is where the story gets interesting. The researchers tested the students in two ways:

1. The Written Exam (Offline):
They gave the models a static test where they could look at all the data at once.

Result: Almost everyone got an A+ (99%+ accuracy). The "Committee" models (Ensembles) and the "Brain" models (Neural Networks) both looked like geniuses.

2. The Live Concert (Real-Time Streaming):
Then, they put the models in a live, fast-moving situation where data comes in one second at a time, and they have to make a decision right now.

The Committee's Problem: The "Committee" models became very cautious. They were so afraid of making a mistake that they often refused to answer at all (called "abstaining"). They would say, "I'm not 100% sure, so I'll stay silent." While they were perfect when they did speak, they stayed silent for nearly half the time.
The Brain's Success: The "Neural Network" (specifically a Multi-Layer Perceptron, or MLP) was much more confident. It kept its cool, made decisions almost all the time (98–99% of the time), and was still very accurate.

The "Smoothing" Trick

To help the AI make better decisions, the researchers added a "smoothing filter." Imagine you are trying to hear a conversation in a noisy room. If you listen to every single split-second sound, you might hear a cough and think it's a word. But if you listen to the flow of the sentence over a full cycle, you understand the meaning.
The researchers made the AI listen to a full "cycle" of electricity (about 1/60th of a second) before making a final call. This stopped the AI from panicking over tiny, harmless blips.

The Verdict

The paper concludes with a crucial lesson for the power industry: Don't trust the "Written Exam" scores alone.

A model that gets 99% on a static test might be useless in the real world if it's too scared to make a decision. The "Brain" model (MLP) proved to be the most reliable "crowd" for this job because it could adapt quickly and keep talking even when things got messy.

One Catch: Even the best model was a little too slow for the strictest safety rules. It took about 60 milliseconds to make a decision, but the grid's emergency brakes need to be pulled in under 50 milliseconds. So, while the AI is smart enough to know what is happening, it needs to get faster to be the one to stop the disaster.

In short: The researchers built a high-tech test lab, found that "cautious committees" often freeze up in real-time, while "adaptable brains" keep working, and proved that we need to test AI in live, fast-moving scenarios, not just on static exams, to ensure our power grid stays safe.

Technical Summary: High-Fidelity Classification of Cyber-Attacks and Faults in Power Systems

Problem Statement
The transition toward smart grids, characterized by high penetration of inverter-based resources (IBRs) and pervasive digital communication, has fundamentally altered power system dynamics and expanded the cyber-physical attack surface. Unlike synchronous machines, IBRs exhibit limited fault currents and non-sinusoidal waveforms, which degrade the efficacy of traditional protection relays. Furthermore, the integration of IEC 61850-based digital substations introduces vulnerabilities to cyber intrusions—such as False Data Injection (FDI), Denial-of-Service (DoS), and replay attacks—that can mimic or exacerbate physical faults. Accurate classification of these disturbances is critical to prevent unwarranted trips or equipment damage. However, existing machine learning (ML) solutions often rely on low-fidelity datasets that omit transient noise and class imbalances, and few studies evaluate performance under live, cycle-by-cycle streaming conditions. Consequently, there is a gap between high offline accuracy claims and reliable field readiness in real-time environments.

Methodology
This study employs a comprehensive, multi-phase framework to evaluate 12 distinct ML models for classifying both physical faults and cyber-attacks.

High-Fidelity Testbed: The research utilizes a realistic digital substation simulation built in WinIGS, featuring a 50 kVA PV system, a 30 kVA battery storage system, and various loads. The system operates at a sampling rate of 4.8 kHz (80 samples per cycle), capturing fast transient dynamics and harmonic content typical of IBRs and cyber-attacks.
Dataset Generation: Two datasets were generated in COMTRADE format: a 22-second training set containing 18 distinct classes (1 normal, 17 anomalies including various line faults, CT/PT ratio attacks, and GPS spoofing) and a separate 6-second streaming prediction set with 6 anomalies.
Model Selection: Twelve algorithms were evaluated, including ensemble methods (AdaBoost, Decision Tree, Extra Trees, Gradient Boosting, Random Forest), statistical models (k-NN, Logistic Regression, Naïve Bayes, SVM), and three Multi-Layer Perceptron (MLP) configurations.
Real-Time Inference Pipeline: To simulate operational constraints, the study implemented a streaming evaluation phase. This pipeline includes:
- Cycle-Length Smoothing: A centered moving average filter (one cycle, 80 samples) applied to model probability outputs to suppress high-frequency noise and jitter.
- Confidence Thresholding: A decision rule where predictions are only emitted if the maximum class probability exceeds a threshold ( $\tau = 0.6$ ); otherwise, the model "abstains" (outputs -1) to prevent low-confidence errors.
- Sub-Cycle Responsiveness: The architecture aims for decision-making within a fraction of a cycle, with a fixed look-ahead of approximately 8.33 ms.

Key Contributions

High-Fidelity Evaluation Framework: Introduction of a robust testing environment using COMTRADE data from a detailed digital substation simulation, enabling transparent assessment of ML performance under realistic transient conditions.
Comparative Benchmarking: A rigorous comparison of 12 ML and Deep Learning algorithms across both static (offline) and dynamic (streaming) scenarios, addressing the gap between theoretical accuracy and operational reliability.
Enhanced Decision Logic: Implementation of moving average filtering and confidence thresholding to stabilize real-time decisions, demonstrating their necessity in distinguishing genuine faults from benign transients (e.g., capacitor inrush).
Open-Source Reproducibility: Public release of the dataset and implementation code to bridge the gap between laboratory models and field-deployable protection systems.

Results
The study reveals a significant divergence between offline and streaming performance:

Offline Performance: Most models achieved near-perfect accuracies (up to 99.88%) during Phase 1 training and validation. Random Forest and MLPs were the top performers.
Streaming Performance: In Phase 2, a stark contrast emerged. While MLPs maintained high classification coverage (98–99%) with robust anomaly detection, ensemble methods (e.g., Random Forest, Extra Trees) exhibited severe coverage drops (10–49%). These ensembles preserved perfect precision on the few cases they classified but frequently abstained from making predictions due to low confidence.
Specific Challenges: The Double Line-to-Ground (DLG) AC-N fault proved the most difficult to classify, causing accuracy drops for many models.
Latency: The best-performing MLP model classified events in under 15 ms, but the average inference latency (including computation) was approximately 60 ms. This exceeds the 50 ms requirement for primary protection relays, indicating that while ML is feasible, immediate deployment for primary protection is currently constrained by computational overhead.

Significance and Claims
The paper asserts that offline accuracy alone is an unreliable indicator of field readiness for power system protection. The "Wisdom of the Crowd" (ensemble diversity) does not necessarily outperform the adaptive learning of neural networks in streaming environments; in fact, ensembles may be too conservative for real-time applications where coverage is critical.

The study emphasizes that realistic testing pipelines, incorporating cycle-based smoothing and confidence thresholds, are essential for ensuring dependable classification in IBR-rich networks. While ML offers granular, data-driven classification capabilities superior to coarse-level dynamic state estimation, the authors conclude that future field deployment requires a balance between physics-based rigor and data-driven flexibility, alongside strategies to reduce inference latency and handle topology changes. The work serves as a reproducible baseline, highlighting the need for streaming-aware evaluation metrics (coverage and abstention) rather than static accuracy scores.

The Experiment: A High-Stakes Simulation

The Big Surprise: The "Exam" vs. The "Live Show"

The "Smoothing" Trick

The Verdict

Technical Summary: High-Fidelity Classification of Cyber-Attacks and Faults in Power Systems

More like this