Anomaly Detection in IEC-61850 GOOSE Networks: Evaluating Unsupervised and Temporal Learning for Real-Time Intrusion Detection

1. Problem Statement

The IEC-61850 GOOSE (Generic Object-Oriented Substation Event) protocol is the backbone of time-critical communication in digital substations, enabling sub-4 ms latency for protection and control signals by operating directly on the Ethernet data link layer. However, this architecture lacks native security mechanisms (encryption, authentication, integrity), making it vulnerable to:

• Replay Attacks: Retransmitting legitimate frames to trigger false state changes.
• Masquerade Attacks: Spoofing authorized device identities.
• Data Injection: Fabricating messages with manipulated payloads.

Key Challenges:

1. Strict Latency Constraints: Intrusion Detection Systems (IDS) must operate within the 4 ms window required for protection signals.
2. Data Scarcity: There is a severe lack of labeled attack data for training supervised models.
3. Class Imbalance: Normal traffic vastly outnumbers attack traffic (approx. 93% normal).
4. Generalizability: Models trained on one substation configuration often fail when deployed in different physical environments due to distribution shifts.

2. Methodology

The study evaluates five machine learning models on the ERENO IEC-61850 dataset, comparing supervised learning against unsupervised temporal modeling.

A. Models Evaluated

1. Supervised Baseline: Random Forest (RF). Trained on labeled data (Normal vs. Attack) with balanced class weights.
2. Unsupervised Static: Feedforward Autoencoder. Trained only on normal traffic to learn a compact representation; anomalies are detected via high reconstruction error.
3. Unsupervised Temporal: Three Recurrent Sequence Autoencoders trained on sliding windows of GOOSE messages:
   • RNN (Recurrent Neural Network)
   • LSTM (Long Short-Term Memory)
   • GRU (Gated Recurrent Unit)

B. Experimental Design

• Feature Engineering: Utilized protocol metadata (StNum, SqNum) and engineered delta features (time differences between messages) to capture temporal dynamics. High-cardinality categorical fields were dropped to improve generalizability.
• Threshold Selection: Anomaly thresholds were determined on a held-out validation set (10% of training data) using the Youden Index to prevent test-set leakage and metric inflation.
• Metrics: Evaluated on Accuracy, Precision, Recall, F1-Score, AUC-ROC, and critically, Inference Latency (time per prediction).
• Cross-Environment Validation: Models were tested without retraining on a second, independent dataset (de Oliveira et al.) to assess generalizability across different physical testbeds.

3. Key Contributions

• Latency-Aware Evaluation: Unlike many prior studies, this paper explicitly measures inference latency against the strict 4 ms GOOSE requirement, demonstrating that high accuracy is useless if it violates real-time constraints.
• Temporal vs. Static Analysis: It demonstrates that modeling the sequence of GOOSE messages (temporal dependencies) is superior to analyzing individual frames for detecting replay and masquerade attacks.
• Generalizability Insight: The study provides empirical evidence that unsupervised temporal models generalize better to new environments than supervised models trained on specific attack distributions.
• Rigorous Thresholding: By selecting thresholds on a validation set rather than the test set, the reported metrics reflect realistic deployment performance.

4. Key Results

A. Detection Performance & Latency (ERENO Dataset)

Model	F1-Score	Latency (ms)	Meets <4ms?
Random Forest	0.9516 (Highest)	21.837	❌ No
GRU	0.8737	1.118	✅ Yes
LSTM	0.8686	1.920	✅ Yes
RNN	0.8637	1.229	✅ Yes
Autoencoder	0.5826	0.039	✅ Yes

• Supervised vs. Unsupervised: The Random Forest achieved the highest F1-score (0.9516) but failed the latency constraint by a factor of 5x.
• Temporal Superiority: All recurrent models (GRU, LSTM, RNN) significantly outperformed the static Autoencoder (F1 0.58) in detecting attacks, proving that temporal sequence modeling captures attack signatures (like StNum/SqNum progression anomalies) that static models miss.
• Best Trade-off: The GRU offered the best balance, achieving an F1 of 0.8737 with a latency of 1.118 ms.

B. Cross-Environment Generalizability

When tested on the independent de Oliveira dataset without retraining:

• Random Forest: Suffered a catastrophic drop in performance (F1 dropped from 0.95 to 0.14). It overfit to the specific statistical patterns of the ERENO dataset.
• Recurrent Models: Retained significantly higher relative performance (F1 drops of ~0.60–0.68).
• Conclusion: Learning the temporal behavior of normal traffic generalizes better across different substations than fitting specific labeled attack distributions.

5. Significance and Conclusion

This paper establishes that unsupervised temporal modeling is the most practical approach for real-time GOOSE intrusion detection.

• Feasibility: Only unsupervised models met the critical 4 ms latency requirement, making them viable for inline deployment in digital substations.
• Robustness: The GRU and RNN architectures proved robust against distribution shifts, a critical requirement for deploying security systems across diverse, heterogeneous substation networks where labeled attack data is unavailable.
• Recommendation: The authors recommend the GRU-based unsupervised autoencoder as the optimal architecture for real-time GOOSE IDS, offering the best compromise between detection accuracy (F1 0.87), latency (1.1 ms), and cross-environment generalizability.

The study highlights a shift in industrial cybersecurity strategy: moving away from supervised classifiers that require extensive labeled attack data and fail in new environments, toward unsupervised temporal models that learn the "normal" behavior of the protocol and detect deviations in real-time.