🤖 machine learning

Robust Unsupervised Network Intrusion Detection via Federated Learning with Selective Aggregation under Anomalous Sample Contamination

This paper proposes a robust unsupervised network intrusion detection framework for IoT environments that leverages federated learning's inherent underrepresentation of minority data and an Expectation-Maximization-based selective aggregation mechanism to effectively mitigate the impact of anomalous sample contamination during training.

Original authors: Shohei Kamiguchi, Takayuki Nishio

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Shohei Kamiguchi, Takayuki Nishio

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a massive, bustling city where billions of tiny devices—smart fridges, security cameras, and factory sensors—chat with each other every second. This is the Internet of Things (IoT). While this connectivity makes life easier, it also turns every device into a potential front door for digital burglars. To keep the city safe, security guards called Network Intrusion Detection Systems (NIDS) constantly scan the traffic for suspicious behavior. Traditionally, these guards learn what "normal" looks like by studying a library of labeled examples, like a police academy with a file on every known criminal. But in the real world, new types of malware appear faster than anyone can label them, and asking for a perfect, clean list of "good" traffic is like asking for a library with zero typos.

This is where "unsupervised learning" comes in. Instead of needing a teacher to point out the bad guys, these systems try to learn the shape of normal behavior on their own and flag anything that looks weird. However, there's a catch: what if the library itself is already contaminated? What if the "normal" traffic the system is studying secretly includes a few infected devices? If the system learns from this dirty data, it might start thinking the malware is actually normal, leaving the city wide open. The paper you're about to read tackles this exact problem: how to build a security guard that stays sharp even when the training data is secretly poisoned by a few bad actors, using a clever trick involving teamwork and a bit of mathematical detective work.

The Poisoned Well and the Teamwork Solution

In the world of cybersecurity, imagine you are trying to teach a robot to spot a fake coin. You give it a bucket of coins to study, hoping they are all real. But, unknown to you, a few bad coins have slipped in. If the robot studies too many of these fakes, it might start thinking, "Oh, this weird shape is actually normal!" and fail to catch the real fakes later. This is the "anomalous sample contamination" problem. In the real world, when we collect network traffic from thousands of IoT devices to train our security AI, some of those devices might already be infected. We can't easily tell which ones, so the "clean" training data we think we have is actually a mix of good and bad.

The authors of this paper, Shohei Kamiguchi and Takayuki Nishio, propose a solution called FLANDRE (Federated Learning-assisted Anomalous Node Detection & REmoval). Their big idea is to stop trying to mix all the data into one giant bucket and instead let the devices train their own little models locally, then share only their "lessons learned" (mathematical updates) with a central server. This approach is called Federated Learning.

Usually, people think Federated Learning is just about privacy—keeping data off a central server. But the authors found a secret superpower in it. They realized that in a team of 100 devices, if only a few (say, 10%) are infected, the "bad" data is a tiny minority. In standard Federated Learning, the central server averages everyone's updates. Because the infected devices are so few, their weird, "poisoned" lessons get drowned out by the hundreds of "good" devices. It's like trying to change the flavor of a giant pot of soup by adding a single drop of hot sauce; the soup stays mostly mild. The authors decided to lean into this "minority gets ignored" weakness and turn it into a strength.

The Detective Game: Finding the Bad Apples

However, just averaging the updates isn't enough if the bad devices are really loud or if the contamination is high. So, FLANDRE adds a second layer of defense: a Selective Aggregation mechanism. Think of the central server as a detective holding a map of where everyone stands.

Here's how the detective works:

The Warm-up: First, the server lets everyone train for a little while.
The Distance Check: The server looks at how much each device's new model has changed compared to the group's "average" model.
The Clustering: Using a mathematical tool called the Expectation-Maximization (EM) algorithm, the server groups the devices into two clusters based on how far their models are from the center.
- Cluster A: The "Good Guys." Their models are close to the average because they are all learning from normal traffic.
- Cluster B: The "Bad Guys." Their models have drifted far away because they are trying to learn from infected, weird data.
The Cut: The server simply ignores the "Bad Guys" (Cluster B) and only uses the updates from the "Good Guys" to update the main global model.

It's like a teacher noticing that a few students are trying to solve a math problem using a completely different (and wrong) method. Instead of letting those few students confuse the whole class, the teacher quietly asks them to sit out the group discussion, ensuring the final answer is based only on the correct logic.

What They Found in the Lab

The authors tested this idea on three famous cybersecurity datasets: ToN IoT, CSE-CIC-IDS2018, and NF-UQ-NIDS-v2. They simulated a scenario where 10% to 40% of the total training data was actually malicious, hidden inside the datasets of a few compromised devices.

The results were impressive. When they compared FLANDRE to other methods:

Standard Centralized Training: When they mixed all the data together (even the bad parts) and trained one big model, the performance dropped significantly. For example, on the IDS2018 dataset, the F1 score (a measure of accuracy) dropped to 0.590.
State-of-the-Art Competitors: They tested against a top-tier method called LOE-S, which tries to mathematically separate good and bad data. LOE-S scored 0.586 on the same dataset, barely better than the standard approach.
FLANDRE: Their new method scored 0.823 on IDS2018, 0.969 on ToN IoT, and 0.838 on NF-UQ-NIDS.

Crucially, FLANDRE's scores were almost identical to the "Ideal" scenario—a perfect world where the training data was 100% clean to begin with. On the ToN IoT dataset, FLANDRE was only 0.004 points away from the ideal score, whereas the other methods were far behind. This suggests that by filtering out the suspicious clients, FLANDRE can essentially "un-poison" the training process.

They also tested how the system handles different levels of infection. Even when the amount of bad data in the network increased, FLANDRE stayed stable. Interestingly, they found that if the infected devices had a very high amount of bad data (like 80% of their local files were malware), the system still worked well, but if the infection was very subtle, a specific variant of their method (called FLANDRE-R, which uses a different measurement) struggled a bit more. However, the main FLANDRE method remained robust across the board.

The Bottom Line

This paper doesn't claim to have solved every cybersecurity problem, but it offers a clever, practical way to handle a very real issue: training AI on data that might be secretly corrupted. By using Federated Learning not just for privacy, but to naturally suppress the influence of a few bad actors, and then adding a "detective" step to kick out the outliers, the authors created a system that learns what "normal" really looks like, even when the classroom is full of tricksters.

In their simulations, FLANDRE proved that you don't need to know exactly which devices are infected to protect the network. You just need to let the majority vote, and then have the courage to ignore the noisy minority. It's a reminder that sometimes, the best way to find the truth is to stop listening to the people who are trying to confuse you.

Technical Summary: Robust Unsupervised Network Intrusion Detection via Federated Learning with Selective Aggregation

Problem Statement
Network Intrusion Detection Systems (NIDS) are critical for securing Internet of Things (IoT) environments, yet they face a significant challenge in practical deployment: the contamination of training data. While unsupervised learning offers a scalable alternative to supervised methods by removing the dependency on labeled datasets, it traditionally assumes that training data consists entirely of benign traffic. In real-world IoT scenarios, traffic is often collected from operational gateways where infected devices may already be communicating. Consequently, training datasets frequently contain unlabeled anomalous samples. When unsupervised models are trained on such contaminated data, they risk internalizing malicious behavior as normal, leading to degraded detection performance. Existing contamination-robust anomaly detection (AD) methods often fail to exploit the specific structural characteristics of NIDS, where malicious activity is typically confined to a small subset of devices while the majority of traffic remains benign.

Methodology: FLANDRE
The authors propose FLANDRE (Federated Learning-assisted Anomalous Node Detection & REmoval), a novel unsupervised NIDS framework designed to operate robustly under data contamination. The method integrates Federated Learning (FL) with a selective aggregation mechanism.

System Model: The framework assumes a distributed IoT setting where multiple gateways (clients) hold local traffic datasets. A small fraction of these clients are "compromised" (containing both benign and anomalous flows), while the majority are "benign" (containing only benign flows). The goal is to train a global anomaly detector without centralizing raw data.
Leveraging FL Limitations as an Advantage: The authors exploit a known limitation of FL—its tendency to underrepresent minority patterns due to data imbalance. In this context, the minority pattern corresponds to the anomalous traffic from compromised clients. By distributing training, the influence of these minority anomalous samples is naturally suppressed compared to centralized aggregation.
Underlying AD Model: The framework utilizes Deep Support Vector Data Description (Deep SVDD), a deep learning-based one-class classification method. It learns a compact hypersphere in the feature space that encompasses normal data, with anomalies lying outside this boundary.
Selective Aggregation Mechanism: To further mitigate contamination, FLANDRE introduces a client-level filtering process during the Federated Averaging (FedAvg) rounds:
- Clustering: After local training, the server calculates the Euclidean distance between each client's local model update and the global model from the previous round.
- Gaussian Mixture Model (GMM): These distances are clustered into two groups using the Expectation-Maximization (EM) algorithm. One cluster represents clients with models close to the global consensus (likely benign), and the other represents divergent models (likely compromised).
- Exclusion: The cluster with the larger average distance (indicating higher divergence and likely contamination) is excluded from the aggregation step. The global model is updated only using the remaining "benign" cluster.
- Alternative Criterion: An optional variant, FLANDRE-R, uses the hypersphere radius ( $R$ ) of the Deep SVDD model instead of model parameter distances for client selection, though the primary method relies on parameter distance.

Key Contributions

Contamination-Robust Framework: The paper introduces a training framework that addresses the practical issue of unlabeled anomalous contamination in IoT NIDS. It uniquely repurposes the FL tendency to underrepresent minority data as a mechanism for anomaly suppression.
Selective Aggregation without Labels: The method employs a client-level filtering strategy based on model divergence (via EM clustering) that requires no labeled data, distinguishing it from prior sample-level filtering approaches.
Comprehensive Evaluation: The authors evaluate FLANDRE on three representative NIDS benchmarks: ToN IoT, CSE-CIC-IDS2018, and NF-UQ-NIDS-v2. They compare the method against:
- Ideal Deep SVDD (trained on clean data).
- Centralized Deep SVDD (trained on contaminated data).
- LOE-S (Latent Outlier Exposure), a state-of-the-art contamination-robust AD method.
- Standard Federated Deep SVDD (without selective aggregation).

Results
Experimental results demonstrate that FLANDRE consistently outperforms existing approaches in contaminated environments:

Performance: On the ToN IoT dataset with a 10% contamination ratio, FLANDRE achieved an F1 score of 0.969, which is only 0.004 below the ideal contamination-free reference. In contrast, the state-of-the-art LOE-S method scored 0.916, and centralized Deep SVDD scored 0.925.
Robustness: FLANDRE maintained high performance across varying contamination ratios (up to 40%) and varying proportions of anomalies within compromised clients. It significantly outperformed both centralized baselines and standard federated learning without filtering.
Stability: Learning curves showed that FLANDRE achieves stable performance without the sharp peaks and subsequent degradation observed in centralized baselines, making it more suitable for unsupervised settings where early stopping is difficult to tune.
Client Selection: The method demonstrated high accuracy in distinguishing compromised clients from benign ones, particularly on the ToN IoT dataset.

Significance and Claims
The paper claims that FLANDRE provides a practical solution for deploying unsupervised NIDS in real-world IoT environments where data contamination is inevitable. By transforming a known limitation of Federated Learning (minority underrepresentation) into a security feature, the method effectively isolates the impact of compromised devices. The authors assert that their approach offers superior robustness and adaptability compared to current state-of-the-art methods, specifically tailored to the structural reality that malicious activity in IoT is typically localized to a minority of devices.

The work builds upon a previous conference publication by introducing the selective aggregation strategy, expanding the evaluation scope, and providing detailed comparisons with recent SoTA methods. The authors note that while the method is robust, future work will focus on theoretical analysis of FL's mitigation capabilities and addressing non-IID (non-independent and identically distributed) normal traffic across clients.

The Poisoned Well and the Teamwork Solution

The Detective Game: Finding the Bad Apples

What They Found in the Lab

The Bottom Line

Technical Summary: Robust Unsupervised Network Intrusion Detection via Federated Learning with Selective Aggregation

More like this