🤖 machine learning

Byzantine-Robust Aggregation for Securing Decentralized Federated Learning

This paper proposes WFAgg, a novel Byzantine-robust aggregation algorithm that enhances the security and convergence of Decentralized Federated Learning by employing multiple filters to mitigate attacks in dynamic topologies, outperforming existing centralized robust schemes.

Original authors: Diego Cajaraville-Aboy, Ana Fernández-Vilas, Rebeca P. Díaz-Redondo, Manuel Fernández-Veiga

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Diego Cajaraville-Aboy, Ana Fernández-Vilas, Rebeca P. Díaz-Redondo, Manuel Fernández-Veiga

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Learning Without a Boss

Imagine a group of students trying to learn how to solve a complex puzzle.

Traditional Learning (Centralized): Usually, all students send their puzzle pieces to a single "Teacher" in the middle of the room. The Teacher looks at all the pieces, figures out the picture, and tells everyone what the final image looks like.
The Problem: If the Teacher is attacked, gets sick, or is tricked by a liar, the whole class fails. Also, sending all those pieces to one spot is slow and risky for privacy.
The New Idea (Decentralized Federated Learning): Instead of a Teacher, the students sit in a circle. They only talk to the people sitting next to them. They share their puzzle pieces, agree on a picture, and move on. There is no boss. This is faster and safer because if one student leaves, the others can keep going.

However, there is a catch: In a circle without a boss, it's very easy for a "troublemaker" (a Byzantine node) to join in. This troublemaker might hand out fake puzzle pieces or tell everyone the picture is upside down, ruining the learning process for everyone.

The Solution: The "WFAgg" Security Team

The authors of this paper created a new security system called WFAgg (Weighted Filter Aggregation) to protect these student circles from troublemakers.

Think of WFAgg as a triple-check security guard at the door of every student's desk. Before a student accepts a puzzle piece from a neighbor, they run it through three different tests (filters) to see if it's real or fake.

The Three Filters (The Security Checks)

The "Distance" Check (WFAgg-D):
- The Analogy: Imagine the group has a "standard" puzzle piece in the middle. The guard measures how far away the new piece is from that standard. If a piece is floating miles away from the group, it's probably a fake.
- What it does: It uses math to measure the physical distance between the new model and the average of the group. If it's too far, it gets rejected.
The "Direction" Check (WFAgg-C):
- The Analogy: Imagine everyone is pointing their fingers toward the correct answer. If a new person arrives pointing in the exact opposite direction, they are likely a troublemaker.
- What it does: It checks if the new model is pointing in the same "direction" as the group. Even if the numbers are close, if the idea is backwards, it gets flagged.
The "History" Check (WFAgg-T):
- The Analogy: The guard looks at the student's past behavior. "Hey, you usually send pieces that look like this. Suddenly, you're sending something totally different today? That's suspicious."
- What it does: It watches how a student's model changes over time. If a student suddenly makes a giant, weird jump in their data, the guard suspects an attack.

The Final Decision: The "Voting System"

Here is the clever part. The paper argues that one security check isn't enough. A smart troublemaker might be able to fool the "Distance" check but not the "Direction" check.

So, WFAgg uses a voting system:

If a puzzle piece passes all three checks, it gets a huge vote (high weight).
If it passes two, it gets a medium vote.
If it only passes one, it gets zero votes and is ignored.

This ensures that a troublemaker has to be perfect at fooling all three different types of guards to succeed, which is very hard to do.

What Did They Find?

The authors tested this system in a computer simulation using a famous image dataset (MNIST, which is like a digital sheet of handwritten numbers). They pitted their new system against old, famous security methods (like "Krum" or "Clustering") in two scenarios: with a central teacher and without one.

The Results:

Old Methods Struggled: The traditional security methods, which were designed for the "Teacher" scenario, often failed when there was no boss. They let fake pieces in or got confused by the noise.
WFAgg Won: The new triple-check system kept the learning accurate and fast, even when there were troublemakers trying to sabotage the process.
Consistency: Not only did the students learn the right answer, but they all agreed on the answer quickly. In the old methods, students often ended up with different, confused pictures.

The Bottom Line

The paper claims that by using a combination of three different security filters and a smart voting system, you can make Decentralized Learning (learning without a boss) much safer and more reliable. It proves that you don't need a central authority to stop hackers; you just need a smart, multi-layered way for neighbors to check each other's work.

Technical Summary: Byzantine-Robust Aggregation for Securing Decentralized Federated Learning

Problem Statement

Decentralized Federated Learning (DFL) extends the Federated Learning (FL) paradigm by eliminating the central server, thereby enhancing scalability, fault tolerance, and privacy by avoiding a single point of failure. However, this decentralization introduces significant security vulnerabilities, particularly regarding Byzantine failures, where malicious clients (Byzantine nodes) attempt to disrupt the learning process by sending arbitrary or poisoned model updates.

While numerous Byzantine-robust aggregation algorithms exist for centralized FL (CFL), the paper notes that most are designed for centralized scenarios and may not be equally efficient in decentralized environments. In DFL, the absence of a central coordinator complicates the implementation of uniform security algorithms, and the dynamic nature of decentralized topologies, combined with heterogeneous node capabilities, creates unique challenges. Existing literature lacks a dedicated Byzantine-robust aggregation algorithm specifically designed for DFL that accounts for both spatial and time-varying statistics.

Methodology

The authors propose WFAgg, a novel Byzantine-robust aggregation algorithm designed specifically for DFL environments. The core of the methodology involves a multi-stage filtering and weighted aggregation process executed locally by each node based on models received from its neighbors.

1. System Configuration:
The system is modeled as a cross-device scenario where $N$ clients collaborate over an undirected graph $G$ . Clients communicate via a Gossip-inspired protocol, forwarding trained models to all neighbors. The goal is to minimize the empirical loss function across the network without a central server.

2. The WFAgg Algorithm:
WFAgg employs a three-filter approach to identify and mitigate malicious models before aggregation. A model is only considered "benign" if it passes specific criteria across these filters. The filters operate sequentially:

WFAgg-D (Distance-based): Calculates the Euclidean distance between each received model and the median of the received models. It selects the $K-f-1$ models with the smallest distances, where $K$ is the number of received models and $f$ is the estimated number of malicious nodes.
WFAgg-C (Similarity-based): Calculates the cosine distance between received models and the median model. It also employs magnitude clipping based on the median magnitude. It selects the $K-f-1$ models with the smallest cosine distances, effectively filtering models that deviate significantly in direction.
WFAgg-T (Temporal-based): Analyzes the temporal evolution of model updates. It compares the current update of a neighbor against a time window of its previous updates (using Euclidean distance and cosine similarity). It utilizes an Exponentially Weighted Moving Average (EWMA) to establish thresholds for mean and standard deviation. Updates falling outside these thresholds are flagged as abrupt and potentially malicious.

3. Weighted Aggregation (WFAgg-E):
Once the filters identify subsets of benign models ( $T_1, T_2, T_3$ ), the algorithm assigns weights to each received model based on how many filters identified it as benign.

A model identified by all three filters receives the highest weight ( $\tau_1 + \tau_2 + \tau_3$ ).
A model identified by only one filter is assigned a zero weight (rejected).
A model identified by two filters receives a partial weight.
The final aggregation uses a coordinate-wise weighted average (similar to FedAvg but with differential weighting for the local model and neighbors), controlled by a smoothing factor $\alpha$ .

Key Contributions

The paper outlines four primary contributions:

Novel Algorithm Design: The development of WFAgg, a Byzantine-robust aggregation algorithm specifically for DFL that integrates spatial (distance/similarity) and temporal filtering techniques.
Comparative Evaluation: A systematic evaluation of state-of-the-art (SOTA) centralized Byzantine-robust algorithms (Mean, Median, Trimmed-Mean, Krum, Multi-Krum, Clustering) when applied to decentralized settings, highlighting their limitations.
Empirical Validation: Extensive experiments on an IID image classification task (MNIST) under various Byzantine attack scenarios (Noise, Sign-Flipping, Label-Flipping, ALIE, IPM) demonstrating that WFAgg outperforms centralized schemes in decentralized environments.
Simulation Framework: The implementation of a Python-based DFL simulator to conduct these experiments, supporting both centralized and decentralized topologies.

Results

The experiments were conducted on a network of 20 nodes (10% malicious) using an 8-regular graph topology. The results indicate:

Robustness Against Attacks: WFAgg successfully mitigated all tested attacks (including Noise, Sign-Flipping, Label-Flipping, ALIE, and Inner Product Manipulation variants) in both centralized and decentralized scenarios.
Comparison with SOTA:
- Standard centralized algorithms like Mean and Trimmed-Mean failed to converge or suffered significant accuracy drops under Noise and IPM-100 attacks in decentralized settings.
- Distance-based algorithms (Krum, Multi-Krum) struggled with IPM-0.5 attacks in decentralized scenarios due to the inability to distinguish subtle directional changes.
- Clustering-based methods failed under Noise attacks in decentralized settings.
- WFAgg maintained high accuracy (e.g., ~95-98% in various scenarios) and achieved faster convergence compared to the SOTA algorithms tested in the decentralized setting.
Model Consistency: Using the R-squared ( $R^2$ ) metric to measure the similarity of local models among benign nodes, WFAgg demonstrated superior convergence consistency compared to filtering-based SOTA algorithms like Krum and Clustering, which showed lower $R^2$ values due to nodes selecting different subsets of models.
Individual Node Performance: Analysis of individual node accuracy evolution showed that benign nodes converged to high accuracy within a few rounds, even when surrounded by malicious neighbors, whereas malicious nodes were effectively isolated by the weighting mechanism.

Significance and Claims

The paper claims that WFAgg represents a novel approach in the literature, as it is the first to explicitly address Byzantine robustness in decentralized scenarios by considering both spatial and time-varying statistics. The authors assert that their integration of multiple lightweight filters allows the system to adapt to the dynamic and adversarial nature of decentralized IoT environments.

The study concludes that while existing centralized robust aggregation schemes offer some protection, they are insufficient for the specific challenges of DFL. The proposed WFAgg algorithm provides a more efficient and robust alternative, ensuring model accuracy and convergence even in the presence of diverse Byzantine attacks. The authors suggest that future work should focus on optimizing communication networks (e.g., using Software-Defined Networks) to further reduce overhead and latency in these decentralized learning processes.