🤖 machine learning

Rethinking the Transferable Adversarial Attacks and Robust Defense in Federated Learning

This paper analyzes the transferability of adversarial examples across client models in federated learning to reveal system vulnerabilities and proposes a robust defense mechanism based on adversarial training that outperforms existing state-of-the-art methods.

Original authors: Zuobin Xiong, Deval Mukherjee, Homook Cho, Wei Li

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zuobin Xiong, Deval Mukherjee, Homook Cho, Wei Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern digital world, our smartphones and devices are constantly learning from us, predicting the next word we type or recognizing our voice, all without ever sending our private photos or messages to a central server. This is made possible by a method called federated learning, where many devices work together to train a single smart model while keeping their own data locked away on their local hardware. It is a powerful way to build intelligent systems that respect privacy, but it relies on a delicate trust: that every device participating in the training is honest and following the rules. However, just as a group project can be sabotaged by a single dishonest member, this distributed system is vulnerable to attackers who might try to trick the model into making mistakes. While researchers have long worried about attackers who corrupt the training data itself, a more subtle threat has emerged: the ability to craft a single, carefully altered input that can fool many different versions of the model, even if those models were trained on different sets of data.

A team of researchers set out to understand this specific danger within the complex environment of federated learning. They focused on a scenario where an attacker, acting as one of the many participating devices, creates a deceptive input designed to cause a specific error. The core of their investigation was to determine if an attack crafted on one device could successfully travel across the network to fool other devices that had never seen the attack before. This phenomenon, known as transferability, had been observed in experiments, but the researchers wanted to understand the mathematical reason behind it. They discovered that the success of such an attack is not random; it is directly tied to how similar the data distributions are between the attacker and the victim. If the attacker's local data looks statistically similar to the victim's data, the deceptive input is far more likely to cross the gap and cause a misclassification. Conversely, if the data landscapes are very different, the attack loses its power. This finding challenged the assumption that these attacks would work equally well against any target, revealing instead that the specific makeup of the data on each device acts as a natural shield or a vulnerability.

To prove this connection, the researchers analyzed the relationship between the model parameters and the data distributions of the attackers and victims. They found that the difference in how data is distributed across the network creates a measurable distance between the models. When this distance is small, meaning the data is similar, the attack transfers easily. When the distance is large, the attack fails. This insight allowed them to move beyond simple observation to a theoretical understanding of why some devices are more susceptible than others. They demonstrated that in a realistic setting where devices have different types of data, the effectiveness of an attack is bounded by these differences. This means that in a diverse network, an attacker cannot simply assume their crafted trick will work on everyone; the specific nature of the victim's data determines the outcome.

Having identified the root cause of the vulnerability, the team designed a new defense mechanism to protect the system. Instead of trying to detect the attacker or filter out bad updates, they focused on making the models themselves more robust. They introduced a method that acts like a filter at the very beginning of the model's processing, stripping away the specific features that are most easily manipulated by these deceptive inputs while keeping the core, useful information intact. This process is combined with a training technique where the models are exposed to these tricky inputs during their learning phase, teaching them to recognize and ignore the deception. By doing this, the models learn to rely on more stable patterns rather than the fragile details that attackers exploit. The result is a system that can withstand these transferable attacks without sacrificing its ability to perform well on normal, everyday tasks.

The researchers tested their ideas using real-world image datasets, simulating a network of devices with varying levels of data diversity. In scenarios where the data was uniform across all devices, standard models collapsed completely when faced with these attacks, often failing to recognize images correctly almost every time. The new defense method, however, maintained high accuracy, successfully resisting the attacks and keeping the system functional. Even in more complex, realistic scenarios where the data was highly uneven and skewed, the defense held strong, outperforming existing methods that were previously considered the best available. The experiments confirmed that by understanding the link between data distribution and attack transferability, it is possible to build federated learning systems that are not only private but also resilient against sophisticated manipulation. This work suggests that the path to secure artificial intelligence lies not just in blocking attackers, but in understanding the fundamental structure of the data that powers these systems.

Technical Summary: Rethinking Transferable Adversarial Attacks and Robust Defense in Federated Learning

1. Problem Statement

Federated Learning (FL) enables collaborative model training across distributed clients without sharing raw data, yet it remains vulnerable to adversarial attacks. While existing literature extensively covers poisoning and Byzantine attacks, these often rely on unrealistic assumptions (e.g., controlling 10–25% of clients or assuming i.i.d. data) that fail in large-scale, real-world FL scenarios. Consequently, evasion attacks using transferable adversarial examples present a more practical and operable threat.

The core problem addressed is the transferability of adversarial examples in FL systems, specifically within the FedProx framework (chosen for its ability to handle system and statistical heterogeneity). The authors investigate how an attacker, possessing only their local data and model, can craft adversarial examples that successfully misclassify inputs on victim clients with different local data distributions and model parameters. Furthermore, the paper addresses the lack of efficient, robust defense mechanisms against such transferable attacks in federated settings, noting that existing defenses like Federated Adversarial Training (FAT) are computationally expensive and limited to known attacks.

2. Methodology

A. Theoretical Analysis of Transferability

The authors provide a theoretical framework to quantify the transferability of adversarial examples between an attacker ( $A$ ) and a victim ( $V$ ).

Metric: Transferability is defined by the similarity in loss function values: $\|l(f(w_A; x'), y) - l(f(w_V; x'))\|$ . A smaller distance implies higher transferability.
Theorem 1: The paper derives an upper bound for this loss difference, demonstrating that transferability is positively correlated with the similarity in local data distributions between the attacker and the victim.
- The bound depends on the Lipschitz constant of the loss function, the learning rate, and the gradient magnitude.
- Crucially, the term $\| \sum p_V(y_i=c) - p_A(y_i=c) \|$ indicates that as the data distribution of the victim diverges from the attacker (i.e., the distance increases), the attack's transferability decreases. Conversely, similar distributions facilitate successful transfer.

B. Proposed Defense Mechanism

To mitigate these attacks, the authors propose a robust defense framework combining Singular Value Decomposition (SVD) and Adversarial Training:

SVD-driven Feature Evolution Module (SDFEM):
- Deployed as a plug-in at the input layer of local client models.
- It decomposes input feature maps ( $x = U\Sigma V^T$ ) and applies a learnable evolution mask ( $M$ ) to re-weight singular values ( $\Sigma \odot M$ ).
- Goal: Suppress feature channels highly sensitive to adversarial perturbations while retaining robust semantic features. This effectively lowers the maximum gradient term ( $G_{max}$ ) identified in the theoretical analysis, reducing the model's vulnerability to transfer-based attacks.
Federated Adversarial Training:
- The local objective function is modified to jointly optimize model weights ( $w_k$ ) and the feature mask ( $M_k$ ).
- Clients generate adversarial examples ( $D_{adv}$ ) locally using a surrogate model (e.g., PGD) and train on both clean data and adversarial examples.
- The global aggregation step propagates these robust features across the network, enhancing the global model's resilience against unseen data.

3. Key Contributions

Theoretical Insight: This is the first work, to the authors' knowledge, to theoretically analyze the transferability of adversarial examples in FL. It establishes a mathematical link between attack success and the data distribution shift between clients.
Robust Defense Framework: The design of a defense mechanism (SDFEM + Adversarial Training) that is efficient and effective against transferable attacks without requiring the server to possess a clean dataset or assume i.i.d. data.
Empirical Validation: Extensive experiments on real-world datasets (CIFAR-10, SVHN) under various settings (IID and Non-IID) validating the theoretical correlation and demonstrating superior defense performance compared to state-of-the-art baselines.

4. Experimental Results

The authors evaluated their approach against four gradient-based attacks (VMIFGSM, BIM, VNIFGSM, DIFGSM) on VGG11 and WideResNet-28-10 architectures.

Verification of Theory (RQ1): Experiments confirmed a negative correlation between model parameter distance and Attack Success Rate (ASR), meaning ASR drops as the distance between attacker and victim models increases. Furthermore, experiments showed that the attack transferability gap (the difference in loss values) increases as the distribution distance gets large. This positive correlation confirms that as the data distribution distance between attacker and victim grows, the attack becomes less transferable, aligning with Theorem 1.
Defense Performance (RQ2):
- IID Settings: Standard FedProx models collapsed under attack (ASR > 99%, accuracy < 1%). The proposed method significantly outperformed the baseline Federated Adversarial Training (FAT). For instance, on CIFAR-10 with WRN28, the proposed method improved robust accuracy by +3.36 to +4.28 percentage points over FAT across different attacks while maintaining comparable clean accuracy.
- Non-IID Settings: In heterogeneous environments, the proposed method continued to outperform FAT. On CIFAR-10 (Non-IID) with WRN28, it improved robustness by approximately +9 percentage points over FAT across all attack types.
- Comparison to Centralized: The proposed federated defense approached the robustness of a centralized upper bound in several scenarios, suggesting that federated constraints do not necessarily preclude high robustness.

5. Significance and Claims

The paper claims that transferability is a critical security dimension in Federated Learning that is fundamentally tied to data distribution shifts. By revealing this relationship, the authors argue that defenses must account for the heterogeneity of client data.

The significance of the work lies in:

Moving beyond unrealistic assumptions (like high malicious client ratios) to address practical evasion threats.
Providing a theoretical basis for why attacks transfer in FL, moving beyond purely experimental observations.
Demonstrating that a combination of feature-level robustness (via SVD) and federated adversarial training can effectively secure FL systems against transferable adversarial examples, offering a viable path for robust deployment in real-world, heterogeneous environments.

The authors conclude that their findings suggest robustness-aware defenses can substantially strengthen FL in practical environments, outperforming existing state-of-the-art methods in both IID and Non-IID settings.