🤖 AI

From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning

This survey provides the first data-centric review of Federated Learning by systematically analyzing how non-IID data traits, experimental splitting artifacts, and data-related vulnerabilities impact model convergence and stability, offering actionable guidance for designing robust FL systems.

Original authors: Huong Nguyen, Mickaël Bettinelli, Amirhossein Ghaffari, Alexandre Benoit, Hong-Tri Nguyen, Susanna Pirttikangas, Lauri Lovén

Published 2026-06-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Huong Nguyen, Mickaël Bettinelli, Amirhossein Ghaffari, Alexandre Benoit, Hong-Tri Nguyen, Susanna Pirttikangas, Lauri Lovén

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a class of students how to recognize different animals. In a traditional classroom (Centralized Learning), you gather all the students' homework, photos, and notes into one big pile on your desk. You study everything at once, find the patterns, and give everyone the same final answer.

Federated Learning (FL) is different. Imagine the students are scattered across the globe, and they are too shy or legally restricted to share their personal notebooks with you. Instead, you send them a "starter guide" (the model). They study their own local notebooks, figure out what they think the answer is, and send you back only their notes on what they learned—not the actual photos or homework. You combine these notes to update the guide, send it back out, and repeat.

This paper is a deep dive into why this process sometimes works like a charm and other times grinds to a halt, specifically looking at the data itself. The authors argue that most people focus on the algorithm (the math), but the real troublemaker is the data (the homework).

Here is the breakdown of their findings using simple analogies:

1. The "Homework" Quality: What Makes Learning Fast or Slow?

The paper asks: Does the type of homework matter? The answer is a resounding yes. They broke down data traits into three levels of importance:

The "Big Three" (Strong Impact):
- Provenance (Where the data comes from): Imagine a classroom where some students are reliable and show up every day (Cross-Silo, like hospitals), while others are part-time students who only show up occasionally and have messy handwriting (Cross-Device, like phones). The paper finds that learning is much faster and steadier when everyone is reliable. If students are unreliable, the teacher (the server) gets confused updates, and the class never agrees on the answer.
- Intra-Class Variability (How messy the examples are): If you are teaching "Cat," and every student only has pictures of fluffy orange cats, it's easy. But if one student has a black cat, another has a hairless cat, and another has a cat in a box, the "Cat" concept becomes messy. The more variety within a single category, the harder it is for the group to agree on a definition.
- Inter-Class Separation (How different the categories are): If you are teaching "Cat" vs. "Dog," and the pictures look very different, learning is fast. If you are teaching "Siamese Cat" vs. "Persian Cat" (which look very similar), the students get confused, and the teacher struggles to separate the two groups.
The "Middle Tier" (Medium Impact):
- Label Skew: Imagine one student only has homework about "Cats," another only has "Dogs," and a third has a mix. If the teacher tries to average their opinions, the "Cat" student might drag the whole class toward thinking everything is a cat. The more unbalanced the data is across students, the slower the learning.
- Number of Classes: Teaching 10 animals is easy. Teaching 100 animals (where many look alike) is much harder and takes longer.
The "Weak Link" (Low Impact):
- Number of Samples: Surprisingly, just having more homework doesn't always help if the homework is still messy or unbalanced. It's better to have a few students with clear, balanced examples than 1,000 students with confused, messy examples.

2. The "Simulation" Trap: Why Practice Doesn't Always Match Reality

Researchers often test these systems in a lab by artificially shuffling data to make it look "messy" (non-IID). They use a mathematical tool called a Dirichlet distribution (think of it as a random number generator that decides how much "Cat" homework goes to Student A vs. Student B).

The paper warns: This is a fake mess.

Real Life: In the real world, the messiness is "coupled." A student from a rural area might have fewer photos, different types of animals, and missing categories entirely because of where they live.
The Lab: The artificial shuffling usually just changes the count of photos but keeps the types the same.
The Result: The paper argues that if you only test with the artificial "fake mess," you might think your system is great, but when you deploy it in the real world, it will crash because it wasn't tested on the real kind of messiness (like missing data types or huge differences in student participation).

3. The "Cheaters" and the "Bodyguards" (Security)

Since the students are sending notes back and forth, bad actors (hackers) might try to sneak in.

Poisoning (The Saboteur): A bad student submits notes that say, "A picture of a toaster is actually a Cat." If the teacher trusts this, the whole class learns the wrong thing.
Evasion (The Magician): A bad student tries to trick the teacher during the final exam by adding a tiny, invisible speck of dust to a picture of a Dog so the teacher thinks it's a Cat.
Inference (The Spy): A spy tries to guess, "Did Student A's notebook contain a picture of my pet?" just by looking at the teacher's final notes.

The Trade-off:
The paper looked at "Bodyguards" (defense methods) to stop these cheaters. They found a surprising sweet spot:

Good News: Most bodyguards can stop the saboteurs and spies without slowing down the class or making the teacher less accurate on normal days.
The Cost: If you want super strong security, you might lose a tiny bit of accuracy (about 1–3%), but it's usually worth it.
The Catch: Some defenses work great in a clean classroom (IID) but struggle when the students are already messy and unbalanced (Non-IID). You have to pick the right bodyguard for the specific type of chaos you have.

Summary of the Paper's "Takeaway"

The authors want to tell practitioners: Stop just tweaking the math. If your Federated Learning system is slow or inaccurate, look at your data.

Check your sources: Are your data sources reliable? Are the categories distinct?
Test realistically: Don't just use random shuffling in your experiments. Try to mimic the real-world messiness (like missing data types or uneven participation).
Balance security: You can usually protect your system without sacrificing performance, but you need to choose the right defense for your specific data chaos.

By understanding these data traits, you can design a system that learns faster, stays stable, and doesn't get tricked by bad actors.

Technical Summary: From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning

1. Problem Statement

Federated Learning (FL) has emerged as a critical paradigm for training shared models across distributed clients without centralizing data, addressing privacy concerns and data scarcity. However, existing surveys predominantly adopt a method- or system-centric lens, focusing on general foundations, security practices, or broad applications. These reviews often fail to delve deeply into the specific aspects of data as a primary driver of system behavior. Consequently, there is a lack of a "data-lens synthesis" that explicitly links concrete data properties (traits), partitioning protocols, and defense mechanisms to convergence speed, stability, and final accuracy. Practitioners often struggle to predict how specific data characteristics (e.g., label skew, intra-class variability) or splitting strategies will impact the training dynamics, leading to unpredictable convergence and suboptimal system design.

2. Methodology

The authors employed a structured, data-centric survey methodology to address this gap:

Search Strategy: Primary searches were conducted on Google Scholar using keyword combinations related to specific topics (e.g., "federated learning" + "attack," "heterogeneity") coupled with "federated learning."
Selection Criteria: Priority was given to highly cited and recent works (2020–2026). Secondary criteria included the reputation of the publication venue (Q1/Q2 quartiles).
High-Quality Evidence: For the specific analysis of how data splitting affects convergence, the evidence table was restricted to publications from A to A*-ranked venues (CORE rank) such as AAAI, ICLR, CVPR, NeurIPS, etc., to ensure a scoped comparative synthesis of high-quality works.
Scope: The survey strictly focuses on data as a determining factor and a reason for challenges. It excludes environment and system-level constraints (e.g., network latency, hardware heterogeneity) unless they are direct consequences of data handling choices.
Analysis Framework: The study organizes findings around four Research Questions (RQs):
1. What data source features affect convergence?
2. What factors prevent simulation from reflecting real-world data?
3. How are datasets typically split in prior experiments?
4. What are the convergence-privacy trade-offs of defense methods?

3. Key Contributions and Findings

RQ1: Data Traits and Convergence

The paper analyzes intrinsic data traits and ranks their influence on convergence as Strong, Medium, or Light:

Strong Impact:
- Provenance: Cross-device settings (many, intermittent clients) generally lead to slower, less stable convergence compared to cross-silo settings (few, reliable clients) due to higher statistical heterogeneity and dropout.
- Intra-Class Variability (ICV): Higher variability within a class (e.g., cluttered backgrounds in SVHN, diverse handwriting in FEMNIST) inflates gradient variance and slows convergence.
- Inter-Class Separation (ICS): Larger margins between classes (e.g., MNIST vs. Fashion-MNIST) accelerate convergence and improve final accuracy.
Medium Impact:
- Label Skew: Native label imbalance (common in cross-device data) misaligns client gradients, slowing convergence and reducing accuracy at finite round budgets.
- Number of Classes: More classes reduce inter-class separation and increase confusion, slowing convergence (e.g., CIFAR-10 vs. CIFAR-100).
- Feature Dimension: Higher dimensionality increases sample complexity and gradient variance, though its effect is secondary to geometry.
- Graph Homophily: Higher homophily aligns message passing with label smoothness, speeding up convergence in graph FL.
Light Impact:
- Number of Samples: Increasing sample count only helps if it increases label coverage or reduces variance; otherwise, it primarily increases per-round compute without accelerating convergence.

The authors provide a "trait-behavior map" (Table 3) and a pairwise relation analysis (Figure 1), showing that geometry (ICV/ICS) and provenance are the dominant drivers, while scale (sample count) is the least influential factor.

RQ2 & RQ3: Data Splitting and Simulation

The paper highlights a significant gap between synthetic simulation and real-world deployment:

Simulation Limitations: Standard benchmarks often use simple, equal splits or synthetic Dirichlet distributions that fail to capture coupled heterogeneities like long-tailed client sizes, missing classes, domain shifts, and dynamic participation.
Splitting Strategies:
- Dirichlet-based: The most common synthetic method, controlled by parameter $\alpha$ . Small $\alpha$ creates extreme skew; large $\alpha$ approaches IID.
- Natural Splitting: Partitions data by real-world provenance (e.g., user ID, hospital site). This preserves realistic heterogeneity (e.g., FEMNIST, LEAF datasets) but is harder to control and compare.
- Cluster-based & Temporal: Grouping by geography/similarity or time intervals to mimic specific application scenarios.
Recommendation: The authors advocate for prioritizing natural splits over synthetic Dirichlet splits for external validity, while using synthetic baselines for controlled ablation. They emphasize reporting quantitative heterogeneity diagnostics (e.g., client-size distribution, class coverage) to ensure comparability.

RQ4: Vulnerabilities and Defense Trade-offs

The paper systematically examines data-related vulnerabilities (Poisoning, Evasion, Membership Inference, Unintended Leakage) and their defenses using a Pareto frontier lens (Clean Utility vs. Under-Attack Utility):

Poisoning (Label Flipping/Backdoor): Defenses like FLDetector and DOS can reduce attack success to near-zero while maintaining or slightly improving clean accuracy. SparseFed offers large attack reduction with a small clean accuracy drop (~1–3 pp).
Evasion: Methods like USD-FL (using reserve-set scoring) and FedDef (pseudo-gradients) achieve significant robustness gains with minimal clean accuracy loss.
Membership Inference: Defenses such as MIST, HAMP, and Purifier drive attack metrics toward chance levels with negligible or even positive impacts on clean utility.
Unintended Exposure (Replay Buffers): Techniques like KDRSFL and FedPMR mitigate leakage and catastrophic forgetting with near-zero accuracy costs.
Trade-off Insight: Most well-designed defenses improve robustness with negligible impact on convergence. Stronger protection typically incurs only a small cost (1–3% accuracy drop), whereas Differential Privacy often involves larger trade-offs.

4. Significance and Claims

The paper claims to be the first survey to provide a complete, data-centric understanding of the challenges governing FL convergence. Its significance lies in:

Actionable Guidance: It distills clear takeaways for practitioners, enabling them to design systems with predictable convergence and stability by understanding how specific data traits (e.g., ICV, ICS, provenance) dictate optimization behavior.
Bridging the Gap: It links concrete data decisions (splitting protocols, dataset curation) directly to optimization outcomes, moving beyond abstract theoretical bounds to empirical realities observed across images, text, and graphs.
Explicit Trade-offs: By mapping defenses onto a Pareto frontier, it clarifies when robustness gains come at material costs versus when they preserve near-clean accuracy.
Future Directions: The authors identify open issues, particularly regarding model initialization (simple vs. pre-trained) and its interaction with non-IID data, noting that no consensus exists on whether specific initializations guarantee convergence.

The work serves as a foundational guide for researchers and practitioners to navigate the complex interplay between data heterogeneity, partitioning strategies, and security, ultimately paving the way for more robust and comprehensive federated learning systems.