🤖 machine learning

On the Push-Based Asynchronous Federated Learning: A Bias-Correction Aggregation Approach

The paper proposes PushCen-ADFL, a communication-efficient asynchronous federated learning framework that utilizes centroid-based message exchange, push-sum mixing for bias correction, and centroid regularization to mitigate model drift and significantly improve accuracy while reducing communication costs in heterogeneous, decentralized systems.

Original authors: Jiahui Bai, Hai Dong, A. K. Qin

Published 2026-05-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiahui Bai, Hai Dong, A. K. Qin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of friends trying to solve a massive jigsaw puzzle together, but they are all in different rooms, can't talk to each other directly, and have different pieces of the picture. This is the real-world scenario for Federated Learning: many devices (clients) trying to learn a shared skill (like recognizing images) without ever sharing their private data.

Usually, these friends wait for everyone to finish a step before moving on. But in the real world, some friends are fast, some are slow, some have bad internet, and some join the game late. This is called Asynchronous Decentralized Federated Learning (ADFL). While flexible, this messy setup has three big problems:

Too much talking: Sending full puzzle pictures back and forth clogs the phone lines (high communication cost).
Wrong averages: If the fast friends talk more often, their opinion dominates the group, skewing the final picture (aggregation bias).
Drifting apart: Because everyone has different puzzle pieces (non-IID data) and works at different speeds, they start building different pictures that don't fit together (model drift).

The paper introduces a new method called PushCen-ADFL to fix these issues. Here is how it works, using simple analogies:

1. The "Centroid" Shortcut (Solving the Traffic Jam)

Instead of sending the entire, heavy puzzle picture every time a friend updates their work, they send a summary.

The Analogy: Imagine instead of mailing a 1,000-page book, you send a list of 32 "key themes" (centroids) and a map saying which page belongs to which theme.
The Result: This shrinks the message size by over 80%. It's like sending a text message summary instead of a video file. The paper calls this "Weight Clustering Pruning."

2. The "Fairness Token" System (Solving the Bias)

In a chaotic group chat, the loudest person (the fast computer) often drowns out the quiet ones.

The Analogy: Imagine every friend starts with one "voting token." When they send a message, they split their token in half and give a piece to the person they talk to. If you talk to 5 people, you keep a piece and give a piece to each.
The Result: This "Push-Sum" method ensures that even if some friends talk more often or have faster connections, the final group decision is still a fair average of everyone's input. It corrects the bias caused by uneven communication.

3. The "Shared Anchor" (Stopping the Drift)

When friends work on their own pieces for too long without checking in, they might start drawing things that don't match the rest of the group.

The Analogy: The paper gives everyone a "magnetic anchor" based on the group's current best summary. Even while they are working on their own unique pieces, they are gently pulled back toward this shared magnetic center.
The Result: This "Centroid Regularization" keeps everyone's work aligned with the group's direction, preventing them from drifting too far off course, even if their local data is very different from others.

4. The "Smart Buffer" (Handling Late Arrivals)

In an asynchronous system, messages can arrive in bursts or be very old (stale).

The Analogy: Imagine a friend's mailbox. If they get three messages from the same person, the system throws away the old ones and only keeps the newest one. It also has a limit on how many messages it holds so it doesn't get overwhelmed.
The Result: This prevents old, outdated information from messing up the current calculation.

What Did They Find?

The authors tested this system on standard image datasets (like CIFAR-10 and Tiny-ImageNet) where the data was split unevenly among the "friends."

Accuracy: Their method was up to 6% more accurate than other efficient methods, especially when the data was very messy (non-IID).
Speed/Efficiency: They reduced the amount of data sent per message by more than 80% compared to sending full models.
Late Joiners: The system handled "delayed clients" (friends who joined the game late) very well, helping them catch up quickly without ruining the group's progress.

In short, PushCen-ADFL is a way for a chaotic, disconnected group of computers to learn together efficiently. It uses smart summaries to save bandwidth, a token system to ensure fairness, and a magnetic anchor to keep everyone on the same page, resulting in a better final model with less data traffic.

Technical Summary: Push-Based Asynchronous Federated Learning: A Bias-Correction Aggregation Approach

1. Problem Formulation

The paper addresses the challenges inherent in Asynchronous Decentralized Federated Learning (ADFL), a paradigm designed to eliminate central coordination and global synchronization. While ADFL offers advantages in scalability, fault tolerance, and adaptability to heterogeneous devices, the authors identify three coupled difficulties that degrade performance in real-world deployments:

Communication Overhead: Frequent peer-to-peer exchanges of full model parameters are prohibitive in bandwidth-constrained or large-scale systems.
Aggregation Bias: Asynchronous updates on directed, potentially unbalanced topologies lead to biased aggregation. Naive neighbor averaging fails to guarantee unbiased consensus when clients push models at different rates or when communication links are asymmetric.
Model Drift: Statistical heterogeneity (non-IID data) combined with asynchronous execution causes local models to drift significantly. This drift is exacerbated by delayed updates and the distortion introduced by model compression.

The core research question is how to jointly design compression, de-biased aggregation, and local-update stabilization to achieve accurate and stable ADFL under asymmetric communication and non-IID data distributions.

2. Methodology: PushCen-ADFL

The authors propose PushCen-ADFL (Push-Sum Centroid Asynchronous Decentralized Federated Learning), a framework that unifies communication, aggregation, and optimization within a shared centroid representation space. The method operates without a central server or global synchronization, relying on event-driven, neighbor-to-neighbor communication.

Key Components:

Centroid-Based Model Compression (WCP):
Instead of transmitting dense parameters, clients encode weights into a centroid form consisting of a centroid table ( $V$ ) and an assignment map ( $A$ ). This reduces the communication payload from $O(d)$ to $O(K + d \log K)$ , where $K$ is the number of clusters (typically $K \ll d$ ). One centroid is fixed at zero to naturally encode pruned weights.
Average-Preserving Push-Sum Aggregation:
To correct aggregation bias on directed and unbalanced graphs, the framework employs a push-sum mechanism. Each client maintains a scalar "mass" ( $s_i$ ) that tracks mixing weights.
- Mass Splitting: When a client sends updates, it splits its mass among its out-neighbors and itself.
- Bias Correction: Aggregation normalizes updates by the total accumulated mass ( $S$ ) rather than raw message frequency. This ensures that the influence of a sender is determined by conserved mass, preventing fast clients from dominating the network dynamics.
- Buffered Updates: A sender-deduplicated bounded buffer is used to cache incoming messages. It retains only the most recent message from each sender, preventing bursty arrivals or stale messages from distorting the aggregation weights.
Centroid-Aligned Proximal Regularization:
To mitigate model drift caused by non-IID data and asynchrony, the local optimization step includes a lightweight proximal regularizer.
- Clients construct a centroid anchor ( $\tilde{w}_i$ ) by decoding their current centroid dictionary estimate ( $G_i$ ).
- The local objective becomes $f_i(w_i) + \lambda \|w_i - \tilde{w}_i\|^2_2$ .
- This term contracts local trajectories toward a shared, compressed reference structure, stabilizing updates without decoupling them from the centroid-based compression used for communication.

Convergence Analysis

The paper provides a theoretical convergence analysis under an event-driven asynchronous model on directed graphs. Under standard assumptions (smoothness, bounded variance, bounded staleness, and bounded compression error), the authors prove that PushCen-ADFL converges to a stationary neighborhood. The bound explicitly accounts for the effects of heterogeneity, staleness, directed mixing, and compression distortion.

3. Key Contributions

The authors summarize their contributions as follows:

Framework Design: Proposal of PushCen-ADFL, which unifies centroid-based communication, push-sum de-biasing, and buffered neighbor aggregation into a single decentralized pipeline where both aggregation and local optimization operate in the same centroid space.
Bias Mitigation: Design of an average-preserving push-sum aggregation with mass splitting and a deduplicated bounded buffer to mitigate bias from asymmetric information flow and prevent stale messages from dominating.
Stabilization: Development of a centroid-aligned proximal regularizer that stabilizes local updates in the compressed centroid space, mitigating non-IID drift by contracting trajectories toward a shared reference.
Theoretical and Empirical Validation: Provision of a convergence analysis and extensive experiments demonstrating improved accuracy and reduced communication costs.

4. Experimental Results

Experiments were conducted on vision datasets (CIFAR-10, CIFAR-100, and Tiny-ImageNet) under varying levels of data heterogeneity ( $\alpha \in \{0.1, 0.4, 1.0\}$ ) and delayed client scenarios.

Accuracy: PushCen-ADFL consistently outperformed communication-efficient baselines (e.g., DivShare) and full-communication baselines (e.g., Async-DFedAvg, SWIFT) under high heterogeneity.
- On Tiny-ImageNet with $\alpha=0.1$ (highly non-IID), PushCen-ADFL improved accuracy by 6.18% over DivShare and 6% over full-communication baselines in some configurations.
- On CIFAR-100 with $\alpha=0.1$ , it improved accuracy by 2.22% over DivShare.
Communication Efficiency: The framework significantly reduced per-push communication costs.
- Compared to full-precision methods, PushCen-ADFL reduced per-push payload by more than 80% (e.g., from ~449 MB to ~76 MB on CIFAR-100).
- It achieved a 1.27x reduction in bandwidth per push compared to the communication-efficient baseline DivShare on CIFAR-10.
Delayed Clients: In scenarios where 10% of clients joined late, PushCen-ADFL enabled delayed clients to achieve higher maximum test accuracy (e.g., 72.77% on CIFAR-10) with lower variability (standard deviation) compared to other methods.
Ablation Studies: Removing the regularization term or the buffer mechanism resulted in consistent accuracy drops, confirming the necessity of both components for stability and performance.

5. Significance and Claims

The paper claims that PushCen-ADFL achieves a favorable accuracy-communication trade-off in dynamic, asynchronous decentralized settings. By coupling compression, de-biased aggregation, and local stabilization in a shared centroid space, the method forms a "closed loop" that addresses the coupled challenges of bandwidth constraints, aggregation bias, and model drift simultaneously.

The authors emphasize that their approach is particularly suitable for edge computing, peer-to-peer networks, and large-scale distributed systems where device heterogeneity, network variability, and partial participation are prevalent. The work demonstrates that stable and robust training is achievable without relying on a central server or global synchronization, provided that the communication and optimization processes are co-designed to handle the specific distortions of asynchronous decentralized environments.