🤖 machine learning

Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem

This paper proposes Federated Queue-regulated Continual Learning (FedQCL), a novel framework that casts Federated Continual Learning as a stochastic control problem using Lyapunov drift-plus-penalty optimization to explicitly manage forgetting through virtual queues, thereby achieving superior accuracy and reduced forgetting on heterogeneous benchmarks without requiring gradient projection or extra communication overhead.

Original authors: Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban, Ranjitha Prasad, B. N. Bharath

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nazreen Shah, Naveen Kumar Reddy Somireddy, Zubair Shaban, Ranjitha Prasad, B. N. Bharath

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, machines are increasingly asked to learn not just once, but continuously, adapting to new information as it arrives while remembering what they learned before. This challenge is known as continual learning. Imagine a student who must take a new exam every day, but the classroom rules forbid them from keeping their old textbooks or notes. They must study the new material without losing the ability to answer questions from previous days. In the real world, this scenario plays out across millions of devices, from smartphones to medical sensors, each holding private data that cannot be shared with a central server. This setup is called federated learning, where many devices work together to train a single smart model without ever revealing their private information. The difficulty arises when these devices face different types of data and changing tasks over time; the model often forgets old lessons as it learns new ones, a problem known as catastrophic forgetting.

Researchers at IIIT Delhi and IIT Dharwad have tackled this specific problem by viewing the learning process not just as a series of math problems, but as a dynamic system that needs constant regulation. They developed a new method called FEDQCL, which treats the forgetting of information like a physical quantity that can be measured and controlled. Instead of trying to force the model to remember everything through rigid rules, their approach uses a virtual tracking system. Think of this system as a set of local counters on each device that keep a running tally of how much the model is forgetting about past tasks. If the forgetting gets too high, the system automatically adjusts the learning process to slow down and protect the old knowledge. This allows the model to stay flexible enough to learn new things while remaining stable enough to retain what it already knows, all without the devices needing to send extra data back and forth.

The core of this work lies in how the researchers managed the tension between learning new tasks and remembering old ones. In previous approaches, devices would often try to solve each new task in isolation, or they would use complex mathematical tricks to prevent changes to the model that might hurt past performance. These methods often struggled when the data on different devices was very different from one another, leading to a situation where the global model became confused or unstable. The new method introduces a concept called a virtual queue. At every step of the learning process, each device checks how much its current learning is hurting its performance on past tasks. If the performance drops, the virtual queue grows, signaling that the device is forgetting too much. This signal then acts as a gentle brake on the learning process, encouraging the model to prioritize stability. If the model is doing well and not forgetting, the queue stays small, allowing the model to learn new things quickly.

What makes this approach distinct is that it does not require the devices to share their private data or even send extra information to the central server to make this work. The tracking happens entirely on the local device, preserving privacy while still contributing to a stable global model. The researchers tested this system on standard image recognition tasks, where the model had to learn to identify different groups of objects in a sequence. They compared their method against several existing techniques, including those that rely on storing past examples or using complex mathematical projections to protect old knowledge. The results showed that their new method consistently achieved higher accuracy on the tasks while significantly reducing the amount of forgotten information. In tests involving thousands of images across hundreds of categories, the new approach outperformed the best existing methods, proving that managing forgetting as a continuous, regulated process is more effective than trying to fix it after the fact.

The study also explored how the method behaves under different conditions, such as when the data on each device is very uneven or when the memory available to store past examples is limited. Even when the data was highly unbalanced, with some devices seeing mostly one type of image and others seeing something completely different, the method remained robust. It maintained high accuracy and low forgetting, whereas other methods struggled to keep the model stable. The researchers found that they could tune a single control knob to decide how much the model should prioritize learning new things versus remembering old ones. This flexibility allowed them to find a sweet spot where the model could adapt quickly without losing its foundation. The findings suggest that treating the learning process as a dynamic control problem, where forgetting is actively monitored and regulated in real-time, offers a powerful way to build intelligent systems that can learn continuously in the real world.

This work does not claim to have solved every problem in machine learning, but it provides a clear and effective framework for one of the most persistent challenges: keeping a model stable while it learns. By shifting the focus from static rules to dynamic regulation, the researchers have shown that it is possible to balance the need for plasticity with the need for stability. The method works without requiring extra communication between devices, making it practical for real-world applications where bandwidth and privacy are concerns. As machines continue to be deployed in environments where data changes constantly, approaches like this one, which can adapt and regulate themselves without human intervention, will likely become essential for building reliable and long-lasting artificial intelligence.

Technical Summary: Federated Continual Learning as a Distributed Drift-Plus-Penalty Control Problem

Problem Formulation

Federated Continual Learning (FCL) addresses the challenge of training machine learning models across distributed clients where data arrives sequentially and tasks evolve over time. Unlike standard Federated Learning (FL), which assumes static datasets, FCL requires models to adapt to new tasks while retaining knowledge of previous ones without access to raw historical data. This setting introduces two primary challenges: catastrophic forgetting (the rapid loss of previously acquired knowledge) and client drift (divergent local updates caused by non-IID data distributions).

Existing approaches typically formulate FCL as a sequence of per-task optimization problems, relying on heuristic mechanisms such as replay buffers, regularization (e.g., EWC, LwF), or gradient projection. However, the authors argue that forgetting in FCL is inherently a long-term, distributed phenomenon arising from the interaction of temporal task evolution and cross-client heterogeneity. Current methods often optimize myopic, round-wise objectives that fail to explicitly regulate the accumulation of forgetting over time and across clients. Furthermore, many existing methods incur significant communication overhead (e.g., gradient projection) or impose rigid constraints that limit model plasticity.

Methodology: FEDQCL

The paper proposes Federated Queue-regulated Continual Learning (FEDQCL), a framework that casts FCL as a stochastic control problem. The core innovation is the application of Lyapunov drift-plus-penalty (DPP) optimization to explicitly regulate the stability-plasticity trade-off.

Key Components:

Virtual Queues for Forgetting Tracking:
Instead of treating forgetting as a static constraint, FEDQCL introduces virtual queues $Q_i^{(k)}[t]$ at each client $i$ for every past task $k$ . These queues track the accumulation of constraint violations (i.e., the increase in replay loss) over time. The queue update rule is:
$Q_i^{(k)}[t] = \max \left\{ Q_i^{(k)}[t-1] + \left[ \hat{\Phi}_i^{(k)}(w_i^{(t)}) - \hat{\Phi}_i^{(k)}(w^{(t-1)}) - \delta \right], 0 \right\}$
where $\hat{\Phi}$ represents the local replay loss, $w^{(t-1)}$ is the global model from the previous task, and $\delta$ is a tolerance threshold.
Drift-Plus-Penalty (DPP) Objective:
At each communication round, clients minimize a local DPP objective that balances current-task performance (plasticity) against the drift of the virtual queues (stability):
$\text{DPP}_{V,i}^{(t)} = \min_{w^{(t)}} \left[ V \Phi_i(w^{(t)}; D_i^{(t)}) + \Delta L_i[t] \right]$
Here, $V$ is a control parameter governing the trade-off, and $\Delta L_i[t]$ is the Lyapunov drift. Using an upper bound on the drift, the optimization simplifies to:
$V \Phi_i(w; D_i^{(t)}) + \sum_{k=1}^{t-1} Q_i^{(k)}[t-1] \Delta \hat{\Phi}_i^{(k)}(w, w^{(t-1)})$
The first term encourages learning the current task, while the second term penalizes increases in replay loss on past tasks, weighted by the current queue state.
Per-Round Queue Updates:
A critical design choice in FEDQCL is updating the virtual queues at the granularity of communication rounds rather than task boundaries. Since forgetting evolves continuously through iterative local updates and partial participation, per-round updates allow for fine-grained tracking and timely correction of constraint violations.
Communication Efficiency:
The method requires no additional communication overhead beyond standard FL (e.g., FedAvg). Virtual queues are maintained locally at each client, and no gradient projection or high-dimensional embeddings are exchanged.

Key Contributions

Control-Theoretic Formulation: The paper is the first to cast FCL as a queue-stabilization problem, modeling forgetting as a controllable, accumulative process across time and clients.
Single-Parameter Trade-off Control: FEDQCL achieves a principled balance between adaptation and retention using a single hyperparameter ( $V$ ), avoiding the complex tuning required by projection-based or multi-constraint methods.
Privacy and Efficiency: The framework enforces constraints implicitly through local queue dynamics, preserving privacy (no raw data or complex embeddings shared) and maintaining communication efficiency comparable to FedAvg.
Fine-Grained Regulation: By updating queues per communication round, the method captures the continuous nature of forgetting in distributed settings, offering superior control over client drift compared to task-bound formulations.

Empirical Results

The authors evaluated FEDQCL on standard benchmarks: Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet under non-IID data distributions (Dirichlet parameter $\alpha$ ).

Performance: FEDQCL outperformed state-of-the-art baselines, including replay-based methods (DER++, ER-FL), regularization-based methods (EWC-FL, LwF-FL), and gradient projection methods (FOT).
- On Split-CIFAR-100, FEDQCL achieved 54.52% average accuracy with 0.05 forgetting, surpassing the best baseline (DER++-FL: 51.04% accuracy, 0.06 forgetting).
- On Split-TinyImageNet, it achieved 33.37% accuracy with 0.13 forgetting, compared to 31.02% and 0.17 for DER++-FL.
Robustness: The method demonstrated robustness across varying levels of data heterogeneity ( $\alpha \in \{0.001, 10, 10^5\}$ ) and memory buffer sizes ( $M \in \{200, 500, 1000\}$ ).
Pareto Optimality: In plasticity-stability trade-off plots, FEDQCL consistently occupied the upper-left region of the Pareto front, indicating a superior balance between high accuracy and low forgetting compared to methods that sacrifice one for the other.
Ablation Studies: The study confirmed that the control parameter $V$ effectively interpolates between forgetting-averse and plasticity-focused regimes. The queue dynamics were shown to adapt to varying constraints ( $\delta$ ) and heterogeneity levels without structural degradation.

Significance and Claims

The paper claims that treating FCL as a dynamic control problem, rather than a sequence of static optimization tasks, is essential for managing the complex interplay of forgetting and heterogeneity in distributed systems. By leveraging Lyapunov optimization, FEDQCL provides a systematic and interpretable mechanism to regulate forgetting without the computational or communication burdens of existing advanced methods.

The authors emphasize that their approach is projection-free and privacy-preserving, making it scalable to realistic federated settings where resources and bandwidth are constrained. The work highlights that explicit modeling of forgetting as an accumulative process via virtual queues allows for more effective regulation of the stability-plasticity trade-off than heuristic constraints or myopic updates. The results suggest that FEDQCL offers a principled alternative to current FCL paradigms, achieving state-of-the-art performance while significantly reducing forgetting under heterogeneous and non-stationary data distributions.