🤖 machine learning

Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning

This paper introduces Relative Parameter Importance, a novel measure for offline, task-agnostic continual learning that dynamically balances stability and plasticity by selectively regularizing parameters based on their relative significance to both current and past tasks, thereby enabling backward knowledge transfer and achieving state-of-the-art performance in text classification.

Original authors: Malavika Suresh, Ikechukwu Nkisi-Orji, Nirmalie Wiratunga

Published 2026-08-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Malavika Suresh, Ikechukwu Nkisi-Orji, Nirmalie Wiratunga

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a brilliant, but very forgetful, robot to do new chores. You start by teaching it to wash dishes, then you teach it to fold laundry, and finally, you ask it to cook dinner. The problem? Every time you teach the robot a new skill, it seems to completely forget how to do the old ones. This is a famous headache in the world of artificial intelligence called "Catastrophic Forgetting."

To fix this, scientists usually try to balance two opposing forces: Stability (keeping the old knowledge safe) and Plasticity (being flexible enough to learn new things). Most methods try to lock down the parts of the robot's brain that handle the dishes so they don't get messed up while learning to cook. But this is like putting the robot in a straitjacket; it stops the robot from being creative or using its dish-washing skills to help it cook (a concept called "Knowledge Transfer"). Furthermore, many current solutions require the robot to have a reference guide telling it exactly which task it is doing right now, or they need the robot to remember every single dish it ever washed, which takes up too much memory and might violate privacy rules. The big question is: Can we teach a robot new tricks without making it forget the old ones, without needing a reference guide, and without filling its memory with old photos?

This paper introduces a clever new strategy called Look-Ahead (LA) continual learning, which acts like a "trial run" before the real lesson begins. Instead of just locking down the robot's brain based on what it already knows, the researchers let the robot practice the new task first, in a safe, temporary sandbox. During this practice, they watch closely to see which parts of the robot's brain are working hard to learn the new skill and which parts are just coasting.

The magic happens when they compare this "practice session" to the robot's past knowledge. They invented a new measuring stick called Relative Parameter Importance. Think of it like a traffic light for the robot's brain cells:

If a brain cell is super important for the old dishes and super important for the new cooking, it gets a Red Light. It stays frozen to protect the past.
If a brain cell is important for the new cooking but not so important for the old dishes, it gets a Green Light. It is allowed to change freely, helping the robot learn the new skill.
Here is the twist: If a brain cell was very important for the old dishes, but the robot discovers it is not needed for the new cooking, the researchers give it a Yellow Light. They allow it to change slightly! This is the paper's big surprise: they found that sometimes, even "old" parts of the brain can be tweaked to help with new tasks without causing total amnesia. This allows for "backward knowledge transfer," where learning to cook actually helps the robot remember how to wash dishes better.

The researchers tested this idea on text classification tasks, which are like teaching a computer to sort emails into "Music," "Fact," or "Negate" categories, or to understand different types of client conversations. They compared their Look-Ahead method against several other top-tier methods. The results showed that their approach was quite successful. On the "Intent Classification" task, their method achieved an average score (called an Overall F1 score) of 34.20, beating the next best method which scored 31.19. It also significantly reduced "Catastrophic Forgetting," dropping the score for lost knowledge from 71.86 down to 67.44. On the "Behaviour Classification" task, they reached an overall score of 50.48, again outperforming the competition.

The paper suggests that this method works because it doesn't just blindly protect old knowledge; it intelligently decides which old knowledge is actually getting in the way and which can be safely adjusted. While the results on text sorting are promising, the authors also ran some early experiments on text generation (like having a robot write math stories or code). These initial tests suggested that while the method works in theory, balancing the "protect the past" and "learn the new" forces is still tricky for these complex generative models, as the training can become unstable if the balance isn't just right. However, the core idea—that looking ahead before locking things down helps the robot learn better—seems to be a solid step forward for teaching AI to learn continuously without needing to memorize every single thing it has ever seen.

Technical Summary: Relative Parameter Importance in Task-Agnostic Replay-Free Continual Learning

Problem Statement

The paper addresses the challenge of Continual Learning (CL) in deep neural networks under two strict constraints:

Replay-Free: No access to training data from prior tasks.
Task-Agnostic: No access to task identifiers (task-ids) at inference time.

The core difficulty lies in balancing the stability-plasticity trade-off: preventing Catastrophic Forgetting (CF) of previously learned abilities while maintaining plasticity to learn new tasks and enabling Knowledge Transfer (KT) between old and new tasks. Existing solutions face limitations:

Replay methods violate privacy and incur high memory costs.
Parameter masking requires task-ids, which are often unavailable.
Gradient projection and traditional regularisation methods often over-constrain the model. Specifically, standard parameter importance methods (e.g., EWC, MAS) permanently freeze parameters deemed important for past tasks. This prevents necessary updates even when those parameters could facilitate backward knowledge transfer or when the new task is highly correlated with the old one, leading to sub-optimal performance and reduced plasticity.

Methodology: Look-Ahead (LA) Continual Learning

The authors propose a novel method called Look-Ahead (LA) continual learning, which introduces a Relative Parameter Importance measure to enable informed regularisation without replay or task-ids.

Core Concept: Relative Importance

Unlike traditional methods that estimate importance solely with respect to past tasks, LA estimates the importance of a parameter with respect to both the past tasks and the current task.

Past Importance ( $\alpha_{k-1}$ ): Estimated using the sensitivity of the output function to the parameter (via MAS) on the accumulated history of tasks.
Current Importance ( $\alpha^{la}_k$ ): Estimated by training a temporary auxiliary model solely on the new task data (the "Look-Ahead" phase) without regularisation.

The Relative Importance ( $\alpha^{rel}_k$ ) is calculated as:
$\alpha^{rel}_k = \frac{\alpha_{k-1}}{\alpha_{k-1} + \alpha^{la}_k + \epsilon}$
where $\epsilon$ ensures numerical stability.

High $\alpha^{rel}_k$ : Indicates the parameter is crucial for past tasks relative to the new task. These are heavily regularised to prevent forgetting.
Low $\alpha^{rel}_k$ : Indicates the parameter is more relevant for the new task or shared knowledge, even if it had some past importance. These are allowed to update more freely to enable plasticity and backward KT.

Training Phases

For each new task $k$ (after the first), the training occurs in two phases:

Look-Ahead (LA) Phase: A temporary model is initialised from the frozen past model and trained only on the new task data. This phase identifies the gradient update path and estimates the current task importance ( $\alpha^{la}_k$ ).
Main Continual Learning (MCL) Phase: The main model is trained on the new task using a modified regularisation objective. The regularisation weight $\alpha'_k$ $α_{k}^{'}$ is dynamically adjusted based on $\alpha^{rel}_k$ $α_{k}^{r e l}$ :
- If $\alpha^{rel}_k > \tau_{\alpha^{rel}}$ (high relative importance), the constraint is scaled up ( $\lambda_{up}$ ) to strictly prevent forgetting.
- If $\alpha^{rel}_k \le \tau_{\alpha^{rel}}$ (low relative importance), the constraint is scaled down ( $\lambda_{down}$ ) to allow updates.

The final loss function for the MCL phase is:
$L_{mcl} = L_{task} + \frac{\lambda}{2} \sum_j \alpha'_{k,j} (\theta_{k-1,j} - \theta_{k,j})^2$

Hyper-parameter Selection

The paper acknowledges the difficulty of tuning hyper-parameters without past data. It proposes a Continual Hyper-parameter Search (CHSF) framework adapted for the LA method. This involves an iterative search on the validation set of the current task to find the optimal $\lambda$ , $\lambda_{up}$ , $\lambda_{down}$ , and threshold $\tau_{\alpha^{rel}}$ that maximise performance while maintaining a desired level of stability.

Key Contributions

Novel Measure: Introduction of Relative Parameter Importance, which quantifies a parameter's significance for both current and past tasks, enabling a more granular approach to regularisation.
Backward Knowledge Transfer: Unlike existing methods that rigidly protect high-past-importance parameters, LA allows these parameters to be updated if they have low relative importance, thereby facilitating backward KT.
Task-Agnostic, Replay-Free Framework: The method operates effectively without access to prior data or task identifiers, addressing a practical constraint often ignored in CL research.
Empirical Validation: Demonstration of state-of-the-art performance on text classification benchmarks (Intent and Behaviour classification) and preliminary insights for extending the method to generative models.

Experimental Results

The method was evaluated on two text classification datasets:

Class-Incremental Learning (CIL): Intent classification (home assistant queries).
Domain-Incremental Learning (DIL): Behaviour classification in motivational interviewing conversations across different domains.

Performance Metrics: Overall F1 score (Ov), Catastrophic Forgetting (CF), Positive Backward Transfer (BWT+), and Forward Transfer (FWT).

Findings:

Superior Overall Performance: The LA method (specifically LA-MAS) achieved the highest overall F1 scores on both CIL and DIL tasks compared to baselines like AdaBOP, UPGD, RP2F, and ANCL (Auxiliary Network based CL).
Balanced Trade-off: While gradient projection methods (AdaBOP) reduced forgetting but severely hampered plasticity (negative FWT), and some regularisation methods improved plasticity but suffered high forgetting, LA successfully balanced both.
Backward Transfer: LA demonstrated significantly higher Positive Backward Transfer (BWT+) compared to other methods, confirming that allowing updates to specific parameters aids in transferring knowledge from new tasks back to old ones.
Generative Models: In preliminary experiments on Llama-3.2 for mathematical reasoning, the authors found that standard parameter importance regularisation led to training instability. They concluded that balancing cross-entropy and regularisation objectives is a critical next step for generative models, but the importance estimation method itself correlated well with parameter changes.

Significance and Claims

The paper claims that the Look-Ahead approach provides a more informed regularisation strategy than existing methods. By distinguishing between parameters that are only important for the past and those that are relatively more important for the past, the method avoids the "over-constraining" of the network.

The authors emphasize that this approach:

Enables backward knowledge transfer, a capability often lost in strict stability-focused methods.
Offers a practical solution for real-world scenarios where data privacy prohibits replay and task-ids are unavailable.
Provides a foundation for future work in continual pre-training and fine-tuning of generative models, suggesting that relative importance can guide the adaptation of large language models without catastrophic forgetting of core capabilities.

The work is presented as a step toward bridging the gap between Continual Learning and Multi-Task Learning by dynamically managing the stability-plasticity trade-off at the parameter level.