FGGM: Fisher-Guided Gradient Masking for Continual Learning

Technical Summary: Fisher-Guided Gradient Masking (FGGM) for Continual Learning

1. Problem Statement

The paper addresses catastrophic forgetting in Large Language Models (LLMs) during Continual Learning (CL). As LLMs are adapted to a stream of new tasks, they tend to abruptly overwrite parameters critical for previously learned tasks, leading to a loss of prior knowledge.

Existing mitigation strategies face significant limitations:

• Replay-based methods require storing or generating historical data, raising privacy and scalability concerns.
• Parameter-freezing methods (e.g., LoRA, Prompt Tuning) often freeze base parameters, potentially limiting downstream task performance.
• Regularization-based methods like EWC rely on historical data, while others like Orthogonal Gradient Descent (OGD) impose prohibitive memory costs for full fine-tuning of LLMs.
• MIGU, a recent replay-free method, selects parameters based on empirical output magnitudes but lacks a theoretical foundation for estimating parameter importance.

2. Methodology: FGGM Framework

The authors propose Fisher-Guided Gradient Masking (FGGM), a framework that mitigates catastrophic forgetting by strategically selecting parameters for updates using diagonal Fisher Information without requiring access to historical data. The framework operates in two phases:

A. Fisher-Guided Mask Initialization
For a new task $T_t$, FGGM estimates the diagonal elements of the Fisher Information Matrix (FIM) using the new task's data. The diagonal FIM entry $\hat{F}^{(t)}_i$ quantifies the importance of parameter $\theta_i$ by measuring the squared gradient of the log-likelihood:
$$\hat{F}^{(t)}_i = \frac{1}{M_t} \sum_{j=1}^{M_t} \left( \frac{\partial \log p(y_j|x_j, \theta)}{\partial \theta_i} \right)^2$$
To handle the non-stationary distribution of parameter importance across layers, FGGM employs adaptive thresholding. A binary mask $M^{(t)}$ is generated where a parameter is retained (value 1) if its Fisher score exceeds the $(1-\alpha)$-th quantile of the empirical Fisher values, and masked (value 0) otherwise.

B. Mask Normalization via Input-Dimension Aggregation (IA)
A key innovation is the normalization of masks based on neural network architecture. For fully connected layers with weight matrix $W \in \mathbb{R}^{D_{out} \times D_{in}}$, Fisher scores are aggregated across the input dimension for each output neuron:
$$\hat{F}^{(t)}_r = \sum_{c=1}^{D_{in}} \hat{F}^{(t)}_{r,c}$$
This strategy preserves the functional integrity of output neurons as atomic computational units. The paper argues that aggregating across inputs (IA) is superior to aggregating across outputs (OA) because it prevents the dispersion of feature-specific sensitivity signals and maintains distinct task boundaries, whereas OA can blur task distinctions and increase forgetting.

C. Gradient Projection
During backpropagation, raw gradients $g^{(t)}$ are element-wise multiplied by the binary mask $M^{(t)}$:
$$\tilde{g}^{(t)} = g^{(t)} \odot M^{(t)}$$
This restricts updates to only the critical parameters identified by the Fisher information, effectively freezing non-critical regions of the parameter space.

3. Key Contributions

1. Novel Framework: Introduction of FGGM, a replay-free method for LLM continual learning that relies on theoretically grounded Fisher Information rather than empirical magnitude observations.
2. Theoretical Importance Estimation: A mechanism that directly measures parameter-task relevance via Fisher Information, enabling dynamic gradient gating to balance plasticity (acquiring new capabilities) and stability (retaining pre-existing knowledge).
3. Architectural Normalization: The proposal of Input-Dimension Aggregation (IA) to align mask generation with the functional structure of feedforward layers, ensuring robust parameter selection.
4. Empirical Validation: Demonstration that FGGM outperforms state-of-the-art baselines, including MIGU, on standard benchmarks without compromising model adaptability.

4. Experimental Results

Experiments were conducted on the TRACE benchmark (covering general capabilities and diverse downstream tasks) and code generation tasks (HumanEval/BBH) using Qwen2 models (1.5B and 7B scales).

• General Capabilities (Stability): On the aggregate General metric (MMLU, BBH, TyDiQA, etc.), FGGM achieved 55.75, outperforming MIGU (55.21) and significantly surpassing SFT (50.89) and LoRA (49.48). Notably, FGGM outperformed replay-based methods (REP, EWC) on General metrics despite not accessing historical data, highlighting the privacy and scalability advantages of the approach.
• TRACE Overall Performance (Plasticity): FGGM achieved 46.00 on TRACE-OP, showing a 4.4% relative improvement over MIGU (44.08) and a 9.6% relative improvement over SFT (49.22) in retaining general capabilities while learning new tasks.
• Code Generation: In experiments on the HumanEval benchmark, FGGM consistently achieved higher accuracy across all evaluation metrics compared to SFT and MIGU, while simultaneously demonstrating reduced forgetting on the BBH benchmark.
• Scaling: The performance gap between FGGM and MIGU widened as model size increased from 1.5B to 7B, indicating FGGM's superior scalability.
• Ablation Studies:
  • Dimension Aggregation: Removing Input-Dimension Aggregation (w/o IA) reduced General performance from 55.75 to 54.85. Applying Output-Dimension Aggregation (OA) caused a more severe drop (53.16), confirming IA's necessity for preserving task-specific sensitivities.
  • Masking Rate ($\alpha$): A trade-off was observed where smaller $\alpha$ (more updates) increased plasticity (TRACE-OP) but decreased stability (General). The authors set $\alpha=0.7$ for fair comparison with MIGU.

5. Significance and Claims

The paper claims that FGGM establishes an effective solution for continual learning in LLMs by offering a mathematically principled approach to parameter importance estimation. Unlike magnitude-based methods like MIGU, FGGM utilizes Fisher Information to quantify the contribution of parameters to task loss.

The authors emphasize that FGGM achieves a dynamic equilibrium between stability and plasticity without the need for historical data, addressing the privacy and scalability bottlenecks of replay-based methods. The results suggest that principled gradient masking strategies are a versatile and effective tool for maintaining LLM performance over time, particularly in scenarios where historical data is unavailable or restricted. The paper concludes that FGGM represents a state-of-the-art approach for stability among methods that do not access previous task data.