Technical Summary: PeCL – Token-Level Differential Privacy in Memory Sculpting for Continual Learning

1. Problem Statement

Continual Learning (CL) enables models, particularly Large Language Models (LLMs), to sequentially acquire new knowledge without forgetting previous tasks. However, this capability introduces significant privacy challenges. As models assimilate diverse and evolving datasets, they inherently accumulate vast amounts of information, including sensitive personal or proprietary data. Once embedded, such data becomes difficult to audit or erase, posing legal and ethical risks.

Existing privacy solutions face critical limitations in the CL context:

• Uniform Privacy Budgets: Traditional Differential Privacy (DP) methods (e.g., DPSGD) apply a uniform noise budget across all data. This indiscriminate protection degrades model utility by injecting excessive noise into non-sensitive, general knowledge while potentially under-protecting highly sensitive entities.
• Static vs. Dynamic: Many token-level DP approaches (e.g., PMixED) are designed for static datasets and do not address the catastrophic forgetting inherent in sequential learning.
• Selective Forgetting: Achieving "selective forgetting" (removing sensitive data while retaining general knowledge) without degrading overall utility is theoretically challenging. Current methods often fail to differentiate between varying levels of sensitivity within data, leading to either insufficient protection or unnecessary utility loss.

The core challenge lies in balancing three conflicting goals: discerning fine-grained sensitivity at the token level, protecting privacy without corrupting historical knowledge, and integrating these mechanisms into a dynamic, sequential learning paradigm.

2. Methodology: The PeCL Framework

The authors propose PeCL (Privacy-enhanced Continual Learning), a framework built on the principle of "forgetting what's sensitive and remembering what matters." It comprises two core modules: Token-level Dynamic Differential Privacy (TDP) and Privacy-Guided Memory Sculpting (PMS).

2.1 Token-level Dynamic Differential Privacy (TDP)

TDP adaptively allocates privacy budgets based on the semantic sensitivity of individual tokens rather than applying a uniform budget.

• Sensitivity Scoring: For each token $t_i$, a sensitivity score $Score(t_i) \in [0, 1]$ is computed using a weighted combination of two indicators:
  1. Model Uncertainty ($Score_1$): The negative log-likelihood of the token given its context. High uncertainty suggests the token is rare, context-dependent, or potentially sensitive.
  2. Contextual Discriminativeness ($Score_2$): Adapted from TF-IDF, this measures how specific a token is to a particular task versus the entire corpus. Tokens highly specific to a single task (low cross-task support) are flagged as more sensitive.
• Dynamic Budget Allocation: The privacy budget $\epsilon_i$ for each token is inversely related to its sensitivity score. High-sensitivity tokens receive a smaller $\epsilon_i$ (stronger protection), while low-sensitivity tokens receive a larger $\epsilon_i$ to preserve utility.
• Noise Injection: Calibrated Gaussian noise is injected into the input embeddings of tokens based on their specific $\epsilon_i$. This occurs at the embedding level before self-attention, ensuring Local Differential Privacy (LDP) at the source.

2.2 Privacy-Guided Memory Sculpting (PMS)

While TDP protects input embeddings, PMS reshapes the model's internal parameters to prevent the memorization of sensitive information and preserve task-invariant knowledge.

• Memory Regularization: To mitigate catastrophic forgetting, the framework adapts the Elastic Weight Consolidation (EWC) principle to the LoRA parameter space. It calculates a task-specific importance score based on parameter updates and input activations. Crucially, the regularization strength ($\lambda_{dyn}$) is dynamically modulated by the average sensitivity of the current task. Tasks with low sensitivity are regularized more strictly to retain knowledge, while high-sensitivity tasks allow for greater flexibility.
• Privacy-Aware Unlearning: A specific loss term ($L_{unlearn}$) is introduced to actively suppress the reinforcement of high-sensitivity tokens. For tokens exceeding a sensitivity threshold $\theta$, the cross-entropy loss is weighted by $(Score(t_i) - \theta)$, encouraging the model to "unlearn" or softly suppress these specific traces.
• Dual-Layer Defense: TDP provides "soft suppression" via noise for near-threshold tokens, while PMS triggers explicit parameter-level unlearning for tokens exceeding the threshold.

3. Key Contributions

The paper outlines three primary contributions:

1. Token-Level Dynamic DP Strategy: A novel mechanism that adaptively allocates privacy budgets based on the semantic sensitivity of individual tokens, enabling fine-grained privacy protection in continual learning.
2. Privacy-Guided Memory Sculpting: A module that intelligently forgets sensitive information from model memory while explicitly preserving task-invariant historical knowledge, addressing the trade-off between privacy and catastrophic forgetting.
3. Empirical Validation: Demonstration that the framework achieves a superior balance between privacy and utility, outperforming state-of-the-art baselines in continual learning scenarios.

4. Experimental Results

The authors evaluated PeCL on a multi-task benchmark comprising six domains (FOMC, Yelp, AGNews, Amazon, Mentill, Yahoo) formulated as text classification tasks.

• Performance Metrics: The framework was assessed using Average Accuracy (Avg), Last Accuracy (Last), and Backward Transfer (BWT).
• Comparative Analysis: PeCL outperformed various baselines, including:
  • Standard CL methods combined with uniform DP (ER+DPSGD, EWC+DPSGD, GEM+DPSGD).
  • Methods using frozen embeddings with noise (FN).
  • The token-level DP baseline PMixED (which lacks CL-specific memory sculpting).
  • Sequential Finetuning (SeqFT) variants, which suffered severe catastrophic forgetting.
• Key Findings:
  • PeCL achieved the highest scores across all metrics, surpassing even the Multi-Task Learning (MTL) upper bound in some accuracy metrics, despite operating in a sequential setting.
  • Ablation studies confirmed that removing either TDP or PMS significantly degraded performance and increased forgetting, validating the necessity of both components.
  • The framework demonstrated robustness across different task arrival orders (natural, reversed, and shuffled).

5. Significance and Claims

The paper claims that PeCL represents a significant advancement in privacy-preserving continual learning by moving beyond "one-size-fits-all" privacy approaches. By integrating fine-grained sensitivity analysis with dynamic memory sculpting, the framework successfully navigates the tension between protecting private entities and maintaining model utility.

The authors emphasize that their approach allows for:

• Robust Privacy: Strong protection for private entities without the utility degradation associated with uniform noise injection.
• Effective Knowledge Retention: Explicit preservation of task-invariant knowledge to mitigate catastrophic forgetting, a common failure point in privacy-constrained CL.
• Practical Applicability: The ability to function in dynamic, sequential data streams where data arrives incrementally, a setting where many existing DP methods fail.

The paper concludes by acknowledging limitations, such as the accumulation of privacy budgets in long contexts and the need for empirical validation against specific attack vectors (e.g., membership inference), suggesting future work in Rényi Differential Privacy and formal theoretical grounding.