Technical Summary: ProCL – Continual Fine-Tuning of Large Language Models via Program Memory

1. Problem Statement

Large Language Models (LLMs) face a significant challenge in continual learning (CL) scenarios where they must sequentially adapt to new tasks using small datasets without forgetting previously acquired knowledge. This is known as the stability–plasticity dilemma.

While Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA), has become the standard for adapting LLMs under limited compute, conventional sequential LoRA updates struggle in continual settings. Standard approaches treat the low-rank subspace as a homogeneous update region. Consequently, sequential fine-tuning introduces interference between tasks, leading to catastrophic forgetting and reduced retention. Existing methods attempt to mitigate this through:

• Adapter-Modular Architectures: Adding task-specific modules or gating mechanisms (e.g., LoraHub, MOLE). While effective at isolation, these increase architectural complexity, memory footprint, and inference latency, making them impractical for long task sequences.
• Regularization-Based Methods: Enforcing orthogonality or subspace constraints (e.g., O-LoRA, DEAL). While these reduce interference, they often limit the model's effective plasticity, hindering the ability to learn new information on later tasks.

Furthermore, existing continual LoRA strategies often lack a principled mechanism to control how new knowledge is written, partitioned, or consolidated within the low-rank space, leaving them vulnerable to gradient interference.

2. Methodology: ProCL

The authors propose ProCL (Program Memory Continual LoRA), a framework inspired by Complementary Learning Systems (CLS) in neuroscience. CLS posits that the brain uses a fast learning system (hippocampus) for rapid episodic indexing and a slow learning system (cortex) for gradual consolidation of distributed representations.

ProCL translates this biological principle into a structured LoRA adaptation mechanism:

2.1 Program Memory Structure

Instead of a monolithic LoRA adapter, ProCL partitions the low-rank adapter weight matrix $W$ into $N$ distinct programs (or memory slots).

• Partitioning: $W$ is divided into $N$ sub-matrices $W^{(1)}, \dots, W^{(N)}$, where each $W^{(n)}$ acts as a specialized memory slot.
• Execution Abstraction: This is an execution-time abstraction over the same adapter tensor, ensuring full compatibility with existing LoRA implementations without adding external parameters.

2.2 Input-Conditioned Program Selection

During training, the model dynamically retrieves and composes these programs based on the input:

1. Fingerprinting: An input $x$ is compressed into a summary vector $z$ via mean pooling and passed through a lightweight encoder to generate a task query $Q$.
2. Attention Routing: A multi-head attention mechanism compares $Q$ against learnable keys associated with the $N$ programs. This produces soft routing weights $\alpha$.
3. Composition: The final executing adapter $\hat{W}(x)$ is a soft composition of the selected programs:
   $$\hat{W} = \sum_{n=1}^{N} \bar{\alpha}_n \cdot \sigma(s_n) \cdot W^{(n)}$$
   This allows similar inputs to reuse shared adapter regions while routing unrelated inputs to different parts of the parameter space, thereby localizing updates and reducing cross-task gradient interference.

2.3 Stable–Adaptive Weight Composition

The executing adapter is formed by combining the dynamically routed programs with a frozen copy of the original adapter ($W_{orig}$):
$$W_{exec} = \gamma W_{orig} + \hat{W}(x)$$

• $W_{orig}$: Provides a stable baseline representing previously learned knowledge, preventing destructive updates.
• $\gamma$: A learnable scaling parameter that balances the contribution of the stable baseline versus the new adaptations.
• Gradient Flow: Gradients flow only through the programs $\hat{W}$, while $W_{orig}$ remains frozen during backpropagation.

2.4 Consolidation

To ensure stable retention and inference efficiency, ProCL employs a consolidation step that periodically updates the persistent weight $W$:
$$W \leftarrow (1 - \lambda)W + \lambda W_{exec}$$
This step distills the input-dependent adaptations into the persistent weight, mimicking the biological process of transferring information from fast episodic memory to slow distributed memory.

2.5 Efficient Inference

At inference time, the routing mechanism is discarded. The model uses the consolidated adapter $W$ directly. This ensures zero additional inference cost compared to standard LoRA, as the complex program selection and composition occur only during training.

3. Key Contributions

• Program Memory Framework: Introduces a novel way to structure LoRA adapters into "program slots" that are dynamically retrieved via input-conditioned attention, enabling rapid, localized adaptation.
• Mechanism for Stability-Plasticity: By separating the stable baseline ($W_{orig}$) from the dynamic programs and using a consolidation step, ProCL explicitly balances the retention of old knowledge with the acquisition of new knowledge.
• Theoretical Guarantee: The authors provide a theoretical analysis (Remark 1) showing that under routing specialization, the cross-task gradient interaction is bounded and approaches zero, formally justifying the reduction in interference.
• Zero Inference Overhead: Unlike modular or gating-based approaches, ProCL consolidates all learned knowledge into a single static adapter for deployment, incurring no extra latency or memory cost at inference.

4. Experimental Results

The authors evaluated ProCL on diverse benchmarks using multiple LLM backbones (Llama-3, Qwen3, Flan-T5) and tasks (Question Answering and Text Classification).

• Question Answering (QA): On a sequential QA benchmark (BoolQ $\to$ SQuAD $\to$ AdversarialQA), ProCL achieved the highest average accuracy (69.1%) across all models, outperforming strong baselines like DEAL (66.2%) and Seq-LoRA (61.2%). Crucially, ProCL demonstrated the lowest catastrophic forgetting, with minimal performance drops on earlier tasks (e.g., BoolQ) compared to other methods which suffered drops of 20–45 points.
• Text Classification (TC): In settings with 3, 4, and 15 tasks, ProCL consistently achieved the best or second-best results in Average Accuracy (AA) and ROUGE-1 metrics. It showed particular strength in long task sequences (15 tasks), improving over DEAL by approximately 0.9–1.3%.
• Efficiency: While training involves routing, ProCL runs 1.7–2.0$\times$ faster than DEAL during training and achieves a 6–8$\times$ speedup at inference by removing the routing mechanism entirely.
• Ablation Studies: Removing the stable baseline ($W_{orig}$) or the consolidation step significantly degraded performance, confirming the necessity of both components for balancing stability and plasticity.

5. Significance and Claims

The paper claims that ProCL offers a principled mechanism for efficient and robust continual learning within the LoRA parameterization. By structuring the low-rank space with program-level memory, ProCL addresses the limitations of existing methods that either increase architectural complexity or reduce plasticity.

The authors assert that ProCL:

1. Mitigates Catastrophic Forgetting: It significantly reduces performance degradation on previously learned tasks compared to state-of-the-art continual LoRA strategies.
2. Maintains Deployment Efficiency: It achieves these gains without introducing additional inference overhead, making it suitable for real-world deployment.
3. Scales Effectively: It demonstrates robustness across different model sizes and architectures, from 3B to 8B parameters.

The paper concludes that organizing adaptation through program memory and consolidation is a promising direction for the scalable continual adaptation of LLMs, though it acknowledges limitations regarding routing collapse in highly similar tasks and fixed capacity constraints as task diversity grows.