Abstraction as a Memory-Efficient Inductive Bias for Continual Learning

1. Problem Statement

The paper addresses the challenges of Online Continual Learning (OCL) for Large Language Models (LLMs). In real-world scenarios, data arrives as a non-stationary stream where each instance is observed only once, and task boundaries are absent.

• Core Challenges:
  • Catastrophic Forgetting: New information overwrites previously learned knowledge.
  • Plasticity Loss: The model's ability to adapt to new tasks degrades over time.
• Limitations of Existing Solutions:
  • Regularization: Often impedes the acquisition of new tasks.
  • Experience Replay (ER): The current state-of-the-art for stability, but it requires storing past data in a replay buffer. In strictly online settings, maintaining these buffers incurs prohibitive memory overhead and is often infeasible.
• The Gap: There is a need for a method that stabilizes learning and prevents forgetting in strictly online regimes without relying on explicit memory buffers or storing past instances.

2. Methodology: Abstraction-Augmented Training (AAT)

The authors propose Abstraction-Augmented Training (AAT), a lightweight, loss-level modification that introduces an inductive bias toward shared relational structures rather than instance-specific details.

Core Mechanism

Instead of storing past data, AAT leverages the concept of abstraction (inspired by human cognitive science and Structure-Mapping Theory) to generalize across examples.

• Dual-Objective Loss: When a model encounters a new batch of data, it is presented with two inputs simultaneously:
  1. Concrete Instance ($x_{inst}$): The raw data point.
  2. Symbolic Abstraction ($x_{abs}$): A representation where entity-specific details are masked or generalized (e.g., replacing specific names with placeholders or categories).
• Training Loop:
  • Step 1 (Joint Optimization): The model performs a forward pass on both the instance and the abstraction. The loss is a weighted combination:
    $$L(\theta) = \alpha \cdot \text{NLL}(\hat{y}_{abs}, y_{abs}) + (1 - \alpha) \cdot \text{NLL}(\hat{y}_{inst}, y_{inst})$$
    Where $\alpha$ is a hyperparameter (default 0.5). This biases the gradient updates toward the shared structural features ($\phi_r$) rather than noisy entity-specific features ($\phi_e$).
  • Step 2 (Local Replay): To ensure factual details are not lost, the model performs $n-1$ additional local updates (local replay) on the concrete instance only. This consolidates specific knowledge without requiring a global buffer of past data.

Theoretical Justification

The paper provides a theoretical analysis showing that abstraction reduces gradient variance.

• In standard OCL, gradients vary wildly due to changing entity identities ($\phi_e$), causing destructive interference.
• By optimizing on the abstraction, the model amplifies updates to the shared relational structure ($\phi_r$), which is invariant across samples.
• This acts as an implicit regularizer, smoothing the loss landscape and reducing interference between consecutive updates, effectively mimicking the stability benefits of Experience Replay without the memory cost.

3. Key Contributions

1. AAT Algorithm: A memory-efficient, loss-level mechanism for OCL that eliminates the need for replay buffers by jointly optimizing instances and their abstractions.
2. New Benchmarks:
   • Relational Cycle Benchmark (RCB): A controlled dataset derived from knowledge graphs where abstraction is achieved via entity masking. It tests the ability to infer hidden edges in relational cycles.
   • Narrative Abstraction Benchmark (NAB): A dataset based on stories and proverbs where abstraction is expressed through shared narrative motifs (high-level relational structures) rather than surface-level similarities.
3. Empirical & Theoretical Evidence: Demonstrates that structural abstraction is a powerful inductive bias that balances stability (retention) and plasticity (adaptation) more effectively than instance-only learning and matches strong ER baselines.

4. Experimental Results

The authors evaluated AAT on Qwen2.5-1.5B and SmolLM-1.7B models across the RCB and NAB benchmarks.

• Performance vs. Baselines:
  • AAT outperformed standard instance-only training (SFT) significantly.
  • AAT achieved performance comparable to or exceeding Experience Replay (ER) with buffer sizes of 50–100, despite using zero additional memory for storing past data.
  • On the SmolLM model, AAT reduced forgetting by 80% on unknown edges compared to SFT and achieved the lowest overall forgetting compared to ER.
• Generalization:
  • AAT showed superior performance on unknown edges (inductive generalization) in RCB and unknown narratives in NAB, indicating it learns the underlying relational rules rather than memorizing specific facts.
• Ablation Studies:
  • Abstraction Level: Entity masking (strong abstraction) performed better than category-based or random abstraction, confirming that meaningful structural preservation is key.
  • Hyperparameters: An abstraction weight ($\alpha$) of 0.5 provided the best balance between generalization and factual retention.
  • Loss Landscape: Analysis showed AAT creates a smoother loss landscape with lower variance compared to standard SFT, confirming the theoretical claim of reduced gradient interference.

5. Significance and Conclusion

This work challenges the prevailing notion that Experience Replay is the only viable solution for stable continual learning in memory-constrained environments.

• Memory Efficiency: AAT offers a "memory-free" alternative to ER, making it highly suitable for edge devices or strictly online streaming scenarios where storage is limited.
• Cognitive Alignment: By mimicking human cognitive strategies (forming abstract schemas rather than storing isolated instances), the method provides a biologically plausible path to robust AI learning.
• Scalability: The results suggest that abstraction acts as a model-agnostic inductive bias, offering a scalable solution for the continual learning of large language models in non-stationary environments.

In summary, the paper demonstrates that structural abstraction is a powerful, memory-efficient inductive bias that stabilizes online learning, reduces catastrophic forgetting, and enhances generalization without the computational and storage costs of replay buffers.