Technical Summary: SPRING

Problem Formulation

The paper addresses the challenge of continual personalization for Large Language Models (LLMs). While existing personalization methods effectively adapt models to user preferences, they typically assume these preferences are static (time-invariant). In reality, user interests evolve continuously, creating a "preference drift."

Current approaches struggle with this dynamic nature:

• Non-parametric methods (e.g., standard Retrieval-Augmented Generation) rely on static or expanding interaction histories. They often suffer from weak recency signals, high storage costs, and the inability to distinguish genuine preference shifts from transient noise or irrelevant context.
• Parametric methods (e.g., fine-tuning user-specific LoRA adapters) face the risk of catastrophic forgetting when updated on noisy streams and often fail to distinguish between genuine preference evolution and temporary context.
• Standard Continual Learning (CL) approaches, designed for task-incremental settings with clear boundaries, are suboptimal here because user interaction streams are continuous and noisy. Indiscriminate updates on all incoming data can cause the model to overfit to transient contexts rather than learning persistent preference patterns.

The core problem is to develop a framework that can selectively adapt to genuine preference drifts while filtering out noise and preserving historical context without catastrophic forgetting.

Methodology: SPRING

The authors propose SPRING (Selective Parametric adaptation and Retrieval-Interpolated Generation), a semi-parametric framework that operates in two distinct phases: training and inference.

1. Training: Drift-Driven Selective Adaptation

Instead of updating the user adapter on the entire incoming stream, SPRING employs a three-step pipeline to identify and learn only from high-novelty interactions:

• Drift Sample Selection: A scoring function $S(q, a)$ is used to quantify sample novelty relative to the current adapter. The score combines two terms:
  • Novelty Term: $-\log(p_{adap} / p_{base})$. This prioritizes samples where the current adapter performs significantly differently from the base model, indicating a potential shift in preference.
  • Quality Term: $\alpha \log p_{base}$. This ensures linguistic validity, filtering out noisy outliers or gibberish that might appear novel simply because they are difficult to predict.
  • Only the top $p\%$ of samples (e.g., 30%) with the highest scores are selected as the "drift set" for training.
• Parametric Adapter Update: The user-specific LoRA adapter is fine-tuned exclusively on this selected drift set. This forces the adapter to focus on acquiring new preference patterns while avoiding overfitting to stationary, well-internalized data.
• Residual Replay Buffer Construction: To prevent forgetting sparse but critical historical nuances that the parametric update might miss, a non-parametric replay buffer is maintained. After the adapter update, all candidate samples (new drift samples + existing buffer) are re-evaluated. Samples that remain "hard" (poorly predicted) by the updated adapter are retained in the buffer, subject to a fixed budget ($N_{max}$). This ensures the buffer stores interactions that the parametric model cannot yet internalize.

2. Inference: Gated Retrieval-Interpolated Generation

During inference, SPRING dynamically balances internalized parametric knowledge with external historical evidence:

• Relevance Gating: For a new query, the system retrieves top-$k$ interactions from the replay buffer. A strict relevance gating mechanism (based on a similarity threshold $\sigma$) filters out irrelevant contexts. If no retrieved documents pass the threshold, the system reverts to purely parametric generation.
• Logit Interpolation: For queries with valid retrieved context, the model generates responses via two parallel paths using the same user-adapted model:
  1. Parametric Path: Generates logits based solely on the query and the adapter's internal weights.
  2. Retrieval-Augmented Path: Generates logits based on the query augmented with the retrieved context.
  • The final output distribution is a weighted interpolation of these two paths ($p_{fin} = \lambda \cdot p_{ret} + (1-\lambda) \cdot p_{par}$), allowing the model to balance learned preferences with specific historical evidence.

Key Contributions

1. Drift-Driven Selective Adaptation: A novel mechanism to distinguish genuine preference shifts from transient noise using a likelihood-based scoring function, enabling efficient and targeted adapter updates.
2. Residual Replay Buffer: A strategy to preserve hard-to-learn historical nuances that parametric updates fail to internalize, mitigating catastrophic forgetting without uniform data retention.
3. Gated Retrieval-Interpolated Generation: A hybrid inference strategy that dynamically fuses parametric and non-parametric knowledge, filtering irrelevant retrieval to prevent noise injection while leveraging history for specific nuances.
4. Empirical Validation: Comprehensive evaluation on the LongLaMP benchmark (Abstract Generation and Review Writing) demonstrating significant improvements over state-of-the-art baselines.

Experimental Results

The authors evaluated SPRING on the LongLaMP benchmark using the Gemma-3-4B-IT backbone, comparing it against non-personalized baselines, standard personalization methods (RAG, PAG, OPPU), and continual learning baselines (CAMA, SAR, O-LoRA, EWC-LoRA).

• Performance: SPRING consistently outperformed all baselines.
  • In Abstract Generation, it achieved a 15.85% relative improvement in ROUGE-L over the strongest CL baseline (CAMA).
  • In Review Writing, it improved ROUGE-L by 13.60% over CAMA and showed an 18.12% relative improvement over standard RAG.
• Ablation Studies:
  • Drift-driven training contributed a 12.35% gain over single-source methods.
  • Interpolating parametric and non-parametric knowledge yielded improvements up to 22.58%.
  • Relevance gating was crucial for Review Writing, filtering out off-topic retrieval that degraded performance in standard RAG setups.
• Robustness: The method demonstrated robustness across varying interaction history lengths (Small vs. Large groups) and different LLM backbones (Qwen2.5-1.5B, Llama3.1-8B). It also showed resilience to abrupt preference shifts and did not exhibit significant memorization risks compared to adapter-only training.

Significance and Claims

The paper claims that SPRING provides a robust framework for real-world continual personalization by effectively reconciling the trade-off between plasticity (adapting to new preferences) and stability (preserving historical context).

• Distinguishing Signal from Noise: The work emphasizes that standard CL approaches fail in personalization because they treat all data equally. SPRING's ability to distinguish genuine preference drift from transient noise is presented as a critical advancement.
• Hybrid Efficiency: By using a semi-parametric approach, SPRING avoids the computational cost of full retraining and the retrieval inefficiencies of pure non-parametric methods. It selectively updates parameters only when necessary and uses retrieval only when relevant.
• Scalability: The authors note that the framework is designed to be scalable, with lightweight per-user storage requirements (only the adapter snapshot and a bounded buffer) and the potential for adaptive inference strategies to manage latency.

The authors conclude that SPRING validates the necessity of moving beyond static personalization and offers a practical path for deploying long-term, personalized LLMs that can adapt to evolving user traits over time.