Technical Summary: ReLoRA – Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed as continuously evolving services where base models are periodically updated to incorporate new knowledge and improve alignment. In service-oriented ecosystems, downstream applications are typically customized using Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA), which freezes the base model and trains lightweight adapters.

However, when the base model evolves (e.g., from version $\Theta$ to $\Theta' = \Theta + \Delta\Theta$), previously deployed LoRA adapters ($\Delta\Theta_i$) optimized for the old backbone often become obsolete or suboptimal. Service providers face a critical maintenance dilemma:

1. Retraining from scratch: While effective, retraining thousands of LoRA adapters for every base-model update is computationally prohibitive and causes significant delays in service rollout (time-to-readiness).
2. Naive transfer: Directly applying the old adapter to the new backbone (as in PortLLM) requires no computation but often leads to degraded service quality due to incompatibility between the old task adaptation and the evolved backbone's parameter shifts.

The core challenge is to rapidly restore service-ready task adapters after base-model evolution while preserving or improving task performance, without incurring the full cost of retraining from scratch.

2. Methodology: ReLoRA Framework

The authors propose ReLoRA, a knowledge-reusing re-adaptation framework that accelerates service rollout by jointly leveraging two sources of existing knowledge: the task-specific knowledge encoded in the previously deployed adapter and the evolution knowledge reflected by the base-model parameter delta. The framework consists of two key optimization steps:

2.1 Adaptive LoRA Initialization

Instead of naively summing the old adapter and the base-model delta, ReLoRA constructs a compatibility-aware starting point ($\Theta_{init}$) by adaptively fusing these components.

• Formulation: The initialization is defined as $\Theta_{init}(\alpha, \beta) = \alpha \cdot \Pi_r(\Delta\Theta) + \beta \cdot \Delta\Theta_i$, where $\Pi_r(\cdot)$ maps the base-model evolution delta into the compatible LoRA parameter space, and $\alpha, \beta$ are scalar coefficients.
• Optimization: Since the optimal fusion coefficients are task-dependent and difficult to predict, ReLoRA employs Bayesian Optimization (BayesOpt) to search for $(\alpha, \beta)$.
  • A Gaussian Process (GP) serves as a surrogate model for the objective function (negative validation loss).
  • An Expected Improvement (EI) acquisition function guides the search to minimize the number of expensive pre-rollout validation probes.
  • This approach identifies a superior initialization that balances the contribution of the base evolution and the original task adapter, avoiding the pitfalls of static fusion rules.

2.2 Fine-tuning with Scheduled Regularization

Once the optimized initialization is obtained, ReLoRA performs re-adaptation using a two-stage regularization strategy to balance rapid recovery with final performance refinement.

• Stage 1: Guided Rapid Adaptation: The adapter is strongly anchored to the initialized state using a high L2 regularization strength ($\lambda_1$). This constrains the optimization trajectory, preventing the model from drifting too far from the promising initialization and rapidly moving it into a service-ready quality region.
• Stage 2: Refinement and Exploration: The regularization strength is relaxed ($\lambda_2 \ll \lambda_1$), and a cosine annealing learning rate schedule is applied. This allows the adapter to explore the parameter space more freely, driven by the task loss, to refine task-specific behaviors and reach a higher local optimum.

3. Key Contributions

The paper outlines the following contributions:

1. Problem Formulation: The authors formalize adapter re-adaptation for evolving LLMs as an adapter backward-compatibility and service rollout problem, prioritizing the rapid restoration of service-ready adapters over simple accuracy maximization.
2. ReLoRA Framework: A knowledge-reusing framework that reduces maintenance overhead by leveraging both the previously deployed task adapter and the base-model evolution delta.
3. Adaptive Initialization Strategy: An approach using Bayesian optimization to fuse knowledge from existing adapters and base-model deltas, constructing a compatibility-aware starting point that outperforms naive transfer and static fusion.
4. Scheduled Regularization: A two-stage fine-tuning strategy that first anchors the adapter for rapid service-quality recovery and then relaxes constraints for task-specific refinement.
5. Empirical Validation: Extensive experiments across six downstream service tasks, three model families (LLaMA2, LLaMA3.1, Mistral), and three update sources demonstrate the framework's effectiveness.

4. Experimental Results

The authors evaluated ReLoRA against baselines including LoRA-Scratch (retraining from scratch), PortLLM+FT (naive transfer followed by fine-tuning), and ORAL (generative weight synthesis).

• Time-to-Readiness: ReLoRA significantly reduces the time required to reach a predefined service-quality threshold. Compared to LoRA-Scratch, it achieves up to an 8.9× speedup (e.g., reducing time on MNLI from ~255k seconds to ~28k seconds).
• Service Quality: ReLoRA improves final task accuracy compared to baselines. It outperforms LoRA-Scratch by up to 4.6 percentage points (e.g., 83.9% vs. 81.4% on MMLU with OpenOrca updates) and consistently beats PortLLM+FT.
• End-to-End Efficiency: Even when accounting for the Bayesian optimization search overhead, ReLoRA reduces the total end-to-end service rollout time by 56.9% compared to LoRA-Scratch across six tasks.
• Robustness: The method demonstrates consistent performance across different LLM backbones (LLaMA2-7B, Mistral-7B, LLaMA3.1-8B) and various update sources (OpenOrca, AlpacaGPT4, OpenPlatypus).
• Ablation Studies: Removing either the adaptive initialization or the scheduled regularization results in lower service quality and slower convergence, confirming the necessity of both components.

5. Significance and Claims

The paper claims that ReLoRA addresses a critical bottleneck in the lifecycle management of LLM-as-a-Service ecosystems. By treating adapter re-adaptation as a knowledge-reusing problem rather than a fresh training task, ReLoRA enables service providers to:

• Mitigate Operational Burden: Drastically reduce the GPU-hours and operational costs associated with maintaining thousands of downstream services during base-model updates.
• Accelerate Rollout: Enable near-instantaneous service updates, ensuring that downstream applications remain available and performant while the underlying models evolve.
• Preserve Compatibility: Provide a mechanism to maintain backward compatibility, ensuring that investments in task-specific adapters are not invalidated by upstream model updates.

The authors position ReLoRA as a practical solution for the "service maintenance problem" inherent in evolving LLM ecosystems, bridging the gap between model evolution and application stability without requiring full retraining.