🤖 machine learning

Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning

This paper proposes Trajectory Regularized Merging (TRM), a novel framework that addresses storage limitations and optimization stagnation in continual learning by reformulating model merging as an optimization process within an augmented trajectory subspace to achieve state-of-the-art performance.

Original authors: Xi Wang, Cheng Deng

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xi Wang, Cheng Deng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Forgetful Chef" and the "Heavy Fridge"

Imagine you are a chef trying to learn a new recipe every day.

The Goal: You want to be a master of all recipes (Continual Learning).
The Problem: If you focus too hard on today's new dish, you might forget how to cook yesterday's dish. This is called "catastrophic forgetting."
The Old Solution: Usually, chefs keep a massive fridge full of every ingredient and recipe they've ever used so they can look back and remember.
The Constraint: In this paper, the authors say, "No big fridges allowed!" You can only keep the recipe you just learned and the one you learned right before that. You cannot store old data or old models. This is a strict privacy and storage rule.

The Current Flaw: The "Bad Handoff"

Existing methods try to combine your "Old Recipe" and your "New Recipe" into one "Master Recipe" (Model Merging). However, the authors found that current methods do this poorly when they can't look at the history.

They identified three specific ways this "handoff" goes wrong:

The "Average is Bland" Problem (Suboptimal Local Convergence):
Imagine you have a perfect lasagna recipe and a perfect sushi recipe. If you just mix them halfway, you get a weird, mediocre dish that tastes like neither. Current methods try to find a "global average," but this ruins the specific details needed for each task. The result is a model that is okay at everything but great at nothing.
The "Broken Structure" Problem (Disruption of Semantic Representation):
Think of a building. The foundation (lower layers) holds the weight, but the specific rooms (higher layers) are where the unique functions happen. When current methods mash two models together, they accidentally break the internal wiring. It's like trying to merge two houses by smashing their walls together; the rooms end up in the wrong places, and the house stops making sense.
The "Stuck in Mud" Problem (Loss of Optimization Plasticity):
Imagine you are driving a car. If you park in a deep, flat valley, it's hard to get the car moving again because there's no slope to roll down. Current merging methods often park the model in a "flat valley" of math. When the next new task arrives, the model is so "stiff" and stuck that it can't learn anything new. It has lost its ability to adapt.

The Solution: TRM (Trajectory Regularized Merging)

The authors propose a new method called TRM. Instead of just blindly mixing the two recipes, they treat the merging process like a guided search for the perfect spot on a map.

Here is how TRM works, using three "rules" to find the best spot:

Rule 1: Stay True to the New Task (Task Alignment)
- Analogy: Before you leave the kitchen, make sure the new dish actually tastes good.
- What it does: It forces the merged model to perform well on the current task immediately, ensuring we don't lose the specific details of the new recipe.
Rule 2: Keep the House Intact (Prediction Consistency)
- Analogy: Make sure the rooms in the new house still line up with the rooms in the old house. Don't let the kitchen end up in the bathroom.
- What it does: It checks that the internal "wiring" of the model hasn't been scrambled. It ensures the model's internal understanding of the world remains stable and logical.
Rule 3: Keep the Engine Running (Gradient Responsiveness)
- Analogy: Don't park the car in a flat valley. Park it on a slight hill so the engine can rev up and move forward easily.
- What it does: It ensures the model isn't "stuck." It keeps the mathematical "slope" steep enough so that when the next new task arrives, the model can learn quickly and easily.

The Secret Ingredient: The "Magic Nudge"

Since the authors aren't allowed to look at old data, they are working with very limited information (just a straight line between the old and new model). To fix this, they introduce a "Perturbation Vector" (a controlled random nudge).

Analogy: Imagine you are walking in a straight line, but you feel like you're hitting a wall. You take a small, calculated step to the side (not a random stumble, but a deliberate step) to see if there's a better path.
Why: This gives the model a little bit of "wiggle room" to find a better spot without needing to remember the entire history.

The Results

The authors tested this "Chef" (the model) on several difficult cooking challenges (datasets like CIFAR100, ImageNet-R, and Stanford Cars).

The Outcome: TRM consistently beat the other methods.
The Score: On the hardest test (ImageNet-R with 20 tasks), TRM achieved 82.7% accuracy, while the next best method only got 79.3%.
The Efficiency: It did this without needing a giant fridge (no stored data) and didn't take much longer to cook (train) than the other methods.

Summary

The paper argues that simply averaging two AI models together destroys their ability to learn new things later. By using three specific rules to check the taste, the structure, and the "engine" of the model, and by taking a small, calculated step to the side, the authors created a way to merge models that keeps them sharp, stable, and ready for whatever comes next—all without hoarding old data.

Technical Summary: Revitalizing the Beginning: Avoiding Storage Dependency for Model Merging in Continual Learning

1. Problem Statement

Continual Learning (CL) aims to enable models to acquire new knowledge sequentially without catastrophic forgetting. While Model Merging has emerged as a training-free paradigm for integrating specialized expertise into a unified multi-task model, existing methods face a critical conflict with the strict memory constraints of CL. Most current merging techniques (e.g., Model Stock, TIES, MagMax) require access to all previous task vectors and the initial model during the merging phase. This is functionally equivalent to storing the entire ensemble of prior experts, violating the CL principle of prohibiting the retention of historical data or models.

The authors demonstrate that under strict CL constraints—where only the current expert and the immediately merged model are available—existing methods suffer from catastrophic performance degradation (up to 6%). Through diagnostic experiments, the paper identifies three root causes for this failure:

Suboptimal Local Convergence: Existing methods prioritize global alignment, causing merged models to reside in high-loss regimes far from the optimal basins of individual tasks, leading to the accumulation of task-specific errors.
Disruption of Structural Semantic Representation: Linear merging shatters the delicate functional dependencies between layers, causing significant semantic drift, particularly in deeper layers where task-specific semantics are encoded.
Loss of Optimization Plasticity: Merged models often exhibit "directional stiffening," where gradient angular deviation is minimized. This indicates the model is trapped in pathologically flat regions of the loss landscape, resulting in vanishing gradients and an inability to adapt to subsequent tasks.

2. Methodology: Trajectory Regularized Merging (TRM)

To address these limitations without storing historical data, the authors propose Trajectory Regularized Merging (TRM). This framework reformulates the merging phase as a guided optimization problem within an augmented trajectory subspace.

Augmented Subspace Construction:
Instead of searching for a solution solely within the 1D linear span of the current task vector ( $\tau_t$ ) and the previous task vector ( $\tau_{t-1}$ ), TRM introduces a perturbation vector $P$ orthogonal to the span of these vectors. This expansion compensates for the information lost due to storage constraints, allowing the search to explore more elastic regions of the non-convex parameter space. The merged task vector is formulated as:
$\tau_{mrg}^t(\alpha, \beta) = \alpha \cdot \tau_t + (1-\alpha) \cdot \tau_{t-1} + \beta \cdot P$
where $\alpha$ and $\beta$ are interpolation and expansion coefficients, respectively.

Multi-Objective Supervisory Signal:
To identify the optimal coefficients $\{\alpha, \beta\}$ , TRM optimizes a unified objective function composed of three synergistic constraints:

Task Alignment ( $L_{align}$ ): Minimizes the empirical risk (cross-entropy loss) on the current task. This ensures the merged model remains anchored to the latest expertise, preventing the displacement from high-precision local basins.
Prediction Consistency ( $L_{pre}$ ): Minimizes the layer-wise L2 distance between the merged model's hidden representations and the functional centroid of the current finetuned model and the previous consolidated model. A progressive weighting scheme is applied, prioritizing higher layers where task-specific semantics reside, to prevent structural semantic drift.
Gradient Responsiveness ( $L_{res}$ ): Maximizes the squared norm of the gradient on the current task. Based on the observation that low-level feature overlaps exist between sequential tasks, maximizing the gradient norm on the current task re-activates the model's kinetic responsiveness, ensuring it is not trapped in flat regions and retains plasticity for future tasks.

The total loss is defined as $L_{total} = L_{align} + \lambda_1 L_{pre} + \lambda_2 L_{res}$ . Additionally, to prevent early convergence to local optima during the search, the initialization involves a stochastic crossover of parameters from the previous and current models.

3. Key Contributions

The paper outlines four primary contributions:

Identification of Limitations: It highlights that existing model merging methods do not fully adhere to CL principles and suffer significant performance degradation when knowledge storage is disallowed.
Theoretical Analysis: It analyzes the causes of this gap (local optimality loss, semantic drift, and plasticity loss) and reformulates merging as an optimal point search problem within an orthogonally augmented trajectory subspace.
Novel Framework: It proposes TRM, a method guided by three constraints (task alignment, prediction consistency, and gradient responsiveness) to harmonize historical stability with future plasticity without relying on stored models or data replay.
Empirical Validation: It demonstrates state-of-the-art performance across multiple benchmarks under strict CL constraints.

4. Experimental Results

The authors evaluated TRM on three datasets: CIFAR100, ImageNet-R, and Stanford Cars, using class-incremental learning (CIL) with varying numbers of tasks (5, 10, and 20). The backbone used was CLIP ViT-B/16.

Performance: TRM achieved state-of-the-art results across most settings. On ImageNet-R, it outperformed previous SOTA methods by margins of 1.4% to 3.4% across 5, 10, and 20 tasks. On CIFAR100, it improved upon the previous SOTA by 3.1% (5 tasks) and 1.5% (10 tasks).
Ablation Studies: Removing any of the three constraints led to performance drops, confirming that stability (alignment/consistency) and plasticity (responsiveness) are mutually reinforcing.
Scalability: Experiments on the large-scale ImageNet1K (100 tasks) and different backbones (ViT-L/14, LAION-pretrained ViT-B/16) confirmed the method's robustness and scalability.
Efficiency: While TRM introduces a slight increase in training time compared to some baselines (e.g., 33 minutes vs. 30 minutes on ImageNet-R), it achieves significant performance gains with minimal overhead.
Optimization Stability: Analysis of the Hessian spectral norm showed that TRM maintains a moderate and stable loss landscape curvature, avoiding the pathological sharpness associated with poor generalization.

5. Significance and Claims

The paper claims that TRM successfully bridges the gap between the theoretical benefits of model merging and the practical constraints of continual learning. By avoiding the need to store historical task vectors or replay data, TRM offers a scalable and privacy-preserving solution for integrating specialized expertise. The authors assert that their method effectively mitigates catastrophic forgetting in continuous data streams by revitalizing the "beginning" of the next training phase—ensuring the merged model starts from a state that is both stable (retaining prior knowledge) and plastic (ready for new optimization). The work suggests that reformulating merging as a constrained optimization within an augmented subspace is a viable path forward for CL, moving beyond simple linear interpolation heuristics.