Agent-Dice: Disentangling Knowledge Updates via Geometric Consensus for Agent Continual Learning

1. Problem Statement

The paper addresses the Stability-Plasticity Dilemma in Continual Learning (CL) for Large Language Model (LLM)-based agents.

• Context: LLM agents are increasingly deployed in dynamic environments (e.g., GUIs, tool usage) requiring them to learn new tasks continuously without retraining from scratch.
• The Challenge: Existing CL methods struggle to balance stability (retaining old knowledge) and plasticity (learning new tasks).
  • Overemphasizing stability leads to catastrophic forgetting of new tasks.
  • Excessive plasticity causes interference, where new task updates overwrite or corrupt shared, common knowledge from previous tasks.
• Root Cause: The authors argue that current methods fail to explicitly distinguish between common knowledge (shared across tasks) and conflicting knowledge (task-specific interference). This lack of disentanglement leads to information loss or destructive interference during parameter updates.

2. Methodology: Agent-Dice

The authors propose Agent-Dice, a parameter fusion framework based on Directional Consensus Evaluation. Instead of sequential fine-tuning, Agent-Dice integrates task vectors from diverse tasks onto the original pre-trained model via a two-stage process.

Theoretical Foundation

The method is grounded in three theoretical lenses:

1. Linear Mode Connectivity: Assumes the pre-trained model lies in a linear basin, allowing task vectors to be combined linearly.
2. Variance Reduction via Consensus: Models task vectors as noisy estimators of a shared latent descent direction. Filtering out outliers (conflicting gradients) reduces update error exponentially.
3. Maximum Entropy Weighting: Assigns weights based on parameter saliency (curvature/magnitude of updates) to maximize entropy, ensuring the most confident updates are prioritized.

The Two-Stage Pipeline

Agent-Dice operates element-wise on the model parameters ($\theta$) to compute a fused update: $\theta_{fused} = \theta_{pre} + \sum W_k \odot \tau_k$.

1. Stage 1: Geometric Consensus Filtering (Stability)

   • Goal: Prune conflicting gradients to preserve stability.
   • Mechanism: For each parameter, the algorithm counts the "votes" (signs) of the update directions across all $K$ tasks.
   • Logic: If the majority of tasks agree on the sign of an update (e.g., positive), updates from minority tasks (outliers) are masked to zero. This ensures the fused model only moves in directions supported by a consensus of tasks, effectively filtering out task-specific noise that causes forgetting.

2. Stage 2: Curvature-Based Importance Weighting (Plasticity)

   • Goal: Amplify shared semantics and prioritize confident updates.
   • Mechanism: Within the set of "voting" (consensus) tasks, weights are assigned based on the magnitude of the update ($|\tau_{k,i}|$).
   • Logic: Large update magnitudes indicate high confidence or traversal through steep gradients (high curvature). A Softmax function (Boltzmann distribution) normalizes these magnitudes. This ensures that the final update step size is governed by the agents exhibiting the strongest local feature response, enhancing plasticity for shared knowledge.

3. Key Contributions

1. Problem Identification: The paper identifies that the stability-plasticity dilemma in agent CL stems from the failure to disentangle common vs. conflicting knowledge.
2. Novel Framework: Introduction of Agent-Dice, a parameter fusion method combining geometric consensus filtering and curvature-based weighting.
3. Theoretical Analysis: Provides rigorous proofs (Theorems 1-3) demonstrating that the fusion scheme minimizes joint loss, reduces variance via consensus, and optimally weights updates based on saliency.
4. Empirical Validation: Extensive experiments across two distinct domains (GUI agents and Tool-use agents) showing superior performance with minimal computational overhead.

4. Experimental Results

The authors evaluated Agent-Dice on GUI Agents (using datasets: AITZ, AndroidControl, GUI-Odyssey) and Tool-Use Agents (using ToolACE dataset) with various base models (OS-Atlas-Pro-7B, Qwen3-VL-8B, Qwen3-8B, Llama-3.1-8B).

• Performance: Agent-Dice achieved the highest Average Z-score (AvgZ) in both domains, outperforming traditional continual learning baselines (e.g., sequential fine-tuning, rehearsal-free methods).
  • In GUI tasks, it significantly mitigated catastrophic forgetting. For instance, when learning from multiple datasets, Agent-Dice maintained high performance on earlier tasks (AITZ, AndroidControl) while learning new ones (GUI-Odyssey), whereas standard methods suffered severe drops.
  • In Tool-Use tasks, it achieved top metrics across nearly all subsets for both tool-capable and non-tool-capable models.
• Efficiency:
  • Overhead: The method is lightweight. On average, it requires only ~77 seconds on GPU and ~10 minutes on CPU to compute the fusion, compared to hours required for full retraining.
  • Parameters: It involves minimal parameter updates, acting as a post-training fusion step rather than a heavy training loop.
• Analysis:
  • Ablation Studies: Removing either stage (filtering or weighting) caused significant performance drops, proving both are essential.
  • Model Similarity: KL divergence analysis showed that Agent-Dice models remain highly similar to the base model (minimal parameter deviation) while achieving better performance than models trained sequentially on all data, indicating effective preservation of the original manifold.

5. Significance

• Scalability: Agent-Dice offers a computationally efficient solution for deploying LLM agents in real-world scenarios where continuous adaptation is required without the cost of massive retraining or external memory buffers.
• Theoretical Insight: It provides a new perspective on the stability-plasticity dilemma, framing it as a problem of gradient consensus and saliency weighting rather than just regularization or replay.
• Generalizability: The method is model-agnostic and domain-agnostic, demonstrated effectively across both visual (GUI) and symbolic (Tool-use) agent tasks.

In conclusion, Agent-Dice successfully resolves the trade-off between stability and plasticity by mathematically disentangling shared knowledge from conflicting interference, enabling LLM agents to learn continuously with high efficiency and robustness.