Modality-Decoupled Online Recursive Editing

Technical Summary: Modality-Decoupled Online Recursive Editing (M-ORE)

Problem Statement
Online model editing for Multimodal Large Language Models (MLLMs) faces two critical failure modes when adapting existing text-only editing methods: cross-modal conflict and inter-edit interference.

1. Cross-modal Conflict: In MLLMs, visual representations often possess significantly higher energy (variance) than text activations. When editors share global statistics across modalities, these high-energy visual features dominate the estimation of covariance and update directions. This biases the editing process toward the visual subspace, skewing updates and degrading the preservation of textual knowledge.
2. Inter-edit Interference: In sequential online editing, successive updates compete for representational capacity within a shared parameter space. Existing methods often cause edits to collapse into a "shared edit core," where new updates repeatedly reuse the same coordinates. This leads to long-horizon entanglement, where new edits overwrite or drift away from previously injected knowledge, resulting in catastrophic forgetting and degraded reliability over time.

Furthermore, practical systems require constant-overhead updates. Many existing parameter-preserving methods incur storage costs that grow linearly with the number of edits, while parameter-modifying methods often fail to maintain stability under long edit streams.

Methodology: M-ORE
The authors propose M-ORE (Modality-Decoupled Online Recursive Editing), a framework derived from a unified proximal-projection formulation. It addresses the identified failure modes through two core mechanisms:

1. Modality-Decoupled Locality Statistics:
   M-ORE avoids the cross-modal conflict by maintaining separate locality statistics for different edited modules. Specifically, it computes and stores distinct second-order statistics (preconditioners) for the text stack (LLM layers) and the visual projector. This ensures that high-energy visual features do not contaminate the update geometry of text layers, preventing visual dominance in the editing process.

2. Fixed Orthogonal Low-Rank Subspace with Sherman-Morrison Recursion:
   To mitigate inter-edit interference and ensure constant overhead, M-ORE restricts all edits to a fixed orthogonal low-rank write space.

   • Fixed Coordinates: The write basis $A^{(l)}$ is orthogonally initialized and frozen, providing a stable coordinate system that does not drift over time.
   • Recursive Updates: The editable parameters are the low-rank matrices $B^{(l)}$. The update rule is derived as a closed-form solution to a proximal projection problem, regularized by a locality statistic $S^{(l)}_{t-1}$ accumulated in the steady subspace.
   • De-entanglement: The update $\Delta B^{(l)}_t = -\eta G^{(l)}_t P^{(l)}_{t-1}$ utilizes a preconditioner $P^{(l)}_{t-1} = (I + S^{(l)}_{t-1})^{-1}$. As $S^{(l)}_{t-1}$ accumulates outer products of past features, $P^{(l)}_{t-1}$ adaptively suppresses updates on coordinates that have been heavily used by previous edits. This geometrically de-entangles sequential updates, preventing collapse into a single edit core.
   • Efficiency: The preconditioner is updated efficiently using the Sherman-Morrison lemma, allowing for $O(r^2)$ updates per layer (where $r \ll d$) without direct matrix inversion. This yields constant per-edit time and memory complexity, $O(1)$, independent of the edit stream length.

Key Contributions

• Diagnosis: The paper identifies and analyzes two specific failure modes in applying LLM editors to MLLMs: modality scale mismatch causing cross-modal conflict, and entangled sequential updates causing inter-edit interference.
• Algorithm: It introduces M-ORE, a modality-decoupled editor that utilizes a fixed orthogonal low-rank write space and a Sherman-Morrison grounded recursion to achieve constant-overhead online editing.
• Empirical Validation: Extensive experiments on BLIP2-OPT and LLaVA-v1.5 backbones demonstrate that M-ORE consistently outperforms strong baselines (including ROME, MEND, AlphaEdit, and LiveEdit) in reliability, generality, and locality, particularly under long edit horizons (up to 100 edits).

Results

• Performance: M-ORE achieves superior stability-plasticity trade-offs. On the E-IC task with 100 edits, M-ORE improved average scores over the best baseline (LiveEdit) by +1.89 points on BLIP2-OPT and +7.07 points on LLaVA-v1.5.
• Robustness: Unlike parameter-modifying baselines (e.g., MEND) which collapse to near-zero reliability after long edit streams, M-ORE maintains high reliability and generality.
• Locality: M-ORE effectively preserves general capabilities (tested on MME benchmark tasks like OCR, counting, and reasoning) and minimizes distributional shifts in hidden states for unrelated inputs.
• Efficiency: The method maintains constant latency and memory usage regardless of the number of edits, unlike parameter-preserving methods that suffer from growing storage or retrieval costs.

Significance
The paper claims that M-ORE provides a principled framework for lifelong MLLM adaptation under strict compute and memory budgets. By decoupling modality-specific statistics and enforcing geometric de-entanglement in a fixed subspace, it resolves the dual challenges of modality heterogeneity and sequential interference. The authors position this as a step toward making MLLMs more reliable and maintainable in real-world scenarios where knowledge evolves, allowing for the correction of factual errors without expensive retraining. The work emphasizes the importance of controlled access and auditing for such editing capabilities to prevent misuse.