🤖 machine learning

Beyond Hard Writes and Rigid Preservation: Soft Recursive Least-Squares for Lifelong LLM Editing

The paper introduces RLSEdit, a recursive least-squares framework for lifelong LLM editing that formulates updates as an online quadratic optimization with soft constraints to effectively balance the plasticity-stability dilemma, enabling stable sequential editing of up to 10,000 facts while preserving general capabilities.

Original authors: Xinyu Wang, Sicheng Lyu, Yu Gu, Jerry Huang, Peng Lu, Yufei Cui, Xiao-Wen Chang

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xinyu Wang, Sicheng Lyu, Yu Gu, Jerry Huang, Peng Lu, Yufei Cui, Xiao-Wen Chang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, well-read librarian (a Large Language Model) who has memorized a massive library of facts. One day, you realize a few books have outdated information. You need to update them without throwing the whole library out and rebuilding it from scratch—that would be too expensive and slow.

This is the problem of Model Editing: fixing specific facts in an AI without retraining the whole thing.

The Problem: The "Overwrite" vs. "Stiffness" Dilemma

The paper argues that current methods for updating these librarians are like using a sledgehammer or a rigid cage, and neither works well when you have to make thousands of updates over time.

The "Hard Write" (The Sledgehammer): Some methods just smash the old fact and write the new one.
- The Analogy: Imagine you are writing in a notebook. Every time you get a new fact, you scribble over the old one. If you do this 10,000 times, your notebook becomes a messy scribble. The new facts might accidentally erase old, unrelated facts you didn't mean to touch. The librarian starts forgetting things they knew perfectly well yesterday.
The "Rigid Preservation" (The Stiff Cage): Other methods try to be very careful, only allowing changes in very specific, pre-approved directions.
- The Analogy: Imagine the librarian is locked in a cage that only opens for specific updates. While this protects some things, the cage is too rigid. It might stop the librarian from learning new things naturally, or it might accidentally let "bad" changes slip through the gaps because the cage wasn't designed for every possible new fact.

The Solution: RLSEdit (The "Soft, Recursive" Approach)

The authors propose a new method called RLSEdit. Instead of a sledgehammer or a cage, they treat the editing process like balancing a scale or tuning a radio.

Here is how it works, using simple metaphors:

1. The "Soft" Constraint (The Elastic Band)

Instead of forcing the librarian to change exactly this way or not at all, RLSEdit uses "soft" rules.

The Analogy: Imagine the librarian is tied to their original self by a strong, stretchy elastic band. When you ask them to learn a new fact, they can stretch to reach it, but the band gently pulls them back so they don't wander too far from who they were originally. This prevents them from forgetting their core personality or general knowledge.

2. The "Anchor" (The North Star)

The method also uses a specific "Anchor" (a set of facts the librarian must definitely keep correct).

The Analogy: Imagine the librarian has a compass (the Anchor) that always points to "North" (the original, correct facts). Even as they stretch to learn new things, the compass ensures they don't get lost or spin in circles.

3. The "Recursive" Magic (The Magic Calculator)

The biggest innovation is how they do the math. Usually, if you update a model 10,000 times, the computer has to look back at all 10,000 previous updates to decide what to do next. This gets slower and slower, like a librarian trying to remember every single conversation they've ever had before answering a new question.

The Analogy: RLSEdit uses a "magic calculator" (based on something called the Woodbury identity). Instead of re-reading the whole history book every time, it only looks at the current update and a tiny summary of the past.
The Result: Whether you are making the 10th update or the 10,000th, it takes the same amount of time. It's like a librarian who can instantly update their knowledge base without getting slower as the library grows.

What Did They Find?

The researchers tested this on two popular AI models (Llama-3 and Qwen2.5) and made them learn 10,000 new facts in a row.

Success Rate: RLSEdit was much better at actually remembering the new facts it was taught compared to other methods.
No "Catastrophic Forgetting": While other methods started forgetting old facts or getting confused about basic logic (like math or coding) after a few thousand updates, RLSEdit stayed stable.
General Smarts: The librarian didn't lose their ability to write poetry, solve math problems, or understand code, even after 10,000 changes. Other methods made the librarian "dumber" in these general areas over time.

In a Nutshell

Think of RLSEdit as a smart, flexible memory system. It allows an AI to learn new things continuously without:

Erasing its old memories (like a sledgehammer would).
Getting stuck or rigid (like a cage would).
Getting slower and slower as it learns more (like a human trying to remember every detail of their life).

It keeps the AI sharp, stable, and ready for the long haul of continuous learning.

Technical Summary: Soft Recursive Least-Squares for Lifelong LLM Editing (RLSEdit)

1. Problem Statement

Large Language Models (LLMs) deployed in real-world scenarios inevitably contain outdated or incorrect knowledge. While model editing allows for updating specific facts without full retraining, practical deployment involves long sequential streams of edits. This creates a "plasticity–stability dilemma":

Retroactive Edit Forgetting: Repeated "hard writes" (e.g., locate-then-edit methods) accumulate interference, causing future edits to overwrite or degrade knowledge from earlier edits.
General-Ability Degradation: Rigid preservation constraints (e.g., null-space editors) may protect explicitly constrained directions but fail to prevent the degradation of unconstrained general capabilities (reasoning, coding) or the retention of previous facts as the edit stream grows.

Existing methods struggle to balance the need to memorize incoming information while preserving both earlier edits and the model's general capabilities over thousands of updates.

2. Methodology: RLSEdit

The authors propose RLSEdit, a recursive least-squares editor designed for long sequential editing. The method reframes editing as an online regularized least-squares problem with soft constraints, rather than hard constraints.

2.1 Formulation

The method targets a single linear map $W$ (e.g., an attention projection) within a transformer layer. For a sequence of edits $t=1 \dots T$ , where each edit provides key-value pairs $(K_t, V_t)$ , the objective at time $t$ is to find the optimal weight $W^*_t$ that minimizes:

$W^*_t := \arg \min_W \sum_{i=1}^t \|K_i W - V_i\|_F^2 + \lambda^2 \|W - W_0\|_F^2 + \mu^2 \|K_0 W - V_0\|_F^2$

Where:

Fitting Term: $\sum \|K_i W - V_i\|_F^2$ accumulates the error of all edits observed so far.
Parameter Drift Regularization ( $\lambda$ ): Penalizes deviation from the initial pre-trained weights $W_0$ .
Anchor Mapping Regularization ( $\mu$ ): Penalizes deviation from a designated anchor mapping $(K_0, V_0)$ , which represents a set of constraints to be preserved (e.g., background knowledge).

This formulation creates a "soft-constraint" regime that interpolates between hard writing and hard preservation.

2.2 Efficient Online Recursion

Directly solving the least-squares problem at every step is computationally expensive ( $O(d^3)$ ). RLSEdit utilizes the Woodbury matrix identity to derive an efficient recursive update.

State Maintenance: The algorithm maintains the inverse covariance matrix $C_t = (A_t^\top A_t)^{-1}$ and the cross-term $T_t = A_t^\top B_t$ .
Update Mechanism: Instead of full factorization, the update involves solving a small $u_t \times u_t$ system (where $u_t$ is the batch size of the current edit), which is typically much smaller than the model dimension $d_k$ .
Complexity: The per-edit cost is $O(d_k^2 u_t + d_k d_v u_t + d_k u_t^2 + u_t^3)$ , which simplifies to $O(d_k^2 u_t)$ when $u_t \ll d_k$ . Crucially, this cost is independent of the history length $t$ , unlike methods requiring dense matrix operations over accumulated history.

2.3 Theoretical Analysis

The paper provides deviation bounds showing that $\lambda$ controls global parameter deviation from $W_0$ , while $\mu$ controls the deviation of the anchor mapping. Furthermore, an asymptotic characterization demonstrates that under long edit streams with stable empirical second moments, the estimator converges to a unique minimizer of a regularized limiting risk.

3. Key Contributions

Soft-Constraint Formulation: The authors formulate lifelong editing as online regularized least squares with explicit parameter-deviation and anchor-deviation controls, offering a soft alternative to hard writing and hard preservation.
History-Independent Complexity: They derive a Woodbury-based online update where the per-edit computational cost does not scale with the number of past edits, enabling stable scaling to long edit streams.
Theoretical Guarantees: The paper provides deviation bounds and an asymptotic characterization of the adherence–preservation trade-off in the many-edits regime.
Empirical Validation: Extensive evaluation on Llama-3 and Qwen2.5 models shows the method scales stably to 10,000 edits.

4. Experimental Results

Experiments were conducted on Llama-3-8B and Qwen2.5-7B with 10,000 sequential edits (batch size 100).

Editing Success (CounterFact & ZsRE):
- On Llama-3, RLSEdit achieved the best scores across all five metrics (Efficacy, Generalization, Specificity, Fluency, Consistency), significantly outperforming baselines like AlphaEdit, ROME, and MEMIT. For instance, Efficacy reached 89.94% compared to 74.76% for the next best (FT).
- On Qwen2.5, RLSEdit achieved the highest Efficacy (94.45%) and Fluency, remaining competitive with AlphaEdit on other metrics while substantially outperforming ROME and MEMIT.
- On ZsRE, RLSEdit achieved 81.20% Efficacy, outperforming AlphaEdit (70.04%) and others.
General Capability Preservation:
- Evaluated on GLUE, MMLU, GSM8K, HumanEval, and MBPP.
- RLSEdit consistently preserved general capabilities (reasoning, coding, language understanding) throughout the 10k edit trajectory.
- In contrast, baselines like ROME, MEMIT, and FT showed significant degradation. AlphaEdit performed well initially but degraded noticeably after large numbers of edits.
- Case Study: On HumanEval Task 0, AlphaEdit and MEMIT degraded into boundary errors, semantic bugs, or garbled/empty outputs by 8k–10k edits, whereas RLSEdit maintained a correct implementation across all checkpoints.
Early Edit Retention:
- When re-evaluating the first 500, 1k, 2k, and 4k edits after 10k total edits, RLSEdit consistently achieved the highest retention scores, demonstrating superior resistance to retroactive forgetting.
Efficiency:
- RLSEdit was 1.37×–1.79× faster than AlphaEdit during updates across different batch sizes and model backbones, validating the theoretical complexity analysis.

5. Significance and Claims

The paper claims that RLSEdit provides a practical approach for continuous model editing by resolving the plasticity-stability dilemma inherent in long sequential streams. By treating editing as a soft-constrained optimization problem, it avoids the interference accumulation of hard-write methods and the rigidity of null-space methods.

The authors emphasize that their method allows for:

Scalability: Stable performance up to 10,000 edits without increasing computational cost per edit.
Holistic Stability: Simultaneous improvement in edit success, retention of early edits, and preservation of general model capabilities.
Theoretical Grounding: A clear characterization of the trade-off between learning new facts and preserving existing knowledge via the regularization parameters $\lambda$ and $\mu$ .

The paper concludes modestly, noting that current experiments focus on 7B–8B backbones and that evaluating larger models and more failure cases remains an important direction for future work.