💬 NLP

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

This paper demonstrates that subsequent fine-tuning generally causes substantial decay in knowledge edits, revealing that fine-tuning only the edited layers is an effective method for removing such edits while maintaining downstream performance, thereby highlighting the critical need to evaluate knowledge editing within the broader context of model adaptation pipelines.

Original authors: Yinjie Cheng, Paul Youssef, Christin Seifert, Jörg Schlötterer, Zhixue Zhao

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yinjie Cheng, Paul Youssef, Christin Seifert, Jörg Schlötterer, Zhixue Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Can You Paint Over a Correction?

Imagine you have a giant, incredibly smart encyclopedia (a Large Language Model, or LLM) that was written by a team of experts. Sometimes, the encyclopedia has mistakes. Maybe it says the capital of France is Berlin.

Knowledge Editing (KE) is like sending in a specialist to fix that specific mistake. They go in, find the page, and change "Berlin" to "Paris." They do this without rewriting the whole book. It's fast, cheap, and precise.

Fine-Tuning (FT) is like taking that encyclopedia and training it on a new subject, say, "Modern Art." You feed it thousands of new articles to help it learn about painting and sculpture. This is the standard way to make these models useful for specific jobs.

The Problem: The researchers asked a simple question: If you fix a mistake (KE) and then train the book on new topics (FT), does the fix survive? Or does the new training accidentally wash away the correction, putting "Berlin" back in?

The Main Discovery: The Correction is Fragile

The paper's main finding is that fine-tuning often erases the edits.

Think of the model's knowledge like a delicate sculpture made of clay.

Knowledge Editing is like carefully adding a tiny, specific detail to the sculpture (e.g., painting a flower on the clay).
Fine-Tuning is like shaking the whole sculpture vigorously to reshape it for a new purpose.

The study found that when you shake the sculpture (fine-tune), the tiny painted flower (the edit) often falls off or gets smudged. In many cases, the model reverts to its original, incorrect answer, or worse, it starts giving a completely new, wrong answer that wasn't there before.

The Experiment: A Massive Stress Test

The researchers didn't just guess; they ran a massive experiment.

They took 5 different models (the "encyclopedias").
They used 3 different editing tools (the "painters").
They applied 254 different combinations of editing and training.

The Results:

Most of the time, the edits died. After fine-tuning, a significant chunk of the successful corrections became failures.
The "Null Space" Vulnerability: One specific editing method (AlphaEdit) was the most fragile. It tries to hide the edit in a "shadow zone" of the model's brain that usually doesn't affect anything. But fine-tuning is so powerful it fills that shadow zone with new information, wiping out the edit completely.
Model Size Matters: Bigger models (like GPT-J) were slightly more robust, holding onto their edits better than smaller ones, but they still suffered from the problem.

The "Flip" Phenomenon

The researchers noticed three weird things happen after fine-tuning:

Stable Edits: The fix stays. (Rare).
Erased Edits: The fix disappears, and the model goes back to the original wrong answer. (Common).
Impossible Edits: The model gets confused and gives a third wrong answer that wasn't the original or the intended fix. (e.g., If you tried to change "Paris" to "London," the model might start saying "Berlin" after training).

The Surprising Twist: How to Delete Edits on Purpose

The paper also looked at how to remove bad edits (like malicious ones planted by hackers) or keep good ones. They tested a clever strategy: Don't train the whole model; just train specific parts.

Hypothesis 1: If you only train the layers where the edit was made, you should wipe out the edit.
- Result: True. This was the most effective way to remove an edit. It's like sanding down just the specific spot where the bad paint was applied.
Hypothesis 2: If you only train the layers not involved in the edit, you should protect the edit.
- Result: False. Surprisingly, training the "safe" layers actually destroyed the edits more than training the whole model. It's like shaking the base of the sculpture so hard that the detail on top falls off, even if you didn't touch the top directly.

The Takeaway: If you want to remove a bad edit, the most efficient way is to fine-tune only the specific layers where that edit lives. It's a precise surgical strike that removes the bad data while keeping the rest of the model's performance mostly intact.

Why Does This Happen? (The Brain Scan)

The researchers looked inside the model's "brain" (its activation space) to see what was happening.

Editing is like a small, localized ripple in a pond. It changes the water in one specific spot.
Fine-Tuning is like a massive wave crashing through the whole pond.

The study found that the "wave" of fine-tuning is so much stronger and moves in a different direction than the "ripple" of the edit. When the wave hits, it completely overwrites the ripple. The model's internal representation shifts so drastically that the tiny correction can't survive.

Summary for the Everyday Reader

Edits are temporary: If you fix a fact in an AI and then train it on new data, your fix will likely disappear.
It's not your fault: The model's architecture makes it very hard to keep a small change while learning a lot of new things.
You can delete edits easily: If you need to remove a bad edit, you don't need to retrain the whole AI. You can just tweak the specific parts where the bad edit lives, and it will vanish.
Safety Warning: If bad actors plant malicious lies in an AI (knowledge editing), and then companies fine-tune that AI for new tasks, those lies might survive and spread, or the AI might get confused and start hallucinating new lies.

The paper concludes that we need to stop treating "fixing facts" and "training for new tasks" as separate steps. We need to build AI systems where these two processes can coexist without one destroying the other.

Technical Summary: Can Fine-Tuning Erase Edits? On the Fragile Coexistence of Knowledge Editing and Fine-tuning

Problem Statement

Large Language Models (LLMs) are typically adapted to new domains via two distinct post-training interventions: Fine-Tuning (FT), which updates model parameters on task-specific datasets (including Parameter-Efficient Fine-Tuning or PEFT methods like LoRA and DoRA), and Knowledge Editing (KE), which updates factual knowledge with limited data and compute. While both techniques are widely adopted, they have historically been studied in isolation.

This paper addresses a critical gap in the LLM deployment pipeline: the interaction between these two processes. Specifically, the authors investigate whether knowledge edits survive subsequent fine-tuning. This question is motivated by two practical concerns:

Safety: If edits persist after fine-tuning, malicious edits (e.g., backdoors, bias injection, or misinformation) introduced into a base model could silently propagate to downstream applications.
Efficiency: If fine-tuning impairs or erases beneficial edits, operators would be forced to re-edit models after every adaptation step, rendering KE inefficient.

Methodology

The authors conducted a comprehensive empirical study covering 254 independent experimental configurations. The study systematically quantified the decay of knowledge edits after fine-tuning across various dimensions:

Models: Five LLMs were evaluated, including GPT-J-6B, GPT2-XL, Llama2-7B, Llama3.1-8B, and DeepSeek (excluded due to poor KE performance).
Knowledge Editing (KE) Methods: Three representative parameter-modifying methods were tested:
- MEMIT: A scalable multi-edit method.
- AlphaEdit: Constrains perturbations to null spaces to minimize interference.
- MEND: A hypernetwork-based approach using auxiliary editing networks.
Fine-Tuning (FT) Methods: Three approaches were compared:
- Full-parameter fine-tuning.
- LoRA (Low-Rank Adaptation).
- DoRA (Weight-Decomposed Low-Rank Adaptation).
Datasets:
- Editing: CounterFact (CF) and Zero-Shot Relation Extraction (zsRE).
- Fine-Tuning: Commonsense Reasoning dataset and HotpotQA (for knowledge-intensive tasks).
Evaluation Metrics:
- Standard KE metrics: Efficacy Success (ES), Paraphrase Success (PS), and Neighborhood Success (NS).
- New Metric: Edit Flip Ratio (EFR), defined as the proportion of edits that were successful after KE but became unsuccessful after fine-tuning. This metric specifically isolates the stability of successful edits.
Mechanistic Analysis: The authors analyzed activation spaces using layer-wise magnitude changes, directional drift (cosine similarity), and Uniform Manifold Approximation and Projection (UMAP) to understand why edits decay.

Key Contributions

Systematic Empirical Study: The first large-scale investigation into the impact of FT on KE, covering 254 configurations across models, datasets, and methods.
Evaluation Framework: Introduction of the Edit Flip Ratio (EFR) to precisely quantify the degradation of successful edits, moving beyond aggregate performance metrics.
Multi-Level Mechanistic Analysis:
- Identification of four behavioral categories post-fine-tuning: Stable Edits, Erased Edits, Emergent Edits (previously failed edits that succeed), and Impossible Edits.
- Demonstration that fine-tuning induces significantly larger and more coherent representational shifts in activation space than KE, explaining the erasure of edits.
Practical Strategies for Edit Removal: Discovery that fine-tuning only the edited layers or only the non-edited layers can effectively remove edits, often more effectively than full-layer fine-tuning, offering new strategies for neutralizing malicious edits.

Key Results

1. General Decay of Edits

Fine-tuning generally impairs knowledge edits. The study found that edits decay substantially after subsequent fine-tuning across most configurations.

Magnitude of Decay: Full-parameter fine-tuning caused the largest fraction of edits to fail (average decrease of 38.10%), followed by LoRA (28.71%) and DoRA (29.88%).
Model Sensitivity: GPT-J exhibited the highest structural robustness (smallest average decrease of 15.93%), while GPT2-XL and Llama2 showed higher variability and greater decay.
Method Sensitivity: AlphaEdit showed the greatest decay. For instance, on GPT-J with the zsRE benchmark, 25.27% of previously successful AlphaEdit edits became unsuccessful after fine-tuning. The authors attribute this to the vulnerability of the null-space storage mechanism, which is nearly orthogonal to the existing knowledge space and thus not penalized during the loss calculation of fine-tuning.

2. Impact of Datasets and Scale

Dataset: Edits on the zsRE dataset generally experienced greater performance decline compared to CounterFact (CF). This is attributed to the stricter evaluation criteria of zsRE (requiring exact generation) versus CF (ranking-based).
Scale: Larger numbers of edits generally exacerbated the decay, particularly for AlphaEdit.

3. Selective Layer Fine-Tuning

A counterintuitive finding was that fine-tuning only the non-edited layers led to greater edit decay than fine-tuning all layers.

Ordering of Edit Removal: The effectiveness of removing edits followed the order: Fine-tuning edited layers > Fine-tuning non-edited layers > Fine-tuning all layers.
Trade-off: While fine-tuning only edited layers was most effective at removing edits, it incurred a larger degradation in downstream task performance compared to fine-tuning non-edited layers.
Mechanism: This aligns with the distributed representation hypothesis; edits are stored across multiple layers. Concentrating weight updates on fewer layers (either edited or non-edited) causes larger per-layer perturbations that disrupt the coordinated structure supporting the edits.

4. Mechanistic Insights (Activation Space)

Magnitude: Fine-tuning induces substantially larger activation magnitude changes across all layers compared to KE, which induces limited, localized perturbations.
Direction: The directional shift induced by fine-tuning is nearly orthogonal to the shift induced by KE.
UMAP Analysis: In representation space, the cluster of fine-tuned models ( $M_{ft}$ ) and fine-tuned-then-edited models ( $M_{ed\_ft}$ ) substantially overlap, while both are clearly separated from the edited-only model ( $M_{ed}$ ). This indicates that fine-tuning pulls the model's internal representations back toward the fine-tuning direction, effectively overwriting the subtle changes introduced by KE.

Significance and Claims

The paper concludes that existing KE methods fail to produce consistently stable edits that persist under subsequent fine-tuning. The authors claim this highlights a critical fragility in the current LLM application pipeline.

Safety Implications: The risk of "inheritance" of malicious edits is real; if edits persist, fine-tuned models may propagate hidden biases or backdoors. Conversely, if edits are erased, the utility of KE for correcting harmful base-model behaviors is compromised.
Deployment Guidance: The study underscores the necessity of evaluating KE within the broader context of the LLM lifecycle, not in isolation.
Future Directions: The authors suggest that future KE research must explicitly account for downstream adaptation. They propose potential mitigation strategies, such as incorporating regularization to penalize activation shifts in edited layers or projecting FT updates onto the orthogonal subspace of KE parameters, though these are presented as directions for future work rather than implemented solutions in this study.

The authors maintain a modest tone, noting that while their findings are robust across the tested architectures (which share the Transformer backbone), the specific degrees of decay may vary with other models or configurations. They provide their code and results to serve as a baseline for future research into robust model editing.