Uncovering Context Reliance in Unstructured Knowledge Editing

1. Problem Statement

Large Language Models (LLMs) acquire knowledge primarily through Next-Token Prediction (NTP) during pre-training. While Knowledge Editing (KE) is essential for correcting factual errors or updating outdated information, existing methods face significant challenges when applied to unstructured text (e.g., paragraphs, articles) compared to structured triplets (Subject-Relation-Object).

The paper identifies a critical failure mode in NTP-based unstructured editing called Context Reliance.

• The Phenomenon: When an LLM is edited using unstructured text, the newly acquired knowledge becomes heavily dependent on the specific preceding context present during the editing process.
• The Consequence: During inference, if the query does not include the exact preceding context used during editing, the model fails to recall the edited knowledge. This effect is exacerbated for knowledge appearing later in a text, where the preceding context is longer, leading to a sharp decline in recall accuracy as the position of the knowledge moves deeper into the text.
• The Gap: Current methods often decompose unstructured text into triplets or construct specific queries, which oversimplifies semantics. Direct NTP-based editing (like Fine-Tuning) preserves semantics but suffers from this context dependency, rendering the knowledge brittle and non-generalizable.

2. Methodology: The COIN Framework

To address Context Reliance, the authors propose COIN (COntext-INdependent editing), a framework designed to decouple knowledge from its specific context. COIN introduces two novel loss functions added to the standard NTP objective:

A. Context Alignment Loss ($L_{align}$)

• Goal: To force the model to internalize knowledge based on local cues rather than memorizing the global context sequence.
• Mechanism: The loss minimizes the Kullback-Leibler (KL) divergence between the model's prediction distribution conditioned on the global context (the full editing text) and the local context (a sliding window of the last $k$ tokens).
• Effect: By aligning these distributions, the model is penalized if it relies on distant tokens to make predictions. It is forced to capture the essential information required for the target token within a short, local scope, thereby breaking the dependency on the full preceding sequence.

B. Knowledge Consistency Loss ($L_{cons}$)

• Goal: To prevent model collapse (degradation of general capabilities) and preserve unrelated knowledge.
• Mechanism: Based on the view that MLP layers act as associative memories, this loss constrains the weight updates ($W$) to ensure that the output for inputs derived from general knowledge (keys $K_0$) remains similar to the pre-edit weights ($W_0$).
• Efficiency: Instead of forwarding a massive dataset, the authors pre-compute the uncentered second-moment matrix ($K_0 K_0^T$) from a general corpus (e.g., Wikipedia) and use it as a proxy to penalize weight changes along frequently activated directions.

Total Objective:
$$L_{COIN} = L_{NLL} + \alpha L_{align} + \beta L_{cons}$$
Where $L_{NLL}$ is the standard negative log-likelihood, and $\alpha, \beta$ are trade-off hyperparameters.

3. Key Contributions

1. Identification of Context Reliance: The paper empirically and theoretically establishes that NTP-based editing inherently leads to knowledge being entangled with aggregated contextual representations.
   • Empirical Evidence: Performance drops significantly (up to 56% in ROUGE) as knowledge position moves later in the text. Restoring the original context during inference recovers performance, confirming the dependency.
   • Theoretical Proof: The authors provide a theorem showing that a single gradient update in a transformer binds the target token to an aggregated representation of context tokens (e.g., $p$ and $q$). If a context token is missing at inference, the aggregated representation shifts, causing the model to revert to pre-edit behavior.
2. Proposed COIN Framework: A simple yet effective method to mitigate this reliance by enforcing context independence via distribution alignment and preserving general capabilities via consistency constraints.
3. Comprehensive Evaluation: Extensive experiments on Llama3-8B and Qwen2.5-7B across AKEW and UnKEBench benchmarks, demonstrating superior performance over state-of-the-art baselines.

4. Experimental Results

The authors evaluated COIN against baselines including Fine-Tuning (FT), LoRA, AdaLoRA, UnKE, AnyEdit, and AlphaEdit-D (triplet decomposition).

• Mitigation of Context Reliance: COIN reduced the performance degradation caused by Context Reliance by 45.2% (measured by the drop in ROUGE scores from early to late text positions) compared to standard Fine-Tuning.
• Editing Success Rate: COIN achieved a 23.6% improvement in editing success rate (F1 score) over the best baselines on unstructured tasks.
• Generalization:
  • Unstructured Tasks: COIN significantly outperformed query-construction methods (UnKE, AnyEdit) which failed to generalize to semantically related questions.
  • Structured/Multi-hop Tasks: Surprisingly, COIN also excelled in structured multi-hop reasoning (MQUAKE dataset), outperforming specialized structured editing methods (like ROME/MEMIT). This suggests that NTP-based editing, when context-independent, captures complex relational dependencies better than isolated triplet editing.
• Robustness: The method maintained general capabilities (GLUE benchmark) even after batch editing, avoiding model collapse.

5. Significance

• Paradigm Shift: The paper challenges the assumption that NTP is a "plug-and-play" solution for unstructured editing. It highlights that without specific regularization, NTP leads to brittle, context-bound knowledge.
• Practical Impact: COIN offers a pathway to update LLMs with real-world, unstructured data (news, reports, articles) without the need for complex triplet extraction or query construction, making knowledge updates more scalable and faithful to the source text.
• Theoretical Insight: The work provides a fundamental understanding of how gradient-based optimization in transformers can inadvertently create "context traps," suggesting that future editing methods must explicitly address the decoupling of knowledge from its acquisition context to achieve robustness.

In conclusion, COIN demonstrates that by forcing models to focus on local, intrinsic knowledge rather than memorizing global contextual patterns, it is possible to achieve robust, context-independent knowledge editing in LLMs.