💬 NLP

Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models

This paper presents a mechanistic study of intra-memory knowledge conflicts in language models, revealing that while such conflicts are typically resolved in the final layers via distinct circuits rather than a universal mechanism, interventions are significantly less effective on real-world conflicts compared to synthetic ones due to differences in specialized attention head distributions.

Original authors: Minh Vu Pham, Hsuvas Borkakoty, Yufang Hou

Published 2026-08-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minh Vu Pham, Hsuvas Borkakoty, Yufang Hou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a giant, digital brain that has read almost everything on the internet. This brain, known as a Large Language Model (LLM), is like a super-smart librarian who can answer questions, write stories, and solve problems. But here's the catch: sometimes, this librarian gets confused. If you ask, "Where did Niels go to school?" the librarian might remember two different answers at the same time. Maybe one part of its memory says "University of A," while another part insists on "University of B." This isn't because the librarian is looking at a new book (external information); it's because the librarian's own internal notes are messy. Scientists call this "intra-memory knowledge conflict." It's a bit like having two different versions of the same story written in the same notebook, and the librarian doesn't know which one to read out loud. This is a big deal because if these AI systems are going to be trusted with real-world tasks, they need to stop hallucinating or giving contradictory answers. Researchers want to know: exactly where in the digital brain does this confusion happen, and can we fix the specific part that's causing the mix-up?

This paper, titled "Where Knowledge Collides," dives deep into the inner workings of these AI brains to find the source of the confusion. The authors built a special testing ground using four different AI models. They created a dataset of fake biographies—think of them as made-up Wikipedia pages for people who don't exist—where they deliberately planted conflicting facts. For example, they told the AI that a fictional person named "Niels Cavalli" graduated from "University of Ukopnwm," but then they also told the AI that the same Niels graduated from "University of Ohfgrgr." By training the AI on these mixed-up stories, they created a controlled environment where the model had to hold two opposing facts in its head at once.

To figure out what was going on, the researchers used a technique called "mechanistic interpretability." Imagine the AI as a multi-story factory where information flows from the bottom floor to the top. Each floor has workers (called "layers") and specific teams within those floors (called "attention heads") that handle different parts of the job. The researchers used a tool called a "logit lens" to peek inside the factory at every single floor, watching how the AI's confidence changed as it processed the question. They found something very consistent: the confusion didn't happen at the beginning or the middle of the factory. Instead, the conflict always erupted in the final few layers. It's as if the AI spends most of its time gathering facts, but right at the very end, just before it speaks, the two conflicting memories crash into each other, causing the model to wobble and become uncertain.

The team then tried to fix the problem by "patching" the AI. This is like reaching into the factory and swapping out the work of a specific team with the work of a team that knows the right answer. They tried two main approaches: fixing an entire floor (layer-wise intervention) or fixing just one specific team (head-level intervention). The results were clear: fixing the whole floor was okay, but fixing the specific team was much, much better. In fact, targeting the specific "attention heads" that were responsible for the wrong fact allowed them to steer the AI toward the correct answer with high success. They even discovered that in their fake, controlled world, there were often "specialist" heads that only cared about one specific fact. However, when they tried this on real-world data (using a dataset called DynamicQA with actual, messy human knowledge), the fixes were less effective. This suggests that real-world conflicts are more tangled and harder to untangle than the clean, synthetic ones they created in the lab.

One of the most interesting things the paper argues against is the idea of a "universal circuit." You might think that all AI models have one specific, dedicated switch or pathway that handles all knowledge conflicts, like a "conflict resolution button." The authors suggest this is probably not true. Instead, their findings point to the idea that different pieces of knowledge are stored in different, separate circuits. The final answer the AI gives depends on which of these competing circuits gets the strongest signal from your question. There isn't one single "conflict manager"; it's more like a tug-of-war between different groups of workers, and the one that pulls the hardest wins.

The researchers also noticed that one of the models, GPT-2 XL, was harder to fix than the others. They figured out this wasn't because it was broken, but because it was just bigger. It had so many more "workers" (attention heads) that they needed to fix a larger number of them to see the same results as the smaller models. It's like trying to fix a traffic jam in a tiny village versus a massive city; you need to clear more cars in the city to get the same flow.

In the end, this study gives us our first real map of where these internal conflicts happen. It tells us that the battle between conflicting facts happens right at the finish line of the AI's thinking process. While we can't yet perfectly fix these conflicts in the messy real world, the paper suggests that by targeting the specific, tiny teams responsible for the wrong facts, we can make these AI systems more reliable. It's a crucial step toward building AI that doesn't just sound smart, but actually knows what it's talking about.

Technical Summary: Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models

Problem Definition

Large Language Models (LLMs) face a critical reliability challenge known as intra-memory knowledge conflict. This occurs when inconsistent information regarding the same subject is encoded within the model's parametric knowledge, leading to unstable internal representations where different prompts elicit conflicting factual associations. While prior research has extensively addressed context-memory conflicts (resolving tensions between internal knowledge and external context via fine-tuning or knowledge editing), the mechanisms governing internal conflicts remain largely unexplored. Specifically, it is unknown where within the model's architecture these conflicts arise, which components are responsible for resolving them, and how effectively targeted interventions can alter model behavior to resolve these conflicts.

Methodology

The authors propose a mechanistic interpretability framework to localize and causally intervene on intra-memory conflicts. The study utilizes a three-stage pipeline:

Dataset Construction:
- Synthetic Data (SynWikiBio): A curated dataset of Wikipedia-style biographies for synthetic entities. The authors generated 2,000 unique individuals, creating 1,000 pairs of conflicting biographies (e.g., differing universities or companies) and 1,000 conflict-free biographies. This allows for controlled experimentation where ground truth and conflicting facts are precisely defined.
- Real-World Data (DynamicQA): To test generalizability, the authors utilized the DynamicQA dataset, which contains static, temporal, and disputable facts. They constructed conflicting contexts by replacing factual knowledge with counterfactuals provided in the dataset.
Model Training:
- Four LLMs of varying architectures and scales were evaluated: GPT-2 XL (1.5B), Llama-3.2-1B, OPT-2.7B, and Qwen3-4B.
- $\ell_{mix}$ : Models were continued pre-trained on the mixed dataset ( $B_{mix}$ ) containing both conflicting and conflict-free biographies, resulting in models with internal knowledge conflicts.
- $\ell_{clean}$ : Models were continued pre-trained on a conflict-free subset ( $B_{clean}$ ), serving as a reference for non-conflicting parametric representations.
Causal Probing and Intervention:
- Logit Lens: Used to inspect residual stream representations at each layer to localize where conflicting knowledge is encoded and how prediction confidence evolves.
- Activation Patching: A causal tracing method where activations from a "source" prompt (conflict-free) are patched into a "target" prompt (conflicting) to measure the causal contribution of specific components (layers or heads) to the model's output.
- Cross-Model Activation Patching (CMAP): An advanced variant where activations are sourced from the clean model ( $\ell_{clean}$ ) and injected into the conflicting model ( $\ell_{mix}$ ). This mitigates issues where finding a non-conflicting source prompt within the same dataset is difficult or prone to entanglement.
- Intervention Techniques: The study employed Activation Patching, Zero Ablation (zeroing out activations), and JUICE (amplifying specific head activations) to test the ability to steer model predictions away from conflicting facts.

Key Results

1. Localization of Conflict

Across all four tested models, intra-memory conflicts consistently arise and are resolved in the final few layers of the network.

Confidence Evolution: Analysis of entropy and KL divergence reveals that models with and without conflicting knowledge converge similarly through the middle layers. Divergence occurs only in the final layers (e.g., layers 40–47 in GPT-2 XL), where the conflicting model exhibits rising uncertainty while the non-conflicting model continues to converge toward a confident prediction.
Component Specificity: Attention components show stronger aggregate contributions to conflict resolution than MLP components.

2. Effectiveness of Interventions

Head-Level vs. Layer-Level: Targeted interventions at the attention-head level significantly outperform layer-wise interventions.
Synthetic vs. Real-World Gap: Interventions are markedly more effective on the synthetic SynWikiBio dataset than on the real-world DynamicQA dataset.
- For example, in GPT-2 XL, the Steering Success Rate (SSR) for layer-wise patching dropped from 0.214 (synthetic) to 0.141 (real-world).
- Filtering analysis suggests this gap exists because synthetic conflicts often rely on heads specialized for a single competing fact, whereas real-world conflicts involve more complex, less specialized decision-making heads due to uncontrolled data distributions.
Method Performance:
- JUICE emerged as the most robust intervention method across models and datasets.
- CMAP performed comparably to standard Activation Patching on synthetic data and offered a more practical setup for real-world conflicts where finding clean source prompts is difficult.

3. Model-Scale and Architecture Dependencies

GPT-2 XL Anomaly: GPT-2 XL showed lower intervention success rates initially. Further analysis revealed this was due to its larger total number of attention heads (1,200). When the number of intervened heads was scaled proportionally, its performance trend aligned with other models, suggesting the task-relevant signals are distributed across a wider set of heads in GPT-2 XL compared to models like OPT-2.7B or Qwen3-4B.
No Universal Circuit: The study found no evidence of a single, universal circuit dedicated to handling knowledge conflict across all models or subjects. Instead, distinct circuits appear to encode different pieces of parametric knowledge, with the final prediction determined by which circuit's signal is most strongly activated by the input prompt.

Significance and Claims

The paper claims to provide the first mechanistic account of intra-memory knowledge conflict resolution in LMs. Its primary contributions are:

Localization: Establishing that internal conflicts are not distributed uniformly but are localized to the final layers and specific attention heads.
Intervention Hierarchy: Demonstrating that fine-grained, head-level interventions are superior to coarse layer-level interventions, particularly when targeting specialized heads.
Methodological Advancement: Validating Cross-Model Activation Patching (CMAP) as a viable and often necessary alternative to standard Activation Patching for studying real-world knowledge conflicts.
Synthetic-Real Gap: Highlighting a substantial performance gap between synthetic and real-world settings, underscoring that while synthetic datasets are useful for mechanistic discovery, the complexity of real-world data (bias, noise, entity distribution) significantly complicates conflict resolution.

The authors conclude that LMs do not rely on a single "conflict resolution" module but rather on a competition between distinct circuits encoding different facts. This suggests that steering model behavior requires amplifying desired signals or suppressing competing ones at the specific head level, rather than applying broad architectural changes. The study acknowledges limitations, including the reliance on Logit Lens projections (which are less reliable in early layers) and the use of synthetic conflicts that may not perfectly mirror the statistical properties of naturally occurring pre-training conflicts.