💬 NLP

Decomposition-Induced Context-Memory Conflict: When Fact-Checking Pipelines Contradict Their Own Source Text

This paper identifies and characterizes "Decomposition-Induced Context-Memory Conflict" (DI-CC), a failure mode in fact-checking pipelines where the decomposition stage substitutes a model's parametric beliefs for source text, creating contradictions that evade standard self-consistency detection but can be partially mitigated by context-aware decoding at the cost of parsing reliability.

Original authors: Yu-Feng Yen

Published 2026-08-12

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu-Feng Yen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to tell the truth. You give it a long, complicated story and ask it to break that story down into tiny, simple sentences so it can check each one for accuracy. This is a popular trick in the world of Artificial Intelligence, known as "decompose-then-verify." Think of it like a student who, before taking a history test, breaks a long textbook chapter into a list of bullet points to study. The hope is that by looking at one small fact at a time, the robot won't get confused.

But here's the catch: the robot doing the breaking-down isn't just a passive pair of scissors; it's a smart, chatty brain that already knows a lot of things from its own memory. Sometimes, when the robot reads a sentence in the story, it gets so confident in its own memory that it accidentally swaps the story's facts with what it thinks is true. It's like a student reading a history book that says "The war started in 1940," but because the student is sure it was 1941, they rewrite their study notes to say 1941. They haven't just made a mistake; they've actively overwritten the source text with their own belief. This paper explores exactly that moment of confusion, where the robot's internal memory fights with the text it's supposed to be summarizing, and asks: does this happen, and can we catch it?

The Paper's Big Discovery: When the "Fact-Checker" Lies to Itself

The researcher behind this study, led by Yu-Feng Yen, decided to investigate a sneaky problem in how AI checks its own work. They call this phenomenon Decomposition-Induced Context-Memory Conflict, or DI-CC for short.

Here is the story of what they found:

1. The "Magic Trick" of the Robot Brain
The team set up an experiment where they asked an AI to break down a biography of a famous person into tiny claims. They secretly changed one detail in the biography (like a birth year) to see if the AI would catch it. They found that when the AI was told, "Hey, check your own memory if you think something is wrong," it often did exactly that—but in the wrong way. Instead of sticking to the story, the AI would look at the changed detail, realize it didn't match what it "knew" from its training, and then rewrite the claim to match its own memory, completely ignoring the story it was supposed to be summarizing.

It's like a translator who is reading a book in French. If the book says "The cat is blue," but the translator is absolutely sure cats are never blue, they might "correct" the translation to "The cat is black" without realizing they just changed the author's story. The paper shows this isn't just a random glitch; it's the same kind of brain-confusion that happens when an AI is asked a direct question, just happening earlier in the process.

2. Catching the Lie with a "Lie Detector"
To prove this wasn't just the AI making up random nonsense, the researcher built a special "lie detector." They trained a simple tool on a different kind of problem (where an AI has to choose between a story and its own memory). Then, they asked this tool to look at the AI's brain activity while it was breaking down the story.

The result was surprisingly clear. The tool could spot the moments when the AI was swapping the story's facts for its own beliefs with about 86% to 88% accuracy (a score called AUC). This is much better than random guessing. It means the "confusion" has a specific fingerprint in the AI's brain that we can see, even if we've never seen this specific type of mistake before.

3. Why the "Self-Check" Failed
The researcher also tried a popular method called "SelfCheckGPT," which works by asking the AI the same question ten times and seeing if it gives the same answer every time. If the answers change, it assumes the AI is lying.

The Result: This method failed completely. It scored right at 51%, which is basically a coin flip.
The Reason: The paper explains this is actually predictable. Because the AI is so confident in its own memory, it gives the same wrong answer every single time. Since the answers are consistent, the "SelfCheck" tool thinks, "Oh, it's consistent, so it must be true!" The tool is fooled because the AI is too stubborn to change its mind, even when it's wrong.

4. The "Fix" That Breaks Things
The team tried a known fix called "Context-Aware Decoding" (CAD), which is like telling the AI, "Please pay extra attention to the story, not your memory."

The Good News: It worked! It reduced the number of times the AI swapped facts from 4.16% down to 2.57%.
The Bad News: It came with a huge price tag. When the story had lots of pronouns (like "he," "she," "they"), the AI got so confused trying to follow the rule that it stopped making sense entirely 19.4% of the time. In 84% of those failures, the AI didn't just skip a detail; it completely made up a different person's identity.
The Conclusion: The author says this fix is not ready for real-world use yet. It's like a bandage that stops the bleeding but makes the patient pass out.

5. How Big Does the Robot Need to Be?
The researcher tested different sizes of AI models to see if bigger brains were better at this.

Small models (3 billion parameters): They couldn't show the signal at all. They were too simple to have this specific kind of conflict.
Medium models (7 billion parameters): They showed the conflict clearly.
Large models (14 billion parameters): They showed the conflict even more strongly, but only if you looked at the right part of their brain.
The Takeaway: This isn't a problem that gets better just by making the AI bigger. There seems to be a "minimum size" required for this specific type of confusion to even happen.

6. How Often Does This Actually Happen?
The most important part of the paper is the honesty about how common this is. The experiments above used a special setup where the AI was invited to check its own memory. But in the real world, we usually just ask the AI to summarize without that invitation.

The Reality Check: When the researcher looked at natural, unedited text, this specific type of fact-swapping was incredibly rare. It happened in only 0.2% to 0.4% of the claims.
Why? Real-world hallucinations (lies) usually happen in areas where the AI doesn't know the answer. This specific problem (DI-CC) only happens when the AI does know the answer but decides to ignore the story anyway.

The Bottom Line

This paper proves that when AI breaks down text into small pieces, it can sometimes get so confident in its own memory that it rewrites the story to fit what it thinks is true. We can detect this with a special tool, and we can even reduce it with a specific technique, but that technique currently causes the AI to make up fake people.

Most importantly, the author warns us not to panic. While this is a real and measurable glitch, it is very rare in natural situations. It's a specific type of "over-confidence" that only happens under certain conditions, not a bug that ruins every single thing the AI does. The paper doesn't offer a perfect solution yet, but it gives us a much clearer map of where the problem lives and how big it really is.

Technical Summary: Decomposition-Induced Context-Memory Conflict (DI-CC)

1. Problem Definition

Long-form factuality evaluation and hallucination detection increasingly rely on decompose-then-verify pipelines (e.g., FActScore, DnDScore). In this paradigm, a language model (the "decomposer") splits a generated passage into atomic claims, which are then independently verified against a reference. This decomposition step is traditionally viewed as a neutral bookkeeping task.

However, decomposers are themselves language models prone to context-memory conflict: the tendency to substitute their own parametric knowledge for information provided in the context when the two disagree. This paper investigates whether this failure mode occurs within the decomposition stage itself.

The author defines Decomposition-Induced Context-Memory Conflict (DI-CC) as a specific failure mode where a decomposer, when converting a source passage $R$ into atomic claims $c_i$ , produces a claim that:

Is not entailed by the source text $R$ .
Contains content traceable to the decomposer's own parametric knowledge (recoverable via closed-book probing).
Directly contradicts information explicitly stated in $R$ .

This is distinguished from Decomposition-Induced Unsupported Elaboration (DI-UE), where the model injects content that fills a gap in $R$ without contradicting it. Both share the same injection mechanism, but only DI-CC represents a strict conflict.

2. Methodology and Experimental Design

The study employs a multi-phase approach to validate the existence, mechanism, and boundaries of DI-CC.

A. Mechanistic Validation (H0)
To prove DI-CC is mechanistically identical to classical context-memory conflict (rather than a distinct form of hallucination), the author utilizes a linear probe trained exclusively on classical conflict data (NQ-Swap dataset).

Setup: The probe is trained on activation vectors from NQ-Swap (where context and parametric knowledge disagree) and frozen. It is then applied zero-shot to the activation vectors of a decomposer generating atomic claims.
Hypothesis: If DI-CC shares the same underlying mechanism as classical conflict, the probe should successfully distinguish DI-CC positions from faithful decompositions and non-contradictory elaborations (DI-UE), despite never having seen decomposition data during training.
Data Construction: To ensure sufficient statistical power, the author uses an elicited construction. They generate biographies with tampered details (e.g., shifted years) and provide an instruction explicitly authorizing the decomposer to check facts against their own knowledge and correct errors. This "licensing" is empirically shown to be a necessary condition for DI-CC to manifest; neutral instructions yield near-zero DI-CC rates.
Baseline Comparison: The study compares this mechanistic approach against SelfCheckGPT, a reference-free baseline that relies on self-consistency sampling (detecting hallucinations via disagreement across resamples).

B. Natural Rate Analysis (H1)
The author investigates the prevalence of DI-CC under non-elicited, natural conditions using FActScore's unlabeled data and neutral decomposition instructions. They regress the injection rate against textual "ellipticity" (pronoun density, low entity density) to see if sparse text triggers more conflicts.

C. Mitigation Study (H3)
The paper tests Context-Aware Decoding (CAD), a training-free mitigation technique from classical conflict settings, adapted for the decomposition pipeline. CAD contrasts the output distribution conditioned on context against the distribution without context to amplify attention to the source text.

D. Scale Ablation (H4)
The author tests the signal across model scales (3B, 7B, 14B) to determine if the phenomenon is scale-dependent. A critical methodological adjustment is made here: the author switches from standard held-out accuracy for layer selection to a variance-ratio criterion (between-class vs. within-class variance), citing literature suggesting accuracy is a poor proxy for out-of-distribution transfer in probing tasks.

3. Key Results

Mechanistic Realization (Main Result)

Probe Performance: A probe trained only on classical conflict data successfully detects DI-CC in decomposition activations zero-shot.
- AUC: 0.86–0.88 (95% CI excludes chance) for distinguishing DI-CC from faithful claims and DI-UE.
- Significance: Permutation $p < 0.0005$ .
- Replication: The effect size of an initial $n=14$ pilot is reproduced at a larger $n=64$ sample, with every positive case independently human-verified.
Baseline Failure: SelfCheckGPT fails to detect DI-CC (AUC $\approx$ $\approx$ 0.51, chance-level).
- Mechanistic Explanation: DI-CC involves stable, recoverable parametric content. Consequently, it recurs consistently across resamples, appearing "trustworthy" to self-consistency methods. In contrast, DI-UE (filling gaps) varies more across resamples, which SelfCheckGPT flags as unreliable.

Natural Occurrence Rate

Under natural, non-elicited instructions, the DI-CC rate is extremely low (0.2–0.4% of atomic claims).
The rate is too sparse to reliably regress against textual features.
Boundary Condition: DI-CC does not manifest on naturally occurring hallucinated text in FActScore benchmarks. Real-world hallucinations tend to occur in knowledge gaps (where the model has no opinion) rather than overriding correct beliefs, which is the specific condition required for DI-CC.

Mitigation and Trade-offs

Suppression: CAD significantly reduces the DI-CC rate from 4.16% to 2.57% ( $p = 0.000041$ ).
Severe Cost: Under coreference-heavy conditions, 19.4% of decompositions fail to parse entirely under CAD.
Nature of Failure: 84% of these parsing failures involve the decomposer fabricating a different person's identity rather than merely omitting detail.
Conclusion: The author does not consider this implementation deployment-ready due to the faithfulness-completeness trade-off.

Scale and Layer Selection

3B Model: Fails to show the signal under both accuracy-based and variance-ratio layer selection.
7B Model: Succeeds under both criteria.
14B Model: Succeeds under variance-ratio selection (AUC 0.983) but fails under accuracy-based selection.
Interpretation: The signal appears to have a capability threshold between 3B and 7B. The erratic results at 14B under accuracy selection are attributed to the poor transfer properties of accuracy-based layer selection, not model cognition.

4. Significance and Claims

The paper establishes DI-CC as a real, mechanistically grounded, and partially treatable failure mode of decompose-then-verify pipelines.

Mechanistic Continuity: The primary contribution is demonstrating that decomposition-induced errors are not merely "adding stuff" but are the same phenomenon as classical context-memory conflict, occurring one pipeline stage earlier than previously examined.
Methodological Rigor: The study highlights that probing layer selection is a dominant factor in detecting these signals. It demonstrates that accuracy-based selection can yield inconsistent transfer signals across scales, whereas variance-ratio selection (grounded in OOD-generalization literature) reveals a coherent capability threshold.
Scope and Modesty: The author explicitly limits their claims regarding prevalence. They state that while the mechanism exists and is detectable under elicited conditions, its natural occurrence rate is currently too low to characterize reliably. They do not claim to have solved the problem; rather, they characterize the boundaries of the failure mode and show that existing mitigations (CAD) introduce severe, previously unreported side effects (identity fabrication).

In summary, the paper argues that decomposition is not a neutral preprocessing step but a site where parametric beliefs can actively overwrite source text, a phenomenon that is detectable via mechanistic probes but remains elusive to standard self-consistency baselines and difficult to mitigate without compromising completeness.