How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation

Technical Summary: How Do LLMs Cite? A Mechanistic Interpretation of Attribution in RAG

Problem Statement
Retrieval-Augmented Generation (RAG) systems aim to mitigate Large Language Model (LLM) hallucinations by grounding outputs in external documents and providing inline citations. However, a critical distinction exists between citation correctness (the source factually supports the claim) and citation faithfulness (the model actually used the source to generate the claim). Current evaluation methods often rely on input-output behavior, which cannot distinguish between a model genuinely reasoning over a retrieved document and one that retrieves a document post-hoc to match a pre-existing parametric memory. This "cosmetic" attribution creates a false sense of security, particularly in high-stakes domains like law and medicine, where users may lack the expertise to verify sources independently. The paper argues that understanding the internal mechanisms of citation generation is necessary to develop reliable evaluation frameworks for faithfulness.

Methodology
The authors employ mechanistic interpretability to reverse-engineer the decision-making process of Llama-3.1-8B-Instruct regarding inline citations. The experimental pipeline involves:

1. Controlled Dataset: A citation-focused adaptation of PopQA is used, where each question is paired with either a relevant supporting document or an irrelevant distractor document. Distractors are structurally matched to support precise component-level comparisons.
2. Activation Patching: The core technique involves running the model with a relevant document (clean) and an irrelevant document (corrupted). Intermediate activations from the clean run are patched into the corrupted run (and vice versa) to measure the causal impact on the model's preference for emitting a citation token ([) versus a sentence-termination token (.).
3. Scoring: A normalized logit-difference score is calculated to quantify the fraction of behavioral change recovered or induced by specific patches.
4. Intervention: Identified components are manipulated via multiplicative scaling (amplification or suppression) to test if citation failures (missed or spurious) can be corrected.
5. Generalization: Findings are validated on a more complex, multi-document setting using HotpotQA.

Key Contributions

1. Mapping the "Attributional Ensemble": The study identifies that citation decisions are not governed by a single specialized module but by a distributed, multi-stage ensemble of specific attention heads and MLP layers.
2. Demonstration of Controllability: The authors demonstrate that targeted interventions on this ensemble can successfully repair citation failures, inducing missing citations and suppressing spurious ones.
3. Generalization Validation: The identified components show generalizability, significantly improving citation accuracy in the complex, multi-document HotpotQA setting without retraining.

Key Results

• Distributed and Fragile Mechanism: The citation mechanism relies on a conjunctive (AND-gated) structure. While many components contribute, the system is "fragile": small corruptions at multiple sites drastically reduce citation behavior, whereas inducing citations via denoising is difficult and requires the accumulation of many signals.
• Shallow Heuristics Drive Decisions: The mechanism relies heavily on entity co-reference matching rather than deep semantic integration.
  • Early Layers (0–2): An "entity enrichment" step occurs where the model identifies the queried entity in the document and marks its corresponding token in the question. This creates an internal flag for cross-context identity.
  • Mid-Layers (10–20): A dense band of components acts as a processing hub, performing conjunctive checks.
  • Late Layers: Final aggregation occurs at the answer token, where specific heads and MLPs increase the logit of the citation marker over termination tokens.
• Independence from Answer Generation: Crucially, the representation of the answer itself at the moment of generation is not the primary driver of the citation decision. This suggests the citation is generated by a parallel, heuristic-based process (checking if the document contains the entity) rather than being a direct result of the answer construction process.
• Intervention Success:
  • Amplifying identified components in PopQA recovered >90% of missed citations with negligible impact on answer correctness.
  • Suppressing necessary components removed 69% of spurious citations.
  • In HotpotQA, amplification increased the strict "correct citation" rate by over 20-fold (from 0.001 to 0.024), though absolute performance remained low, indicating the complexity of multi-hop reasoning.

Significance and Claims
The paper claims that current RAG systems may offer an illusion of verifiability. Because citation decisions are driven by shallow heuristics (like entity matching) rather than a genuine causal link to the document content during answer generation, a model can produce a factually correct answer with a "faithful" citation even if the answer was retrieved from parametric memory.

The authors argue that:

• Auditing outputs is insufficient: Trustworthy attribution requires auditing the computational pathway.
• Mechanistic interpretability is essential: To distinguish faithful from unfaithful attribution, one must look inside the model, as input-output behavior is ambiguous.
• Risk of Misinformation: Unfaithful citations can deepen information disparities by lending credibility to incorrect information, particularly for users who cannot independently verify sources.

The work is presented as a foundational step toward building evaluation frameworks that can distinguish faithful attribution, rather than a complete solution. The authors acknowledge limitations, including the focus on a single 8B model, the lack of directed pathway mapping (only component localization), and the potential for adversarial misuse of these insights to generate unfaithful citations.