Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge: A keynote at ECIR 2025

This keynote presentation at ECIR 2025 explores the critical interplay between a language model's internal parametric knowledge and external contextual information, focusing on diagnostic methods to identify knowledge conflicts and strategies to improve the model's ability to utilize retrieved context effectively.

The Gist

Imagine a Large Language Model (LLM) as a super-smart, well-read librarian who has memorized an entire library of books (the internet) during their training. This librarian has two ways of answering your questions:

1. The Memory Shelf (Parametric Knowledge): The facts they have memorized and stored in their brain.
2. The Reference Desk (Contextual Knowledge): The specific documents or notes you hand them right now to help answer a question.

This keynote by Isabelle Augenstein is like a detective story investigating what happens when these two sources of information clash, and how the librarian decides which one to trust.

Here is the breakdown of the paper's main discoveries, explained with everyday analogies:

1. The Librarian's Memory is a Bit "Glitchy"

The librarian is great at reciting facts they memorized, but they aren't perfect.

• The Problem: Sometimes, the librarian memorized a fact incorrectly, or the fact has changed (like a sports team changing its name), but the librarian still insists on the old answer.
• The "Recitation vs. Reasoning" Trap: The paper points out that these models are often better at reciting what they've heard before than actually reasoning through a new problem. It's like a student who memorized the answers to last year's math test but fails if you change one number in the question.
• The "Hallucination" Dilemma: For creative writing, making things up (hallucinating) is fun. For fact-checking, it's a disaster. The librarian needs to know when to stop making things up and stick to the truth.

2. The "Memory vs. Note" Conflict

This is the core of the research. What happens when you hand the librarian a note that contradicts their memory?

• Scenario A (Static Facts): You ask, "Who is the President?" and hand them a note saying "It's Person X," but their memory says "It's Person Y." The librarian often ignores your note and sticks to their memory, even if the note is right.
• Scenario B (Dynamic Facts): You ask about a sports score that changes every day. Here, the librarian is more likely to listen to your new note because they know their memory might be outdated.

The Big Surprise: The researchers found something counter-intuitive. The librarian is actually easier to trick with static facts (things that never change) than with dynamic ones. If you tell the librarian, "The capital of Japan is Stockholm" (a static fact), they might believe you if you say it confidently enough. But if you try to change their mind about a sports score, they are more skeptical.

3. The "Flashlight" Experiment (Attribution)

The researchers tried to figure out exactly which part of the librarian's brain is doing the thinking. They used a "flashlight" (a technical method called attribution) to see which neurons (brain cells) were lighting up when the librarian answered.

• The Finding: They expected to find that specific "fact neurons" held the answers. Instead, they found that the librarian's brain is much more complex. It's not just one neuron holding a fact; it's a messy network.
• The Takeaway: Trying to "edit" the librarian's memory by just changing a few neurons is like trying to fix a leaky roof by painting a single shingle. It often causes "ripple effects," breaking other things you didn't intend to fix.

4. The "Real World" vs. "Fake World" Test

The researchers realized that most previous studies used synthetic data (fake, made-up scenarios) to test how well the librarian uses notes.

• The Analogy: It's like testing a driver's skills only on a video game simulator. The simulator is perfect, but the real world is messy.
• The Reality Check: When they tested the librarian with real-world documents (like actual news articles and fact-checking reports), the results were different.
  • In the fake world, the librarian would easily ignore conflicting notes.
  • In the real world, the librarian is actually quite good at using the notes, but only if the notes are clear, assertive, and written by experts. If the notes are vague or come from unreliable sources, the librarian ignores them and falls back on their (potentially wrong) memory.

5. The "Spiral" of Research

The paper concludes with a beautiful metaphor from the late Karen Spärck Jones (a pioneer in the field):

"Research is not so much going round in circles as ascending a spiral."

This means that while we keep asking the same old questions (like "How do we make AI smarter?"), we are answering them with better tools and deeper understanding. We aren't just spinning our wheels; we are climbing up a spiral staircase, getting a better view with every step.

Summary for the General Audience

This paper tells us that AI isn't just a magic oracle; it's a complex system with a stubborn memory.

• It often ignores new information if it conflicts with what it already "knows."
• It is surprisingly easy to trick with confident-sounding lies about static facts.
• To fix it, we can't just "retrain" it (too expensive) or "edit" its brain (too risky).
• Instead, we need to understand how it chooses between its memory and the documents we give it, and we need to test it with real-world data, not just fake scenarios.

The ultimate goal is to build a system that knows when to trust its memory and when to listen to the new evidence you provide.

Technical Summary

Based on the keynote extended abstract by Isabelle Augenstein titled "Understanding the Interplay between LLMs' Utilisation of Parametric and Contextual Knowledge," here is a detailed technical summary.

1. Problem Statement

Large Language Models (LLMs) possess parametric knowledge (encoded in weights during pre-training) but often struggle with factuality, reasoning, and handling outdated information. While Retrieval-Augmented Generation (RAG) is a common solution to inject external contextual knowledge, a critical research gap exists:

• Knowledge Conflicts: LLMs frequently ignore retrieved context when it conflicts with their parametric memory (context-memory conflict).
• Intra-Memory Conflict: LLMs may also hold contradictory facts within their own parameters (intra-memory conflict), leading to instability.
• Lack of Understanding: It is unclear when and how LLMs choose to prioritize parametric knowledge over contextual knowledge, or vice versa. Existing attribution methods are computationally expensive or yield contradictory results, and synthetic datasets used to study this often fail to reflect real-world RAG scenarios.

2. Methodology

The keynote synthesizes research from the author's group, employing a multi-faceted approach involving attribution analysis, diagnostic probing, and the creation of new benchmarks.

A. Unified Attribution Framework (Parametric Knowledge)

To understand which parametric knowledge drives predictions, the authors propose a unified framework comparing Instance Attribution (identifying influential training data) and Neuron Attribution (identifying influential neurons).

• Technique: They align influential training instances with important neurons, specifically analyzing MLP classification layers (due to computational efficiency and high correlation with full weights).
• Evaluation: They use faithfulness tests for sufficiency (activating key neurons) and completeness (suppressing key neurons).
• Fine-tuning Analysis: They test if fine-tuning on "influential" instances improves performance compared to random selection.

B. Diagnostic Probing for Knowledge Conflicts

Instead of expensive attribution, the authors use probing tests to measure two types of conflicts:

• Intra-Memory Conflict: Measured using Semantic Entropy (capturing variation in parametric memory).
• Context-Memory Conflict: Measured using a Coherent Persuasion Score (approximating the semantic shift in output distribution when competing context is introduced).
• Dataset: They utilize DynamicQA, a dataset containing static facts, temporal facts (changing over time), and disputable facts (viewpoint-dependent).

C. Real-World RAG Analysis (DRUID Dataset)

To address the gap between synthetic and real-world RAG, the authors introduce DRUID (a dataset for claim verification with real-world contexts) and a new metric, ACU (Actual Context Usage).

• Comparison: They compare DRUID against prior synthetic datasets (e.g., CounterFact, ConflictQA) to analyze how context characteristics (assertiveness, source type, temporal alignment) influence LLM behavior.
• Context Manipulation Techniques (CMT): They study various methods to resolve conflicts, ranging from simple prompting to mechanistic interventions.

3. Key Contributions

1. Unified Attribution Framework: A comprehensive method to compare Instance and Neuron attribution, revealing that they often provide conflicting explanations for model predictions.
2. DynamicQA & Conflict Metrics: A novel dataset and metric suite to quantify intra-memory and context-memory conflicts, moving beyond static fact-checking.
3. DRUID Dataset & ACU Metric: The first dataset designed to measure context usage in realistic, real-world RAG scenarios, accompanied by the ACU metric to quantify how much an LLM actually relies on retrieved context.
4. Context Manipulation Benchmark: A systematic evaluation of techniques to force LLMs to utilize contextual knowledge over parametric memory.

4. Key Results & Findings

On Parametric Knowledge & Attribution

• Neuron Redundancy: Most neurons in the MLP layers can be removed without significantly altering predictions, contradicting the assumption that knowledge is densely packed in specific neurons. The authors hypothesize attention weights play a larger role in encoding knowledge.
• Ineffective Fine-tuning: Fine-tuning on "influential" instances identified by attribution methods yields performance similar to random selection. This is attributed to the low diversity of influential instances compared to random samples.
• Artifact Detection: Combining Instance and Neuron attribution is highly effective at detecting dataset artifacts (e.g., models overfitting to specific lexical patterns).

On Knowledge Conflicts

• Counter-Intuitive Persuasion: LLMs are more easily persuaded by manipulated context regarding static facts (low dynamicity) than dynamic facts (temporal/disputable).
  • Implication: Facts that change regularly are less likely to be updated via retrieval, while static facts are easily overridden, even if the context is false.
• Dynamicity as a Predictor: Fact dynamicity is a stronger negative indicator of model persuasion than fact popularity.
• Intra-Memory Instability: Contradictory representations within training data cause significant uncertainty in LLM outputs.

On Real-World RAG (DRUID)

• Context Characteristics: Context from fact-checking sources leads to higher usage due to assertive language. Evidence published after a claim increases usage.
• Query-Context Similarity: LLMs prioritize contexts with high similarity to the query, which is difficult to achieve in real-world retrieval.
• Synthetic vs. Real: Synthetic datasets oversell the impact of knowledge conflicts and exaggerate "context repulsion." Real-world data shows these conflicts are rarer, and no single characteristic predicts RAG failure.
• Model Size: Larger LLMs generally use context better, but smaller models can outperform them if paired with the optimal Context Manipulation Technique (CMT).

5. Significance

• Paradigm Shift in Evaluation: The work argues that current evaluations relying on synthetic data are misleading. It calls for a shift toward real-world aligned studies to understand RAG behavior accurately.
• Rethinking Factuality: The finding that LLMs are better at "recitation" than "reasoning" and are easily swayed by static context suggests that improving factuality requires more than just better retrieval; it requires understanding the nature of the conflict.
• Future Directions: The research highlights the need for new measures of intra-memory conflict and better indicators for successful context usage, particularly in low-certainty domains. It emphasizes that research in this field is not circular but "ascending a spiral," where old ideas (like recitation vs. reasoning) reappear in new forms (LLMs) requiring new solutions.

This keynote serves as a critical reality check for the NLP and IR communities, urging a move away from idealized synthetic benchmarks toward rigorous, real-world analysis of how LLMs integrate internal and external knowledge.