💻 computer science

Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking

This paper introduces \textsc{PAVE}, a diagnostic testbed that evaluates how LLM verifiers in RAG-based fact-checking arbitrate between their pre-evidence parametric knowledge and retrieved evidence, revealing inconsistent arbitration behaviors across models and proposing a lightweight JSD-based method to improve factual reliability without model modification.

Original authors: Yuxi Sun, Wenbo Shang, Wei Gao, Xin Huang, Jing Ma

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuxi Sun, Wenbo Shang, Wei Gao, Xin Huang, Jing Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read friend (the LLM) who loves to fact-check news stories. You give them a claim, like "The Eiffel Tower is in London," and they usually know the answer is "False" because they've read about it a thousand times.

But now, imagine you also hand them a piece of paper (the Retrieved Evidence) that says, "Actually, the Eiffel Tower is in London, according to this new report."

This creates a conflict. Your friend has to decide: Do they trust their own memory (their Internal Prior), or do they trust the new paper you handed them (the External Context)?

This paper, titled "Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States," is basically a stress test to see how different AI friends handle this specific kind of tug-of-war.

Here is the breakdown of their findings using simple analogies:

1. The Problem: The "Stubborn" vs. The "Gullible"

The researchers noticed that standard tests only look at the final answer: "Did the AI get it right?" But they missed how the AI got there.

The Scenario: Sometimes the AI knows the truth but gets tricked by a fake news article. Other times, the AI is confused or wrong, but the fake news article makes it even more confident in the wrong answer.
The Gap: Existing tests didn't measure if the AI was being stubborn (ignoring good evidence because it was too sure of itself) or gullible (ignoring its own knowledge because it was too easily swayed).

2. The Solution: PAVE (The "Epistemic State" Test)

The authors created a new testing ground called PAVE. Think of this as a "personality test" for AI before they even see the evidence.

They categorize the AI's mindset into four "states" based on two questions:

Do they know the answer? (Are they right or wrong based on their own memory?)
Are they confident? (Are they sure, or are they guessing?)

This creates four personality types:

The Confident Expert: Knows the answer and is right. (e.g., "I know Paris is in France.")
The Confident Fool: Knows the answer but is wrong. (e.g., "I am 100% sure Paris is in Germany.")
The Uncertain Expert: Doesn't know they know, but is actually right. (e.g., "I'm not sure, but I think Paris is in France.")
The Uncertain Fool: Doesn't know and is wrong. (e.g., "I have no idea, but I'll guess Paris is in Germany.")

3. The Experiment: The "Tug-of-War"

The researchers took 7 different AI models (from small ones like Llama-8B to huge ones like Deepseek-v3) and gave them claims where their internal memory clashed with the evidence they were given.

They measured two main behaviors:

Persistence: If the AI was right to begin with, did it stick to its guns when shown a fake article? (Good!)
Correction: If the AI was wrong to begin with, did it admit its mistake when shown the real article? (Good!)

4. What They Found (The Results)

The results were surprising and showed that not all AIs are built the same:

The "Stubborn" Models: Some models (like Llama-8B) were very stubborn. Even when they were wrong, they refused to change their minds when shown the correct evidence. They preferred their own (incorrect) memory over the truth.
The "Gullible" Models: Some models were too easily swayed. If you showed them a fake article, they would immediately drop their correct knowledge and believe the fake one.
The "Balanced" Models: The best performer was Deepseek-v3. It was like a wise judge: it stuck to the truth when it knew the truth, but it was willing to change its mind when it was wrong and saw better evidence.
Size Matters: Generally, bigger models (like the 70-billion parameter versions) were better at this balancing act than smaller ones. They were less likely to be stubborn or gullible.
New vs. Old Knowledge: The AIs were much better at learning new things (like a news event from 2025) than they were at admitting they were wrong about something they thought they knew. It's easier for them to say, "Oh, I didn't know that!" than to say, "Oh, I was wrong about that."

5. The Fix: A "Second Opinion" Trick

Since some AIs are naturally bad at this balancing act, the authors proposed a simple, lightweight trick to fix it without retraining the whole model.

The Analogy: Imagine asking your friend to answer a question, but instead of asking once, you ask them five times in a row.

If they give the same answer every time, they are stable.
If they flip-flop between answers, they are unstable.

The researchers use a math formula (called JSD) to measure this stability.

If the AI is stable in its own memory but unstable when reading the evidence, the trick tells the AI: "Trust your memory."
If the AI is unstable in its memory but stable when reading the evidence, the trick tells the AI: "Trust the evidence."

The Result: This simple "ask five times" trick made almost all the AI models better at fact-checking, helping them avoid being both stubborn and gullible.

Summary

This paper is a diagnostic tool that reveals that AI fact-checkers have distinct "personalities" when it comes to trusting themselves vs. trusting new information. Some are too stubborn, some are too gullible, and some are just right. The authors also found a simple way to make even the "stubborn" or "gullible" ones act more like the "wise judge" by checking how consistent their answers are before making a final decision.

Technical Summary: Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking

1. Problem Statement

In Retrieval-Augmented Generation (RAG) based fact-checking, Large Language Models (LLMs) act as verifiers that must reconcile their internal parametric knowledge with retrieved external evidence. A critical, underexplored challenge arises when these two sources disagree, a condition termed Prior-Context Discrepancy (PCD). This occurs when retrieved evidence is noisy, temporally outdated, or counterfactual, conflicting with the model's pre-existing beliefs.

Existing evaluation frameworks primarily measure final verdict accuracy on clean, gold-standard evidence pairs. Consequently, they fail to:

Characterize the verifier's pre-evidence epistemic state (i.e., whether the model is confidently correct, confidently wrong, uncertain but correct, or uncertain and wrong).
Measure how verifiers arbitrate between conflicting signals (e.g., whether they stubbornly persist in a wrong prior or correctly update based on accurate evidence).
Distinguish between different types of failures, such as a model that is confidently wrong versus one that is uncertain.

2. Methodology: PAVE Framework

The authors introduce PAVE (Prior-Aware Verifier Evaluation), a diagnostic testbed designed to stratify LLM behavior based on epistemic states and measure arbitration dynamics under PCD.

A. Epistemic State Characterization (Knowledge Boundary)

PAVE defines a verifier's pre-evidence state using a Knowledge Boundary (KB) framework, operationalized along two axes:

Factual Correctness: Does the model's prior-only prediction match the ground truth?
Parametric Confidence: Is the model's prediction consistent across repeated stochastic generations? (High consistency = "KNOWN"; Low consistency = "UNKNOWN").

This yields four distinct epistemic states:

Known-Knows (KK): Confident and Correct.
Known-Unknows (KU): Confident but Incorrect.
Unknown-Knows (UK): Uncertain but Correct.
Unknown-Unknows (UU): Uncertain and Incorrect.

B. Benchmark Construction

The authors constructed a PCD benchmark of over 7,000 claims across two dimensions:

Counterfactual Setting: Uses controlled transformations (entity substitution, semantic negation) on gold evidence to create misleading context. This tests Persistence (resisting misleading evidence when the prior is correct) and Correction (updating from accurate evidence when the prior is wrong).
Temporal Setting: Uses claims about events occurring after the model's training cutoff. This tests Acquisition (updating from new evidence when the prior is outdated/missing).

C. Evaluation Metrics

PAVE introduces state-conditioned metrics to quantify arbitration behavior:

Correction Rate (CR): The ability to rectify initial errors when provided with correct evidence (measured on $D_{wrong}$ ).
Persistence Rate (PR): The ability to maintain correct predictions when exposed to misleading counterfactual evidence (measured on $D_{correct}$ ).
Margin: A differential metric defined as $CR + PR - 1$. A positive margin indicates the model is more likely to be corrected by accurate evidence than misled by inaccurate context.
Odds of Influence (OI): Measures the relative elasticity of a model's confidence, comparing the odds of changing an answer versus maintaining it under different epistemic states.

D. Test-Time Arbitration Method

Based on PAVE's findings, the authors propose a lightweight, training-free arbitration method using Jensen-Shannon Divergence (JSD):

The method samples $K$ predictions for a claim under two conditions: with external evidence ( $p_{ext}$ ) and without ( $p_{par}$ ).
It calculates the stability (consistency) of these predictions using JSD.
Decision Rule: If the JSD difference ( $\Delta = JSD_{par} - JSD_{ext}$ ) falls below a calibrated threshold $\tau$ , the model trusts its internal prior; otherwise, it trusts the external evidence. This allows the system to dynamically select the more reliable source without retraining.

3. Key Results

The authors evaluated seven LLMs (ranging from 7B to 617B parameters, including GPT-4o-mini, Llama3, Deepseek-v3, etc.) using the PAVE framework.

Heterogeneous Epistemic States: Models exhibit significant variation in their internal confidence profiles. For instance, Mistral-7B and Phi-4 show high uncertainty (many "UNKNOWN" states), while others are highly confident.
Prior Accuracy $\neq$ Robust Arbitration: High prior-only accuracy does not guarantee robustness under PCD. For example, Qwen3-32B had high prior accuracy but suffered a massive performance drop (43.02 points) under PCD, whereas Deepseek-v3 showed a much smaller drop (19.11 points) and a positive Margin (+0.235).
Model Selection is Critical: Models like Deepseek-v3 and Gemini-2.5-flash demonstrated a desirable balance (high persistence when correct, high correction when wrong). In contrast, models like Llama3-8B and Mistral-7B were "stubborn," frequently favoring erroneous internal priors over accurate external evidence.
Scaling Effects: Larger models (e.g., Llama3-70B) demonstrated better arbitration calibration, achieving higher Margins than smaller variants (1B, 3B), suggesting scaling improves the ability to resolve discrepancies.
Evidence Uptake Patterns:
- Temporal vs. Counterfactual: Models are generally more receptive to acquiring new knowledge (temporal setting) than revising wrong beliefs (counterfactual setting).
- Label Bias: Models show higher internal confidence in refuting false claims than confirming true ones.
Arbitration Method Performance: The proposed JSD-based test-time method achieved the best or competitive overall accuracy in 5 out of 6 evaluated LLM families. It was particularly effective for smaller models (e.g., Mistral-7B), outperforming the second-best baseline by 6.58 absolute points. The method remains effective even with a low computational budget ( $K=3$ runs).

4. Significance and Contributions

The paper claims four primary contributions:

Formulation: It reframes LLM fact-checking under PCD as an epistemic-conditioned arbitration problem, shifting the evaluation focus from final verdict accuracy to the dynamics of balancing internal priors and external evidence.
Diagnostic Testbed (PAVE): It introduces a framework including a controlled benchmark for inducing PCD, a KB stratification for pre-evidence states, and state-conditioned metrics (CR, PR, Margin, OI) to diagnose specific arbitration behaviors.
Systematic Evaluation: It provides a comprehensive analysis of seven LLMs, revealing distinct arbitration profiles and offering practical guidance for model selection in RAG-based systems.
Practical Intervention: It proposes a lightweight, JSD-based test-time arbitration method that improves factual reliability without modifying the underlying model, demonstrating that diagnostic signals can be operationalized for immediate deployment.

The authors emphasize that their work highlights the limitations of standard accuracy metrics and the necessity of understanding how a model reaches a verdict, particularly when sources conflict. They acknowledge limitations regarding the scope (binary fact-checking, excluding "both sources incorrect" scenarios) and the potential for generation artifacts in counterfactual construction, but assert that the core findings regarding arbitration profiles remain stable across configurations.