🤖 machine learning

Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models

This paper introduces AuthorityBench, a large-scale multi-domain benchmark demonstrating that the mere presence of citations—regardless of their factual accuracy—significantly increases hallucination rates in large language models, with the most pronounced effects observed when fabricated citations accompany true claims.

Original authors: Aryan Khurana, Aravind Ramana RN, Dhruv Kumar

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aryan Khurana, Aravind Ramana RN, Dhruv Kumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are taking a trivia quiz. You know the answer is "Paris" because you've been there. But then, a teacher walks in, points to a fancy-looking book on the shelf, and says, "Actually, the capital is London. I read it in this prestigious journal."

Would you change your answer? Most humans would pause and think, "Wait, that book looks important. Maybe I'm wrong."

This paper, AuthorityBench, asks a similar question about Artificial Intelligence (AI). It wants to know: If an AI knows the truth, but someone gives it a "citation" (a reference to a source) that says the opposite, will the AI blindly trust the citation and forget the truth?

Here is the breakdown of their findings, using simple analogies.

The Experiment: A "Fake News" Test for AI

The researchers built a massive test called AuthorityBench. Think of it as a giant, 220,000-question quiz designed to trick AI models.

They set up a "2x2" game with four scenarios:

True Claim + Real Citation: The AI is right, and the source is real. (The easy mode).
False Claim + Fake Citation: The AI is wrong, and the source is fake. (The obvious trap).
False Claim + Real Citation: The AI is wrong, but the source is real. (The tricky trap).
True Claim + Fake Citation: This is the big discovery. The AI is right, but they give it a fake source that says it's wrong.

They tested this across four areas: General Knowledge, Science, Law, and Medicine. They also changed the "flavor" of the fake sources: some looked like they came from Nobel Prize winners (high prestige), others from unknown blogs (low prestige), and some had names from different countries.

The Big Surprise: The "Authority Trap"

The most shocking result is that citations act like a "magic spell" that makes AI hallucinate (lie).

The "Fake Source" Effect: When the AI knew the correct answer (e.g., "Paris is the capital"), but a fake citation appeared saying "London is the capital," the AI often changed its answer to London.
The Numbers: In the "General Knowledge" category, this happened 35% to 77% of the time. That means if you gave an AI a true fact and a fake reference, it was more likely to deny the truth than to stick with it.
The "Real Source" Effect: Even if the citation was real but the claim was false, the AI was still more likely to hallucinate than if there were no citation at all.

The Analogy: Imagine a student who knows the answer is "2+2=4." If you hand them a piece of paper that says "According to the famous Professor Smith, 2+2=5," the student stops trusting their own brain and writes down "5." The paper (the citation) overpowered the math.

What Didn't Matter?

The researchers thought certain things might make the AI trust the citation more, but they were surprised to find these factors had almost no effect:

Prestige: It didn't matter if the fake citation looked like it came from a top-tier university or a random blog. The AI was equally confused by both.
Author Identity: It didn't matter if the fake author had a name associated with a specific country or demographic. The AI didn't seem to care about who wrote it, only that something was written.
Model Size: You might think a "smarter" or bigger AI would be less easily tricked. But the study found that the biggest, most advanced models were just as likely to fall for the trap as the smaller ones.

The One Exception: The "Lawyer" AI

There was one domain where the AI didn't get tricked as easily: Law.

Why? The researchers suggest that legal text has a very specific, formal "language" (like a secret code). When the AI sees a legal citation, it seems to switch into a "scrutiny mode" and checks the facts more carefully, rather than blindly accepting the authority.
General Knowledge was the weakest link, where the AI was most easily fooled.

The "False Claim" Twist

The study also looked at what happens when the AI is already wrong (a false claim).

Some AIs got smarter: For models like Llama and Claude, seeing a citation (even a fake one) made them less likely to hallucinate on false claims. It was as if the citation made them say, "Hmm, I need to double-check this," and they corrected themselves.
Some AIs got dumber: For models like Gemma and Phi-4, the citation made them more likely to hallucinate on false claims. They blindly accepted the wrong info.

The Bottom Line

The paper concludes that adding a citation does not automatically make an AI more reliable. In fact, it often makes it less reliable.

If you are using an AI to help with facts, and you see it citing a source, do not assume it is telling the truth. The AI might be so impressed by the "authority" of the citation that it will happily throw away the actual truth. The study warns that for high-stakes fields (like medicine or law), we cannot assume that grounding an AI's answers in citations will fix its errors; sometimes, the citations themselves become the source of the error.

Technical Summary: AuthorityBench

Problem Statement

Large language models (LLMs) are increasingly deployed in citation-augmented settings, yet the specific influence of citation presence on model behavior, independent of factual content, remains poorly understood. While hallucination (the generation of plausible but factually incorrect content) is a well-documented failure mode, the role of "in-context authority signals"—specifically how models react to the apparent authority of a cited source versus the veracity of the claim itself—is underexplored. Existing benchmarks like TruthfulQA and HaluEval focus on response-level accuracy without isolating the causal role of authority signals. Furthermore, prior work such as FalseCite has only examined false claims paired with fabricated citations, failing to control for venue prestige, author identity, or the interaction between true claims and fabricated citations.

Methodology: AuthorityBench

The authors introduce AuthorityBench, a large-scale, multi-domain benchmark designed to isolate how citation-based authority signals shape epistemic behavior in LLMs.

Experimental Design

The core innovation is a fully balanced 2×2 factorial design that independently manipulates two variables:

Claim Veracity: True vs. False.
Citation Veracity: Real vs. Fabricated.

This design yields four primary conditions, plus a "no-citation" baseline for each claim. A critical novel condition is True Claim × Fabricated Citation (TC×FC), which tests whether models can be induced to deny correct facts under the pressure of a fabricated authority signal.

Dataset Construction

Scale: 220,564 prompts generated from 110,282 base claims.
Domains: Four distinct domains were utilized: General Knowledge (FEVER), Science (SciQ), Law (CaseHOLD), and Medicine (MedMCQA).
Prompt Variation: The benchmark employs 40 structurally diverse prompt templates (e.g., prefix, mid-sentence, suffix, footnote-style) to test structural sensitivity.
Controlled Variables:
- Venue Prestige: Citations are assigned to one of four prestige tiers (Tier 1 to Tier 4) based on rankings (SCImago, Washington and Lee, etc.).
- Author Demographics: For fabricated citations, author surnames are sampled from a country-coded dataset to analyze perceived demographic influence.
- Temporal Framing: Citation years span from 1980 to the present.
Citation Construction: Real citations are drawn from verifiable sources (MTEB SciFact, PubMedQA, CaseHOLD contexts). Fabricated citations are constructed from curated pools of author names and venues. For the General Knowledge domain (FEVER), where structured academic metadata is unavailable, citation metadata is back-filled from other domain pools and flagged for interpretation.

Evaluation

Models: Seven models were evaluated, including open-weight models (Gemma 3 4B, Llama 3.1 8B, Phi-4 Mini, Gemma 4 31B, DeepSeek V3.2) and proprietary models (Claude Haiku 4.5, GPT 5.4 mini). Open-weight models were tested on the full dataset; proprietary models were tested on a balanced 15K subset.
Judge: Outputs were evaluated using Qwen3-8B as a judge, supplied with ground truth labels and source metadata to minimize reliance on parametric knowledge. Human evaluation on a stratified sample confirmed high inter-annotator agreement (Cohen's $\kappa$ = 0.83).
Metrics: The primary metric is the hallucination rate (proportion of outputs labeled hallucinated). Effects are measured via lift (absolute percentage point difference from baseline) and Cohen's $d$ .

Key Results

1. Citation Presence Increases Hallucination

Across all seven models, the presence of any citation—whether real or fabricated—increases hallucination rates relative to a no-citation baseline. This contradicts the assumption that citations inherently ground model outputs.

2. The Critical TC×FC Condition

The most severe failure mode occurs when a fabricated citation accompanies a true claim (TC×FC).

In this condition, models are more likely to deny a correct fact than in any other experimental condition.
Hallucination rates in the General Knowledge domain under TC×FC reach 35% to 77%, depending on the model.
Lifts over the true-claim baseline range from +3.23 to +22.29 percentage points.
Notably, larger models (e.g., Gemma 4 31B, GPT 5.4 mini) are not more resistant than smaller ones; in some cases, they are highly susceptible.

3. Divergent Responses to False Claims

Models exhibit two distinct behaviors when presented with false claims:

Suppression: Models like Llama 3.1, Claude Haiku 4.5, and GPT 5.4 mini show reduced hallucination rates when a fabricated citation is present, suggesting the citation triggers heightened scrutiny.
Amplification: Models in the Gemma family and Phi-4 show increased hallucination rates on false claims when a citation is present, treating the citation as an authority endorsement.

4. Domain and Structural Sensitivity

Domain Vulnerability: The General Knowledge domain is the most vulnerable to citation-induced hallucination. Conversely, the Legal domain is uniformly robust, showing the smallest citation-induced lifts, likely due to distinctive linguistic registers and formal conventions that raise the model's evidentiary bar.
Venue Prestige: Contrary to expectations, venue prestige has no systematic impact. A fabricated citation from a low-tier journal carries the same weight as one from a high-tier journal (e.g., Nature) in terms of inducing hallucination.
Author Demographics: Perceived author identity (via country-coded surnames) shows negligible impact on hallucination rates.
Template Structure: Citation placement matters. Prefix placement ("According to...") is consistently the highest-risk format, while minimal salience formats are often the safest.

Significance and Claims

The paper claims that AuthorityBench provides the first rigorous, large-scale evidence that citation presence degrades factual accuracy, even on claims the model would otherwise answer correctly.

Epistemic Failure: The findings suggest that LLMs suffer from a failure of epistemic reasoning under authority pressure rather than a simple lack of factual knowledge. Models appear to internalize a heuristic where the presence of a citation overrides internal knowledge, leading to "citation-induced denial."
Implications for RAG: For Retrieval-Augmented Generation (RAG) systems, the presence of a citation does not guarantee reliability. In high-stakes domains, grounding outputs in cited sources may actually reduce accuracy if the model defers to the citation's authority over the truth of the claim.
Mitigation Strategy: The authors argue that mitigation requires models to treat citations as evidence to be evaluated, rather than authority to be deferred to. They emphasize that scaling model size does not inherently solve this susceptibility.

The paper concludes by making all datasets and evaluation code available to facilitate the development of models that can better distinguish between the authority of a source and the veracity of a claim.