💬 NLP

Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents

This paper systematically evaluates the validity of citation URLs in large language models and deep research agents, revealing significant rates of hallucination and non-resolving links across domains, and demonstrates that an open-source tool called `urlhealth` can effectively reduce these errors by up to 79 times through agentic self-correction.

Original authors: Delip Rao, Eric Wong, Chris Callison-Burch

Published 2026-04-06

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Delip Rao, Eric Wong, Chris Callison-Burch

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are asking a very smart, well-read librarian (an AI) to write a research paper for you. You ask, "What are the latest breakthroughs in cancer research?" The librarian hands you a beautiful, detailed report. But here's the catch: at the bottom of every paragraph, there are little footnotes pointing to specific books or articles on the shelves that supposedly prove the librarian's claims.

This paper is like a team of detectives who went to the library to check if those footnotes actually lead to real books, or if the librarian just made them up.

Here is the story of their investigation, broken down simply:

1. The Problem: "Ghost Books"

The researchers found that while these AI librarians are amazing at writing, they sometimes suffer from "citation hallucinations." This means they invent URLs (web addresses) that look real but lead nowhere.

The Analogy: It's like the librarian pointing to a book on the shelf and saying, "Read page 42 of The History of Time Travel," but when you go to pull that book off the shelf, it's not there. The book never existed.
The Reality: They found that between 3% and 13% of the links these AIs provide are "ghosts"—they never existed in the first place. Another 5% to 18% are "dead links"—the book used to be there, but someone took it off the shelf (link rot).

2. The Suspects: "Deep Researchers" vs. "Quick Searchers"

The study looked at two types of AI:

The Quick Searchers: These are AIs that do a standard Google search and give you an answer.
The Deep Researchers: These are the new, fancy AIs that spend a lot of time "thinking," reading multiple pages, and writing long, detailed reports with hundreds of footnotes.

The Surprise: You might think the "Deep Researchers" are more careful because they work harder. But the study found the opposite! Because they try to write so much and cite so many sources, they actually make up more fake links than the quick searchers. It's like a student who tries to write a 50-page essay in one night; they are more likely to invent facts just to fill the pages.

3. The "Subject" Effect: Some Topics Are Trickier

The researchers noticed that the AIs struggle more with certain topics.

The Analogy: Imagine asking the librarian about "Business" (stable, predictable) vs. "Theology" or "Medicine" (fast-changing, complex).
The Result: The AIs were much more likely to invent fake links when talking about Theology or Medicine. Why? Because these fields are full of specialized, hard-to-find, or rapidly changing information. The AI gets confused and starts guessing. In contrast, for "Business" or "Architecture," the links were much more reliable.

4. The "More is Not Better" Rule

There was a myth that if an AI gives you more citations, the quality must be better. The study proved this wrong.

The Analogy: If a chef gives you a plate with 100 ingredients, it doesn't mean the meal is better. It just means there's a higher chance of finding a rotten tomato in the pile.
The Result: AIs that generated the most citations actually had the highest error rates. Quantity did not equal quality.

5. The Solution: The "URL Health Check" Tool

The researchers didn't just point out the problem; they built a tool to fix it. They created an open-source tool called urlhealth.

How it works: Imagine a "spell-checker" for links. Before you trust the AI's report, this tool runs a quick check on every single link.
- Is the link alive? (Green light)
- Is it a ghost? (Red light - never existed)
- Is it dead? (Yellow light - used to exist, now gone)
The Magic: When they let the AI use this tool to "self-correct" (check its own work), the number of fake links dropped dramatically—by 6 to 79 times! The AI went from having a messy, unreliable report to a clean, trustworthy one.

The Big Takeaway

This paper teaches us two main things:

Don't trust everything you read: Even the smartest AI can make up fake sources, especially when it's trying to be too helpful or too detailed.
Verification is key: We need tools that automatically check if a link is real before we trust the information. Just like you wouldn't buy a house without checking the deed, you shouldn't trust an AI report without checking the links.

The researchers have made their "URL Health Check" tool available for everyone to use, hoping to stop the spread of fake citations in the future.

1. Problem Statement

Large Language Models (LLMs) and "Deep Research Agents" increasingly provide inline citations (URLs) to support their generated claims. While users rely on these citations as evidence, there is no systematic measurement of their reliability. Previous incidents (e.g., legal sanctions, academic retractions) have highlighted the dangers of citation hallucination, where models fabricate URLs that never existed.

The core problem addressed is the lack of:

Systematic Characterization: Quantifying the prevalence of broken or hallucinated URLs across different providers, model types (search-augmented vs. deep research agents), and knowledge domains.
Differentiation: Distinguishing between link rot (real URLs that have gone offline) and fabrication (URLs that never existed).
Mitigation Tools: Reusable, automated tools to detect and correct these errors in real-world agent workflows.

2. Methodology

Datasets and Models

The study utilized two complementary datasets:

DRBench: 100 multilingual research queries (Chinese/English) covering finance, science, and tech. The authors analyzed outputs from 10 models (Google, OpenAI, Anthropic), including both Deep Research Agents (multi-step synthesis) and Search-Augmented LLMs (single-step search). Total URLs analyzed: 53,090.
ExpertQA: 2,177 expert-curated questions across 32 academic fields. Three models (Claude Sonnet 4.5, Gemini 2.5 Pro, GPT-5.1) were evaluated. Total URLs analyzed: 168,021.

Definitions and Classification

The authors established a strict taxonomy for URL validity:

Non-resolving URL: Returns an HTTP error (4xx/5xx) or fails to connect.
Hallucinated URL: A non-resolving URL with no archived snapshot in the Wayback Machine (indicating it likely never existed).
Stale URL: A non-resolving URL that does have a Wayback Machine snapshot (indicating link rot).
UNKNOWN: URLs returning ambiguous status codes (e.g., 403 bot-blocking, 429 rate-limiting) or connection errors.

Technical Pipeline

Extraction: URLs were extracted via regex (DRBench) or API parsing (ExpertQA).
Liveness Check: An HTTP HEAD request (falling back to GET on specific errors) was issued. A browser-like User-Agent was used to minimize bot-blocking.
Archive Verification: Non-resolving URLs were queried against the Wayback Machine API.
- No snapshot $\rightarrow$ Hallucinated.
- Snapshot exists $\rightarrow$ Stale.
Sensitivity Analysis: The authors excluded HTTP 403 responses (often bot-blocks) from "non-resolving" counts to avoid false positives, treating them as a lower bound for failure rates. A headless-browser audit confirmed that 89% of "UNKNOWN" URLs were actually live or blocked-but-accessible.

Mitigation Tool: `urlhealth`

The authors released urlhealth, an open-source Python tool (83 lines) that automates the classification of URLs into four categories: LIVE, DEAD, LIKELY_HALLUCINATED, and UNKNOWN. It was integrated into an agentic self-correction loop where models could iteratively verify and replace broken citations.

3. Key Results

RQ1: Prevalence of Failures

Hallucination Rates: Range from 3% to 13% across models.
Non-resolving Rates: Range from 5% to 18%.
Top Offender: gemini-2.5-pro-deepresearch had the highest hallucination rate (13.3%) despite high overall quality scores.
Best Performers: Search-augmented Claude models achieved the lowest hallucination rates (3.0–3.2%).

RQ2: Deep Research Agents vs. Search-Augmented LLMs

Volume vs. Quality: Deep research agents generate significantly more citations per query (e.g., 113 URLs/query for Gemini Deep Research) but suffer from higher hallucination rates (10.7% vs. 4.8% for search-augmented models).
Architecture Matters: The multi-step retrieval and synthesis process in deep agents appears to compound errors, leading to more fabrication. However, openai-deepresearch showed a contrasting pattern: high volume but low hallucination (3.5%), suggesting tighter coupling between retrieval and generation.

RQ3: Domain Variation

Citation reliability varies significantly by field.
Lowest Failure: Business (5.4%).
Highest Failure: Theology (11.4%) and Healthcare/Medicine (10.8% overall, but up to 17.4% for specific models).
Model Sensitivity: Per-model variation can be up to 4.3× (e.g., Claude Sonnet 4.5 ranged from 4.0% in Math to 17.4% in Healthcare).

RQ4: Fabrication vs. Link Rot

Diagnosis: The gap between "non-resolving" and "hallucinated" rates reveals the failure mode.
- Pure Fabrication: Some models (e.g., gpt-4.1, gpt-4o-search-preview) had 0% stale URLs. Every non-resolving URL they generated was a fabrication.
- Link Rot: Models like openai-deepresearch and Claude showed substantial "stale" fractions (e.g., 65% of non-resolving URLs were stale), indicating genuine retrieval that later failed due to link rot.

RQ5: Quantity vs. Quality

Generating more citations does not improve per-citation reliability. In fact, models generating more citations often had higher error rates per citation, suggesting a lack of selectivity in retrieving marginal sources.

RQ6: Mitigation via Self-Correction

Using urlhealth in an agentic self-correction loop reduced non-resolving URLs by 6× to 79×, bringing failure rates below 1% for all three models tested.
Tool Competence: Success depends on the model's ability to use tools. A smaller model (gpt-5-nano) failed to act on verification results, persisting with hallucinated URLs, while GPT-5.1 successfully corrected them.

4. Key Contributions

Systematic Characterization: The first large-scale study quantifying citation URL validity across commercial LLMs and deep research agents, revealing that 3–13% of citations are hallucinated even in retrieval-augmented settings.
Failure Taxonomy: A clear distinction between fabrication (never existed) and link rot (existed but died), showing that different models exhibit different failure profiles.
Open-Source Tooling: Release of urlhealth, a lightweight, model-agnostic tool for automated URL liveness checking and hallucination classification.
Mitigation Framework: Demonstration that agentic self-correction using verification tools can reduce citation errors to near-zero levels, provided the agent has sufficient tool-use competence.

5. Significance and Implications

Reliability of AI Research: The findings challenge the assumption that "search-augmented" or "deep research" modes guarantee factual grounding. Even advanced agents fabricate URLs at rates that could mislead users in high-stakes fields like law and medicine.
Domain Sensitivity: The variation in failure rates across academic fields suggests that citation reliability is not uniform; fields with rapidly changing web content (e.g., Medicine, Theology) are more vulnerable.
Future of Verification: The study argues that citation verification must be a standard part of the AI pipeline. Relying solely on the model's generation is insufficient; post-hoc verification is necessary.
Open Science: By releasing data and tools, the authors enable the community to build more robust citation verification systems and hold providers accountable for the reliability of their outputs.

Conclusion: The paper establishes that citation URL validity is measurable at scale and correctable in practice. While deep research agents offer depth, they currently suffer from higher hallucination rates, necessitating the integration of automated verification tools like urlhealth to ensure the integrity of AI-generated research.