LLM hallucinations in the wild: Large-scale evidence from non-existent citations

Technical Summary: LLM Hallucinations in the Wild

Problem Statement
Large Language Models (LLMs) are known to generate plausible but false information ("hallucinations"), yet the real-world magnitude and consequences of this phenomenon remain poorly understood. While hallucinations have been documented in simulated settings and anecdotal reports (e.g., government reports, court filings), there is a lack of systematic, large-scale evidence regarding their prevalence in deployed scientific knowledge production. Studying hallucinations at scale is difficult because they are typically embedded in unstructured text, requiring costly human evaluation or automated pipelines that may reintroduce model-specific biases. Scientific citations offer a unique opportunity for audit because they provide a traceable, binary ground truth: a cited reference either exists or it does not.

Methodology
The authors conducted a population-scale audit of 111 million references across 2.5 million papers from four major corpora: arXiv (mathematical, physical, computational sciences), bioRxiv (biological sciences), SSRN (social sciences, law, humanities), and PubMed Central (peer-reviewed biomedical literature).

1. Verification Pipeline:

   • Extraction & Parsing: References were extracted from raw manuscript files (LaTeX, PDF via GROBID, XML, or metadata).
   • Initial Matching: Parsed titles were queried against a local Elasticsearch index built from Semantic Scholar and OpenAlex. String similarity criteria determined valid matches.
   • Cleaning: Unmatched entries were routed through an LLM-based cleaning step (GPT-4o-mini) to exclude non-academic sources and refine title extraction, reducing the unmatched rate from ~4.9% to 1.54%.
   • Final Validation: A third-party Google Scholar API lookup was performed on remaining unmatched entries. A reference was deemed valid if Google Scholar returned a matching record or consistent evidence of the work being cited elsewhere.
   • Definition of Hallucination: The study defines "hallucinated citations" not as every unmatched reference, but as the excess of unmatched citations observed post-LLM adoption compared to a pre-LLM baseline (late 2022). This approach controls for ordinary bibliographic errors and formatting artifacts.

2. Analytical Framework:

   • Temporal Dynamics: Regression models were used to estimate the rate of unmatched citations over time, identifying the inflection point of hallucination emergence.
   • Distribution Analysis: The study examined the distribution of hallucinated citations across papers to test whether contamination is concentrated in a few "bad apple" manuscripts or diffusely embedded.
   • Author & Credit Analysis: The authors correlated hallucination rates with author productivity (prior publication counts), team size, and the characteristics of cited authors (productivity, citation impact, gender, and team hierarchy).
   • Safeguard Evaluation: The efficacy of existing quality controls was assessed by analyzing rejection rates on arXiv moderation and the persistence of hallucinations from preprints (bioRxiv) to published versions (PMC).

Key Results

• Rise in Hallucinations: A sharp increase in non-existent references began in mid-2024, approximately 18 months after the initial release of ChatGPT. By August 2025, hallucination rates reached 0.39% (arXiv), 0.21% (bioRxiv), 1.91% (SSRN), and 0.27% (PMC).
• Magnitude: Extrapolating these trends, the authors conservatively estimate 146,932 hallucinated citations were introduced across these four corpora in 2025 alone.
• Diffuse Contamination: Hallucinations are not concentrated in a few heavily contaminated papers. Instead, they are diffusely embedded; the increase is driven by a growing proportion of papers containing a small fraction (0–10%) of unmatched references, rather than a few papers with high rates of errors.
• Correlates of Production:
  • Author Experience: Authors citing hallucinated sources ("hallucination citers") had significantly fewer prior publications before 2023 compared to controls. However, by 2025, their productivity surged (1.33x to 3.13x relative increase), suggesting LLMs have lowered barriers to entry for less experienced researchers, leading to a high volume of output with embedded errors.
  • Team Size: Hallucination rates are inversely correlated with team size; solo-authored and small-team papers exhibit higher rates.
  • LLM Usage: There is a strong positive correlation (r = 0.441) between estimated LLM usage in writing and hallucination rates at both the subfield and paper levels.
• Bias in Credit Allocation:
  • Hallucinated citations disproportionately credit authors with high prior productivity and high citation impact.
  • There is a significant gender disparity: hallucinated citations are 7.6% more likely to credit authors with male names.
  • These biases extend to valid citations within the same papers, suggesting a broader reallocation of scientific credit toward prominent scholars.
• Failure of Safeguards:
  • Preprint Moderation: While arXiv moderators reject manuscripts with higher hallucination rates, an estimated 78.8% of non-existent citations still pass moderation.
  • Peer Review: 85.3% of hallucinated citations present in bioRxiv preprints persisted into their published versions in PubMed Central.
  • Database Contamination: Hallucinated references are increasingly appearing as standalone entries in Google Scholar, creating a feedback loop where non-existent works become part of the "ground truth" for future retrieval.

Significance and Claims
The paper claims to provide the first large-scale, cross-disciplinary audit of LLM hallucinations in real-world knowledge work. Its primary significance lies in shifting the discourse from individual carelessness to a systems-level concern. The findings demonstrate that:

1. Scale and Persistence: LLM hallucinations are infiltrating knowledge production at a massive scale and are persisting through preprint and publication stages, outpacing current safeguards.
2. Equity Implications: The errors are not random; they reinforce existing inequities by channeling credit toward established, male scholars while being disproportionately produced by early-career or less-experienced researchers.
3. Generalizability: Science represents a "best-case" domain for detection due to structured citations and rich indexing. The authors argue that if hallucinations are penetrating scientific literature despite these robust verification mechanisms, the problem is likely far more severe and harder to detect in less structured domains such as legal filings, policy reports, and journalism.
4. Feedback Loops: The accumulation of hallucinated citations in bibliometric databases threatens to erode the mechanisms used to detect them, creating a path-dependent degradation of scientific reliability.

The authors conclude that while automated verification tools offer hope for structured errors like citations, the broader challenge of detecting hallucinated claims in unstructured prose remains a critical, unsolved problem for the integrity of future scientific discovery and institutional decision-making.