Technical Summary: Benchmarking Legal Hallucination Detection

Problem Statement

The rapid adoption of Large Language Models (LLMs) in legal practice has transformed the fabrication of legal citations from a rare individual error into a systemic issue. While earlier assumptions suggested that newer models would hallucinate less or that court sanctions would deter negligent filers, empirical evidence indicates the opposite: the number of court filings containing fabricated citations is growing year-over-year, and hallucination rates in newer models (e.g., GPT-5.1) are not consistently decreasing compared to mid-2024 iterations.

This trend creates a compounding verification burden. Newer models generate more citations per document and draw from a broader, less canonical set of cases, making individual verification more difficult. This burden disproportionately affects pro se litigants, who lack access to commercial legal databases (e.g., Westlaw, LexisNexis) and are least equipped to detect hallucinations. Currently, verifying citations is a manual process performed by attorneys, clerks, or judges, consuming significant judicial resources. There is a critical gap in the literature: while prior research evaluates whether LLMs produce factually correct legal explanations or summaries, no prior benchmark evaluates the efficacy of AI systems in automatically verifying legal citations.

Methodology

1. Taxonomy of Legal Citation Hallucinations

The authors propose a taxonomy grounded in actual court filings, categorizing hallucinations into five distinct types:

• Non-existent citation: The citation points to a case that does not exist (e.g., altered reporter volume or page number).
• Case name mismatch: The reporter citation and case name refer to two different real cases.
• Incorrect pincite: The citation refers to the correct case, but the specific page number does not support the quoted proposition.
• Verbatim misquote: The quoted language does not appear exactly as written in the cited case (e.g., synonym substitution).
• Content misrepresentation: The cited case exists but does not support the legal proposition for which it is cited.

2. The LEPHANTOMCITE Dataset

To enable benchmarking, the authors constructed LEPHANTOMCITE, a dataset of 1,300 legal brief excerpts:

• 1,000 excerpts from real federal appellate briefs (pre-2022) with systematically injected hallucinations covering all five taxonomy categories.
• 300 entries from the "Large Legal Fiction" dataset (Dahl et al., 2024), manually verified and reformatted to provide realistic examples of content misrepresentation.
• The dataset contains 4,499 total citation instances, with 1,107 containing hallucinations.

3. Agentic Verification Framework

The study evaluates citation verification using an agentic system based on the Bayesian Optimal Experimental Design (BOED) framework (Zheng et al., 2026). Unlike non-agentic baselines that generate a single response, the agent maintains an explicit, language-based belief state updated after each action.

• Actions: The agent can perform legal database searches (CourtListener, Westlaw), web searches, and extract information.
• Process: The agent extracts citations, decides on information gathering strategies, and iteratively updates its belief state (correct, hallucinated, or pending) until a determination is made or the step budget is exhausted.
• Models Evaluated: Five models were tested in both agentic and non-agentic settings: GPT-5, GPT-OSS-120B, Qwen3-8B, Qwen3.5-27B, and Gemini-2.5-flash. A production-grade Claude Code agent (Opus 4.8) was also evaluated for comparison.

Key Results

Performance Metrics

The study measures performance using Precision, Recall, and F1 score, treating hallucination detection as a retrieval task with relaxed matching criteria.

• Agentic vs. Non-Agentic: Agentic frameworks consistently outperformed non-agentic baselines. For GPT-5, the agentic framework improved recall by 19.7% (from 63.1% to 82.8%).
• Best Overall Performance: GPT-5 in an agentic setting achieved the highest recall (82.8%) and F1 score (60.5%).
• Claude Code: The Claude Code agent achieved the highest precision (76.1%) and F1 score (68.8%), though with lower recall (62.8%) than GPT-5. It utilized tool calls more efficiently (average 15.4 calls per excerpt vs. GPT-5's higher volume).

Performance by Hallucination Type

• High Recall: Models generally performed well on detecting non-existent cases and case name mismatches.
• Low Recall: All models struggled significantly with incorrect pincites (recall as low as 18.2% for GPT-5). This is attributed to the fact that page number information is often only available in commercial databases (Westlaw/Lexis) and is absent from free public repositories like CourtListener.
• Content Misrepresentation: Performance varied by model strength; GPT-5 detected over 80% of these errors, while weaker models detected around 50%.

Error Analysis

• Information Access Limits: A significant portion of false positives occurred because models interpreted the absence of a case in free databases (CourtListener) as a hallucination, rather than a coverage gap.
• Resource Exhaustion: 25% of false negatives occurred because agents hit the 30-step limit before verifying all citations.
• Redundancy: Agents often wasted steps on redundant queries or re-issuing searches that had already reached a dead end.
• Unexpected Utility: The agent successfully identified over 10 human-made citation typos in pre-LLM briefs that were present in the original documents.

Significance and Claims

The paper argues that legal citation hallucination is not a temporary artifact of early LLMs but a persistent, growing challenge that will not self-correct as models improve. The authors claim the following contributions:

1. Empirical Evidence: Longitudinal data across eight ChatGPT generations shows that hallucination rates do not consistently decline, while the volume of hallucinated filings grows.
2. Benchmarking Infrastructure: The introduction of LEPHANTOMCITE and the associated taxonomy provides the first standardized dataset for evaluating and auditing automated legal citation checking tools.
3. Feasibility of Agentic Verification: The study demonstrates that agentic systems can significantly improve recall over static models, offering a structural path forward for reducing the verification burden on courts.
4. Policy Implications: The results highlight that information access is a binding constraint on verification performance. The inability of automated tools to verify pincites without commercial database access creates a disparity that disadvantages pro se litigants and the public. The authors suggest that improving public access to legal data (e.g., official pagination) is a necessary prerequisite for effective automated verification, rather than relying solely on model improvements or reactive sanctions.

The authors conclude that responsible AI deployment in law requires treating verification as a "first-class problem," providing a foundation for building and auditing the tools necessary to maintain the integrity of the judicial process.