💻 computer science

Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents

This paper introduces a unified benchmark for span-level hallucination detection across diverse structured inputs like code and tool outputs, demonstrating that a fine-tuned Qwen3.5-2B model significantly outperforms existing detectors and zero-shot judges on these complex domains while maintaining competitiveness on natural-language RAG benchmarks.

Original authors: Ádám Kovács, Bowei He, Xue Liu, István Boros, Szilveszter Tóth, Gábor Recski

Published 2026-07-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ádám Kovács, Bowei He, Xue Liu, István Boros, Szilveszter Tóth, Gábor Recski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a very smart, fast-talking assistant to write a report for you. You give them a stack of reference books (the "context") and a specific question. They quickly type up an answer.

The problem? Sometimes, even the smartest assistants get a little creative. They might invent a fact, mix up a number, or quote a page that doesn't exist in your books. This is called a hallucination.

For a long time, researchers have built "fact-checkers" to catch these mistakes, but they mostly only worked when the assistant was writing about normal topics like history or science using plain text.

This paper introduces a new, much tougher fact-checker designed for the modern world, where assistants are also writing computer code, summarizing software tool logs, and reading structured documents like tables and manuals.

Here is a breakdown of what they did, using some everyday analogies:

1. The Problem: The "Code" and "Log" Blind Spot

Imagine your assistant is a mechanic.

Old Fact-Checkers: Great at checking if the mechanic said, "The car is red" when the car is actually blue.
The New Reality: The mechanic is now writing a complex repair manual (code) or reading a digital diagnostic screen (tool output). If the mechanic writes a single wrong command like turn_off_engine instead of turn_on_engine, the whole car might break. Or if they list a part number that doesn't exist, the order fails.
The Gap: Existing fact-checkers were like a human reading a novel; they didn't know how to spot a single wrong line in a computer program or a specific error in a software log. They couldn't tell the difference between a real technical term and a made-up one.

2. The Solution: A "Spot-the-Difference" Game

The authors built a massive new training ground (a benchmark) to teach computers how to be these specialized fact-checkers.

How they made the data: They started with perfect, correct answers (like a perfect repair manual). Then, they used a "hallucination injector" (think of it as a mischievous editor) to sneak in tiny, localized lies.
- Example: They changed a real function name set_device to a fake one set_active_device.
- They didn't just say "This answer is wrong." They marked the exact characters where the lie happened.
The Variety: They created over 74,000 examples covering:
- Code: Real GitHub software fixes.
- Tool Output: Logs from software tools (like error messages or search results).
- Structured Docs: Research papers, README files, and Wikipedia pages with tables and lists.
- Normal Text: Standard questions and answers (to make sure they didn't forget how to check normal text).

3. The New Detective: "LettuceDetect"

They trained a new AI model (a 2-billion parameter version of a model called Qwen) to act as the detective.

The Job: The detective looks at the Request, the Reference Books, and the Assistant's Answer. It must point a finger at the exact lie and say, "This specific word is made up," or "This number is wrong."
The Results:
- On Code and Tools: The new detective is a superhero. It caught 60% of the lies in code and tool logs.
- The Competition: The old "off-the-shelf" fact-checkers (like LettuceDetect-large) and even giant, smart AI judges (Zero-shot LLMs) only caught about 17% to 22% of the lies in code. They were essentially blind to technical errors.
- On Normal Text: The new detective is still very good at checking normal text (scoring similarly to the best existing systems), proving it didn't lose its general knowledge while learning to read code.

4. Why "Span-Level" Matters

The paper emphasizes that they don't just say "Reject this answer." They do Span-Level Detection.

Analogy: Imagine a student writes a 10-page essay. One sentence is a lie.
- Old Way: "Fail the whole essay." (Too harsh, the other 9 pages might be perfect).
- New Way: "Highlight the one sentence that is a lie." (Precise and helpful).
In code, this is critical. If a program has 100 lines and only one line is wrong, you don't want to throw away the whole program; you just want to fix that one line.

Summary

This paper presents a new, unified "truth detector" that can handle the messy, technical reality of modern AI assistants. It moves beyond just checking simple text to spotting tiny, dangerous errors in code, software logs, and structured documents.

Their new model, LettuceDetect-Qwen-2B, is significantly better at finding these technical lies than previous tools, especially when the assistant is writing code or reading software logs, while still remaining a top-tier fact-checker for normal language. They have released all their data and models so others can use this "spot-the-difference" game to build better, more reliable AI systems.

Technical Summary: Beyond Document Grounding

Problem Definition

Retrieval-augmented generation (RAG) systems are increasingly deployed in structured environments beyond natural language, including coding agents, developer tools, and research documentation. While existing hallucination detection benchmarks and detectors focus primarily on natural-language document evidence, they fail to address the unique verification challenges posed by structured inputs such as source code, tool output (e.g., test logs, command outputs), and structured documents (e.g., markdown, tables).

In these structured settings, a single unsupported substring—such as a fabricated method name, an incorrect field value, or a misreported tool observation—can alter program behavior or mislead a user, even if the remainder of the response is correct. Current evaluation frameworks lack a unified, span-level formulation that covers generated code, tool observations, and structured documents alongside traditional natural-language RAG tasks.

Methodology

Unified Benchmark Construction

The authors introduce a unified benchmark comprising 74,285 newly constructed examples across five sources, supplemented by converted examples from existing natural-language RAG datasets (RAGTruth and PsiloQA). The sources include:

Code: Derived from SWE-bench, featuring repository-grounded fixes.
Tool Output: Derived from Squeez, containing queries and verbose tool observations.
Structured Documents: ACL paper chunks, GitHub READMEs, and Wikipedia markdown.
Natural Language: RAGTruth and PsiloQA (14 languages).

Data Construction Pipeline:

Grounding: Start with a known-correct, grounded answer derived from the source (e.g., a gold SWE-bench patch).
Injection: Use a source-specific "injector" model (Gemma 4 31B for code; Qwen 3.6 35B for others) to propose localized, structured replacement edits. These edits introduce localized hallucinations categorized as Contradiction (wrong logic/values), Unsupported Addition (extra behavior), or Fabricated Reference (invented methods/identifiers).
Span Labeling: Apply edits deterministically to extract exact character offsets for the hallucinated spans, avoiding the ambiguity of diffing mixed natural-language/code answers.
Reference Grounding: To prevent penalizing correct answers that reference external libraries or sibling methods not present in truncated contexts, the pipeline appends resolved definitions from the repository's base commit and real API signatures from Context7.
Verification: The code-agent test set underwent human review and arbitration, retaining 2,015 examples after filtering for validity and boundary accuracy.

Model Architecture

Two detector families were trained on the unified dataset:

LettuceDetect-Qwen-2B: A fine-tuned Qwen3.5-2B generative detector with a 32,768-token context window. It outputs JSON containing character spans, categories, and subcategories.
LettuceDetect-mmBERT-base: A 307M-parameter token-classification encoder (based on mmBERT) trained with a maximum sequence length of 8,192 tokens.

Both models are trained using a data-agnostic prompt that defines the hallucination taxonomy and provides the request, context, and answer to verify.

Key Contributions

Task Formulation: A span-level task formulation for post-generation hallucination detection specifically designed for structured grounded-generation contexts (code, tool output, structured documents).
Unified Benchmark: A comprehensive dataset of 74,285 new examples covering diverse modalities, plus converted examples from established RAG benchmarks, enabling cross-domain evaluation.
Detector Performance: Demonstration that a fine-tuned generative detector (LettuceDetect-Qwen-2B) substantially outperforms off-the-shelf detectors and zero-shot LLM judges on code-agent and tool-output splits, while remaining competitive on natural-language RAG benchmarks.

Experimental Results

Main Findings

On the unified test set, LettuceDetect-Qwen-2B achieved a span-F1 of 0.689.

Code-Agent Performance: The model achieved a span-F1 of 0.602 on the code-agent split, significantly outperforming LettuceDetect-large (0.17) and the strongest zero-shot LLM judges evaluated (Nemotron-3-Ultra and gpt-oss-120b, both $\le$ 0.22).
Natural Language Performance: The model remained competitive on established benchmarks, achieving 81.8 example-F1 on RAGTruth and 0.724 IoU on English PsiloQA (surpassing the benchmark authors' fine-tuned mmBERT-base at 0.707).
Multilingual Capability: Across 14 languages in PsiloQA, the model reached a mean IoU of 0.689, outperforming the Qwen2.5-32B judge (0.383).

Comparison with Baselines

Zero-Shot Judges: Large zero-shot LLMs (e.g., Nemotron-3-Ultra, gpt-oss-120b) struggled with code and tool output, often flagging correct, newly written code as unsupported or failing to distinguish between valid edits and hallucinations.
Specialized Detectors: Existing detectors trained solely on natural language (e.g., LettuceDetect-large) performed poorly on structured data (0.17 span-F1 on code).
Encoder vs. Generator: The generative detector (Qwen-2B) consistently outperformed the token-classification encoder (mmBERT-base) across all sources (0.689 vs. 0.642 overall span-F1).

Significance and Claims

The paper claims that the proposed benchmark and detector address a critical gap in RAG evaluation: the lack of span-level verification for structured, non-natural-language outputs. The authors emphasize that:

Structured Verification is Distinct: Verification in code and tool output requires different reasoning than natural language, particularly regarding intent errors and structural validity.
Generative Detectors are Superior: Fine-tuned generative models with long context windows are better suited for this task than traditional encoders or zero-shot judges, particularly when dealing with long repository contexts and complex dependencies.
Unified Evaluation: A single model can effectively handle diverse modalities (code, tool output, documents) without sacrificing performance on natural-language tasks.

The authors note limitations, including the reliance on synthetic injection for most labels (though the code test set was human-reviewed) and the focus on final-answer verification rather than full agent trajectories. They conclude that their work provides a necessary foundation for developing robust verification systems for the next generation of grounded generation agents.