VeriTrail: Closed-Domain Hallucination Detection with Traceability
The paper introduces VeriTrail, the first method and dataset designed to detect and trace closed-domain hallucinations in both single-step and multi-step generative processes by analyzing intermediate outputs to identify the origin of unfaithful content.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you hire a team of writers to summarize a massive library of books for you. You give them strict instructions: "Only use the facts from these books. Do not make anything up."
This is what we call a Closed-Domain task. The problem is, even with those instructions, the writers sometimes slip up. They might accidentally invent a detail, mix up a date, or claim something happened that never did. In the AI world, this is called hallucination.
For simple tasks (like summarizing one short article), it's easy to check the work. You just compare the final summary to the original article. If they match, you're good.
But what if the task is huge? What if the writers have to break the library down into chapters, summarize those, then summarize the summaries, and finally combine them into one giant report? This is a Multi-Step Process.
Here is the catch: If the final report has a lie in it, a simple check tells you that there is a lie, but it doesn't tell you where the lie started. Did the first writer make it up? Did the second writer misinterpret the first? Did the final editor combine two true facts in a way that created a false story?
VeriTrail is a new tool designed to solve this exact problem. Think of it as a detective with a time machine.
The Detective's Toolkit: How VeriTrail Works
Instead of just looking at the final report, VeriTrail traces the entire journey of the information. It treats the writing process like a family tree (or a map of a subway system).
- The Map (The DAG): Imagine every sentence written by every writer is a "stop" on a map. The original books are the starting stations (Roots). The final report is the last station. Every time a writer summarizes a chunk of text, they create a new stop connected to the old ones.
- The Claim: VeriTrail breaks the final report down into tiny, testable facts (like "Company X bought two startups in 2020").
- The Reverse Chase: When VeriTrail checks a fact, it doesn't just look at the final sentence. It asks: "Where did this sentence get its info?"
- It looks at the previous summary.
- It looks at the summaries before that.
- It keeps going backward until it hits the original book.
- The Evidence Trail: As it moves backward, it builds a "breadcrumb trail" of the exact sentences that support the fact. If the trail leads to a dead end or a contradiction, VeriTrail knows a hallucination happened.
The "Broken Chain" Analogy
Imagine you are playing a game of "Telephone" (or "Broken Telephone") with 100 people.
- Person 1 reads a sentence from a book.
- Person 2 summarizes what Person 1 said.
- Person 3 summarizes what Person 2 said.
- ...and so on, until Person 100 gives you the final message.
If Person 100 says, "The sky is green," and you ask, "Is that true?"
- Old Methods would just say: "No, the book says the sky is blue. You are wrong." (But they don't know who changed the color).
- VeriTrail walks backward through the line.
- It asks Person 100: "Who told you the sky was green?"
- Person 100 says: "Person 99."
- VeriTrail asks Person 99: "Did you say that?"
- Person 99 says: "No, I said 'The sky is grey,' but Person 100 misunderstood me."
- Verdict: The error happened at Step 100. The original book was fine; the mistake was introduced in the final step.
Why This Matters
In the real world, this is crucial.
- For a Doctor: If an AI summarizes medical research and says "Drug X cures Cancer," the doctor needs to know if that came from a reliable study (the root) or if it was a mistake made by the AI's middle layer of processing.
- For a Lawyer: If an AI summarizes court cases, the lawyer needs to know exactly which part of the process introduced a false precedent so they can fix it.
The New Datasets: FABLES+ and DiverseSumm+
To teach VeriTrail how to do this, the researchers had to build new training grounds.
- FABLES+ is like a giant library of books where they saved every single draft, every summary, and every edit made by the AI.
- DiverseSumm+ is like a massive news archive where the AI had to combine dozens of different articles to answer complex questions.
These datasets are special because they include every intermediate step. Before this, researchers only had the final answer to check. Now, they have the whole "cooking process," not just the final meal.
The Result
The paper shows that VeriTrail is much better at finding lies than previous methods.
- It catches more hallucinations.
- It tells you exactly where the lie started (Error Localization).
- It does this without costing a fortune, even though it's doing a lot more work than the simple checkers.
In short: VeriTrail doesn't just tell you the report is fake; it acts like a forensic accountant, tracing every dollar back to its source to find exactly who, when, and how the fraud happened. This makes AI much more trustworthy for complex, real-world jobs.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.