💻 computer science

Hallucination Span Detection with Input-Side Evidence Alignment

This paper introduces a novel task and an encoder-based method for detecting hallucinated spans in LLM-generated text by leveraging the predictability of faithful tokens from input evidence to simultaneously identify hallucinations and align them with their source.

Original authors: Miyu Yamada, Yuki Arase

Published 2026-08-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Miyu Yamada, Yuki Arase

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Large language models have transformed how we interact with computers, acting as powerful engines that can summarize news, answer complex questions, and draft documents. These systems work by predicting the next word in a sentence based on patterns they have learned from vast amounts of text. However, a persistent flaw undermines their reliability: they sometimes invent facts. This phenomenon, known as hallucination, occurs when a model generates information that sounds plausible but is either unsupported by the source material or directly contradicts it. For users relying on these tools for critical tasks, distinguishing between a helpful summary and a fabricated story is difficult. The core challenge lies not just in knowing that an error exists, but in pinpointing exactly which words are wrong and finding the specific part of the original text that proves they are wrong. Without this granular level of detail, verifying the truthfulness of a long document remains a tedious, manual process.

Researchers at the Institute of Science Tokyo have developed a new approach to solve this problem, shifting the focus from judging an entire text to examining individual words. Instead of asking a computer to simply label a whole paragraph as true or false, their method acts like a magnifying glass, scanning the generated output one word at a time to see if it can be traced back to the source. The team operates on a simple but powerful observation: if a word in the summary is faithful to the original text, a computer should be able to predict it using only the source material as a guide. Conversely, if a word is a hallucination, it will be impossible to predict from the source because the information simply does not exist there. By training a model to test this predictability, the researchers created a system that not only flags the invented words but also highlights the specific evidence in the input that supports the correct information.

The process begins by breaking the generated text into meaningful chunks, such as phrases or clauses, rather than treating it as a single block. The system then takes a word from one of these chunks and temporarily hides it, asking the model to guess what that word should be based solely on the original input text. If the model can confidently guess the hidden word using the source, the system marks it as faithful. If the model struggles to make a guess, or if the confidence score is low, the system flags the word as a potential hallucination. Crucially, because the model had to look at the source text to make its guess, it naturally produces a link between the output word and the specific part of the input that helped it. This creates a map of evidence, showing the user exactly where the model found its information or, in the case of an error, where it failed to find any support.

To test this idea, the researchers used a dataset containing thousands of examples where humans had already identified which parts of generated answers were factually incorrect. They compared their new method against existing techniques, including those that rely on massive, complex models to read and judge text. The results showed that their lighter, more focused approach was exceptionally good at finding the errors. While other methods sometimes missed the mistakes entirely, this new system successfully identified the vast majority of hallucinated words, particularly those that were completely made up. It also proved effective at linking the correct words in the summary back to the right sentences in the source, providing a clear trail of evidence for the user to follow.

However, the study also revealed a specific weakness in the system's ability to detect certain types of errors. The method struggled when the hallucination was a direct contradiction, such as changing a number from "four" to "twenty" in a way that still sounded grammatically and contextually similar to the original text. In these cases, the model found the surrounding context so similar that it remained confident in its prediction, failing to recognize the subtle shift in meaning. The researchers noted that while the system excels at spotting information that is entirely absent from the source, it needs further refinement to catch these more deceptive, conflicting errors. Despite this limitation, the work demonstrates a significant step forward in making artificial intelligence more transparent, offering a practical way to verify the truthfulness of machine-generated content by connecting every claim directly back to its source.

Technical Summary: Hallucination Span Detection with Input-Side Evidence Alignment

Problem Statement

Large Language Models (LLMs) have advanced conditional text generation tasks such as summarization and question answering but remain prone to hallucinations—generating content unsupported by or contradictory to the input text. Existing detection methods primarily treat hallucination as a binary classification problem at the sentence or document level. While effective for estimating overall factuality, these approaches fail to localize specific hallucinated spans or identify the corresponding input evidence. This limitation hinders the interpretability of model predictions and complicates the verification of generated content, particularly in long documents where manual tracing is difficult. Furthermore, obtaining manually aligned input-output pairs for training is expensive and impractical given the rapid evolution of LLMs.

Methodology

The authors propose a novel task: hallucination span detection with input-side evidence alignment. This task jointly identifies hallucinated spans in generated text and aligns output tokens with their corresponding evidence in the input. The approach is grounded in the observation that faithful output tokens are generally predictable from their supporting context in the input, whereas hallucinated tokens are not.

Core Framework

The method formulates hallucination detection as an input-grounded masked token prediction problem, inspired by Non-Parametric Masked Language Models (NPM). Instead of relying on large decoder-based LLMs, the system employs a lightweight encoder-based model.

Training Strategy (Distant Supervision):
- Span Segmentation: Output texts are segmented into semantically meaningful spans using Semantic Role Labeling (SRL). Predicates and their arguments are extracted to form spans; words not part of any predicate or argument (e.g., conjunctions) are excluded.
- Masked Prediction: During training, spans are masked with special tokens (<mask><mask>). The model is trained to predict these masked spans using only the input text representation.
- Confidence Metric: The prediction confidence for a masked span $s$ against an input span $(i, j)$ is defined as the sum of cosine similarities between the masked token embeddings and the input token embeddings:
  $c(s, i, j) := \text{sim}(m_s, d_i) + \text{sim}(m_e, d_j)$
- Loss Function: A custom loss function encourages high confidence for faithful spans and low confidence for hallucinated spans. It penalizes hallucinated spans that achieve high confidence and faithful spans that achieve low confidence, with additional weighting for "hard" examples.
Inference:
- Unlike training, inference operates at the token level. Each output token is masked individually, and the model retrieves the most relevant input token based on the confidence score.
- Classification: An output token is classified as hallucinated if its maximum confidence score falls below a threshold $\lambda_h$ .
- Alignment: The input token yielding the highest confidence score serves as the evidence alignment.
- Aggregation: Token-level predictions are aggregated to the word level; a word is deemed hallucinated only if all its subword tokens are classified as such.

Key Contributions

Task Definition: Introduction of a new task that jointly detects hallucinated spans and aligns them with input evidence, moving beyond binary sentence-level classification.
Efficient Architecture: Development of a lightweight encoder-based method that avoids the computational cost of large decoder-based LLMs and does not require manually aligned training data (utilizing distant supervision).
Dual Output: The framework simultaneously produces hallucination detection labels and token-level evidence alignments.

Experimental Results

The method was evaluated on the RAGTruth dataset, covering Question Answering (QA) and News Summarization tasks.

Automatic Evaluation:
- The proposed method achieved the highest recall on both QA and summarization tasks (82.7% and 38.0%, respectively), outperforming baselines like Llama-SFT and LettuceDetect in recall, which is critical for scenarios where missing hallucinations is unacceptable.
- While precision was lower than some baselines (e.g., Llama-SFT on summarization), the high recall indicates the method effectively identifies the majority of hallucinated content.
- Baseline OTAlign performed poorly, likely due to the difficulty of aligning sentences of significantly different lengths.
Human Evaluation:
- Human annotators assessed the quality of predicted input-side evidence alignments.
- For faithful and baseless hallucination words, the proposed method achieved average alignment scores around 2.0 (on a 0-3 scale), indicating the retrieved input tokens were relevant evidence.
- Conflict Hallucinations: The method struggled with conflict hallucinations (where the output contradicts the input), often assigning high confidence scores to these tokens. This is attributed to the high semantic similarity between the conflicting input and output contexts, which confuses the similarity-based confidence estimation.

Significance and Limitations

The paper claims that the proposed method effectively detects hallucinated spans while identifying meaningful input-side evidence, confirmed by human evaluation of alignment quality. The approach offers a practical solution for verifying generated content without the prohibitive cost of manual alignment data.

However, the authors acknowledge specific limitations:

Conflict Detection: The method is less effective at detecting "conflict" hallucinations compared to "baseless" ones, as the high similarity between conflicting text and input leads to false confidence.
Inference Efficiency: The token-by-token masking and re-encoding process results in linear inference costs relative to output length, which may be computationally expensive for long texts.
Scope: Experiments were limited to QA and news summarization using the RAGTruth benchmark; generalizability to other domains, languages, or LLM families remains uninvestigated.

The authors conclude that while the method is effective for baseless hallucinations, future work should focus on developing more sophisticated confidence estimation methods to better handle conflict hallucinations and improve inference efficiency.