💬 NLP

Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization

This paper extends layer-resolved optimal transport analysis of cross-attention distributions to detect hallucinations in neural machine translation and abstractive summarization, demonstrating its effectiveness in identifying source disengagement while highlighting its inherent limitations in detecting faithfulness failures where attention remains correct but content is misrepresented.

Original authors: Mariia Onyshchuk, Maksym-Vasyl Tarnavskyi, Marta Sumyk

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mariia Onyshchuk, Maksym-Vasyl Tarnavskyi, Marta Sumyk

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot translator (or a summarizer) that reads a story and then writes its own version of it. Sometimes, this robot gets it right. Other times, it "hallucinates"—it makes up facts, repeats itself, or ignores the original story entirely, even though the new text sounds fluent and confident.

This paper is like a detective trying to figure out how to catch the robot lying just by watching how its eyes move while it reads.

Here is the breakdown of their investigation using simple analogies:

1. The Detective's Tool: "Optimal Transport"

The researchers use a mathematical tool called Optimal Transport (OT).

The Analogy: Imagine the robot's attention is like a spotlight shining on the original text.
- Good behavior: The spotlight moves around, scanning different parts of the text as the robot writes, checking facts and context.
- Bad behavior (Hallucination): The spotlight gets stuck on one spot (or a few random spots) and refuses to move, even though the robot is writing about something completely different.
The Measurement: The researchers measure the "distance" between where the spotlight should be (spread out and moving) and where it actually is. If the distance is weird, they know the robot is likely hallucinating.

2. The First Case Study: Machine Translation (DE-EN)

They first tested this on a model translating German to English. They looked at the robot's "brain" layer by layer (like looking at different floors of a skyscraper).

The Layers: They found that the robot's brain isn't uniform.
- Floors 1–4: These are the "detective floors." This is where the robot checks if it's paying attention to the right words. If the robot is lying, these floors show a very clear pattern of the spotlight getting stuck.
- Floor 5: This is the "final boss" floor. Interestingly, for some types of lies, this floor actually gets worse at detecting them. It's like the final floor is so focused on finishing the sentence that it stops caring about whether the facts are right.
- The "Exploration" Phase: When a robot translates correctly, it starts by "scanning" the text (moving the spotlight around) before locking onto specific words. When it hallucinates, it skips the scanning phase entirely. It locks onto a wrong word immediately from the very first step.

The Verdict on Translation: This method is a superhero at catching "full lies" (where the robot ignores the source text completely). It catches these with high accuracy. However, it struggles with "partial lies" (where the robot remembers some words but gets the meaning wrong).

3. The Second Case Study: Summarization (CNN/XSum)

Next, they tried to use the same "spotlight detector" on a different job: summarizing news articles.

The Problem: In translation, a lie usually means the robot ignores the source text. But in summarization, a lie can happen even if the robot is looking at the right words.
- The Analogy: Imagine a student reading a history book.
  - Translation Lie: The student closes the book and writes a story about dragons. (The spotlight is off the book).
  - Summarization Lie: The student reads the book perfectly but then writes, "The book says the war ended in 1990," when it actually ended in 1980. The student was looking at the right page, but misinterpreted the content.
The Result: The "spotlight detector" could still spot some lies (it was better than random guessing), but it wasn't nearly as good as a teacher who actually knows the facts (a supervised AI).
Why? Because the detector only checks if the robot is looking at the text. It cannot check if the robot is understanding the text correctly. If the robot looks at the right words but twists the meaning, the detector stays silent.

4. Mapping the Robot's Brain (T5 Model)

They also did a structural analysis of the summarization model (T5) to see how its brain is organized, even without looking for lies.

Layer 3: This is the "peak" layer. It's the most selective and unique part of the brain.
Layer 12: This is the "critical" layer. If you remove it, the robot's writing quality crashes.
Layers 9–11: These are like "backup copies." If you remove them, the robot doesn't really care; it still works fine.

The Big Takeaway

The paper concludes that this "spotlight" method is a powerful tool, but with a specific job description:

It is excellent at catching robots that stop paying attention to the source text entirely (source disengagement).
It is useful for understanding how a robot's brain is organized (which layers do what).
It is limited when the robot pays attention but gets the facts wrong (content misuse).

In short: You can tell if the robot is ignoring the book, but you can't always tell if the robot is misreading the book just by watching its eyes.

Technical Summary: Layer-Resolved Optimal Transport for Hallucination Detection in NMT and Abstractive Summarization

Problem Statement

Neural Machine Translation (NMT) and abstractive summarization models frequently suffer from "hallucinations" or "faithfulness failures," where generated outputs are fluent but factually inconsistent with the source. While recent work by Guerreiro et al. (2023) demonstrated that Optimal Transport (OT) distances between cross-attention distributions and a reference can detect NMT hallucinations in an unsupervised manner, several critical questions remained unresolved:

Layer Resolution: How does the hallucination signal distribute across individual decoder layers, and are specific layers more diagnostic than others?
Detector Specialization: Do different OT-based metrics (e.g., distance to uniform vs. distance to data) specialize in detecting different types of hallucinations?
Task Transfer: Does the geometric intuition of "source disengagement" (detected via OT in NMT) transfer to abstractive summarization, where failure modes may differ fundamentally?

Methodology

1. Optimal Transport Metrics

The authors extend the Wasserstein-1 ( $W_1$ ) distance framework to analyze cross-attention distributions ( $\pi^{(\ell,t)}$ ) across all decoder layers ( $\ell$ ) and generation steps ( $t$ ). Key metrics include:

Wass-to-Unif (WTU): Measures the concentration of attention by calculating the $W_1$ distance to a uniform distribution. High concentration (low distance) suggests the model is ignoring the source.
Wass-to-Data (WTD): Measures the similarity of a test sentence's attention distribution to a reference set of confirmed-correct translations.
Routing Consistency (RC): A novel metric quantifying how consistently the decoder attends to the same source position across generation steps. It is derived from the entropy of the argmax source positions over time.
Step-to-Step OT: Measures the dynamic shift of attention between consecutive decoding steps to capture the "exploratory" phase of generation.
Layer-Pairwise OT: Computes distances between average attention distributions of different layers to map functional organization.

2. Datasets and Models

NMT (Fairseq DE-EN): Re-analysis of the Guerreiro et al. (2023) corpus ( $N=3,414$ ), covering five hallucination categories (full-unsupport, strong-unsupport, repetitions, etc.).
Abstractive Summarization (AggreFact): Evaluation on CNN/DailyMail and XSum test sets ( $N=1,116$ ) using T5-base and Flan-T5-Large architectures.
Baselines: Comparison against supervised state-of-the-art (MiniCheck-Flan-T5-L) and random baselines.

Key Contributions

Layer-Resolved NMT Analysis: The study extends the aggregate last-layer analysis of Guerreiro et al. (2023) to all six decoder layers. It identifies that detection performance is concentrated in layers L1–L4, while L5 (the final layer) is often anti-predictive for subtle hallucination types.
Complementary Detectors: The paper demonstrates that WTU and WTD are specialized detectors. WTU excels at detecting "full-unsupport" hallucinations (complete source disengagement), while WTD is superior for "strong-unsupport" and "repetitions." The newly proposed Routing Consistency (RC) is identified as the strongest individual detector for full-unsupport hallucinations.
Dynamic Generation Patterns: The authors characterize normal decoding as a two-phase trajectory: an exploratory phase (high attention shift, low concentration) followed by a commitment phase. Hallucinated translations lack the exploratory phase, beginning in a statically concentrated state from the first decoding step.
Summarization Transfer and Limits: The first application of OT-based detection to abstractive summarization. While the unsupervised detector achieves above-chance performance (57.4% Balanced Accuracy), it significantly underperforms supervised baselines (72.1%).
Structural Interpretability: Through OT metrics and ablation studies on T5-base, the paper reveals a consistent decoder organization: Layer 3 acts as a structural peak for concentration and routing divergence, while Layer 12 is critical for generation quality. Layers L9–L11 show functional redundancy.

Results

NMT Detection Performance

Signal Strength: Hallucinated translations exhibit significantly higher mean concentration and lower step-to-step OT compared to correct translations ( $p < 0.001$ ).
Layer Specificity:
- Full-unsupport: Highly detectable in L1–L4 (AUROC > 0.93). RC peaks at L2 (AUROC 0.955).
- Strong-unsupport & Repetitions: Detection is weaker throughout. Notably, Layer 5 becomes anti-predictive (AUROC < 0.5) for these types, as the final layer is naturally the most concentrated regardless of translation quality.
Mechanism: Hallucinations are characterized by a "static" attention signature where the decoder locks onto irrelevant source tokens (or the same token) from the very first step, skipping the dynamic scanning phase observed in correct translations.

Abstractive Summarization Transfer

Performance Gap: The unsupervised OT detector achieves 57.4% Balanced Accuracy on AggreFact, compared to 72.1% for the supervised MiniCheck-Flan-T5-L.
Generalization: OT features do not generalize well across extractive (CNN) and abstractive (XSum) regimes when tested cross-dataset (44.2% BAcc), confirming that the signal is regime-dependent.
Structural Findings: In T5-base, Layer 3 shows the highest concentration peak, and ablation of Layers 1–3 causes a sharp collapse in ROUGE-L, identifying them as an indispensable early context-encoding block.

Significance and Claims

The paper posits that Optimal Transport on cross-attention is a reliable, unsupervised detector specifically when the failure mode is "source disengagement."

Principled Interpretability: The authors argue that OT provides a label-free, geometric lens to understand Transformer architecture, identifying functional specializations (e.g., L3 in T5, L2 in Fairseq) that align with ablation results.
Theoretical Limitation: The performance gap in summarization is attributed to a fundamental difference in failure modes. NMT hallucinations are predominantly intrinsic retrieval failures (ignoring the source), which OT detects well. Abstractive summarization failures are often content misuse (attending correctly to the source but misrepresenting the content), a failure mode "invisible to concentration-based OT metrics by construction."
Conclusion: The work establishes OT as a powerful tool for detecting when a model detaches from its source, offering a principled interpretability tool regardless of the task, while explicitly acknowledging its limitations when faithfulness failures occur downstream of the attention mechanism.