🤖 AI

TIGER: Traceable Inference with Graph-Based Evidence Routing for Mitigating Hallucinations in Multimodal Generation

TIGER is an inference-time framework that mitigates hallucinations in multimodal generation by independently constructing observation and claim graphs to identify and repair high-risk unsupported facts through a frozen backbone, thereby reducing ungrounded content while preserving task quality across various cross-modal tasks.

Original authors: Kaixiang Zhao, Tianrun Yu, Shawn Huang, Porter Jenkins, Yushun Dong, Amanda Hughes

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kaixiang Zhao, Tianrun Yu, Shawn Huang, Porter Jenkins, Yushun Dong, Amanda Hughes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very talented but slightly distracted artist (the AI model). You show them a photo of a quiet lake with two ducks and ask them to write a description. The artist writes a beautiful, fluent paragraph, but in the middle of it, they accidentally claim there are three ducks and that a boat is nearby, even though neither is in the picture.

This is called a "hallucination." The story sounds good, but the facts are wrong.

The paper introduces a new system called TIGER (Traceable Inference with Graph-Based Evidence Routing) to fix these mistakes without firing the artist or retraining them. Here is how it works, using simple analogies:

The Problem: The "Echo Chamber" Effect

Previous methods tried to fix these errors by asking the artist to "critique their own work." They would show the artist the photo and the wrong paragraph, then ask, "Does this look right?"

The paper argues this is a bad idea. Because the artist is looking at their own wrong paragraph while looking at the photo, the wrong ideas (like the "boat") can trick their brain. They might think, "Oh, I see a boat in the water!" even if it's just a reflection, because their own text suggested it first. It's like trying to fact-check a story while reading the story you just wrote; you might accidentally convince yourself the lie is true.

The TIGER Solution: The "Two-Team" Strategy

TIGER changes the process so the artist never sees their own mistakes while checking the facts. It acts like a strict editor with a two-step process:

Step 1: The Independent Fact-Checkers
Instead of asking the artist to critique the whole story, TIGER sends the Photo to one team and the Story to a completely different team.

Team A (The Input Team): Looks only at the photo and writes a list of hard facts: "Two ducks," "Water," "No boat." They make a "Fact Map."
Team B (The Output Team): Looks only at the story and writes a list of claims: "Three ducks," "Boat," "Water." They make a "Claim Map."

Because Team A never saw the story, they aren't influenced by the mistake. They are purely objective.

Step 2: The Risk Score
Now, TIGER brings the two maps together. It compares every claim in the story against the facts in the photo.

"Two ducks" vs. "Three ducks"? High Risk.
"Water" vs. "Water"? Low Risk.
"Boat" vs. "No boat"? High Risk.

TIGER gives every sentence in the story a "Risk Score." It doesn't just say "this is wrong"; it says, "This specific sentence is 90% likely to be a lie."

Step 3: The Targeted Surgery
Instead of rewriting the whole story (which might ruin the good parts), TIGER only sends the high-risk sentences back to the artist. It says, "You said there were three ducks and a boat. Look at the photo again. Fix only these two things."

The artist fixes those specific spots, and the rest of the beautiful story remains untouched.

Why This is Better

No Bias: By separating the photo-checking from the story-checking, the system stops the "echo chamber" where mistakes reinforce themselves.
Precision: It doesn't guess which parts to fix; it uses math to rank exactly which facts are dangerous.
Efficiency: It only spends energy fixing the broken parts, not the whole story.

Does it work?

The authors tested TIGER on many different tasks:

Images to Text: Describing photos.
Audio to Text: Summarizing sound clips.
Video to Text: Describing movie scenes.
Crisis Reporting: A real-world test where they had to summarize news from Twitter, Facebook, and photos about a hurricane.

In all these tests, TIGER successfully removed the fake facts (like the extra ducks or the non-existent boat) while keeping the story fluent and accurate. It even worked well when the input data was messy and noisy, like in the hurricane case study.

In short: TIGER is like a smart editor who separates the "evidence" from the "draft," scores the mistakes, and only asks the writer to fix the specific lies, ensuring the final story is both beautiful and true.

Technical Summary: TIGER

Problem Statement

The paper addresses faithfulness hallucination in multimodal generation, where a fluent output contains specific facts unsupported by the input (e.g., an image of two trucks described as having three). While existing iterative repair methods attempt to correct these errors by generating feedback conditioned on both the input and the current output, the authors identify a critical failure mode: spurious correlations. When a feedback generator jointly conditions on the input and the hallucinated output, the hallucinated claims can bias the model's interpretation of the input, causing the feedback to endorse unsupported content rather than flagging it. Furthermore, existing methods often produce free-form natural language feedback, which lacks explicit fact-level risk scores, making it difficult to prioritize repairs under a limited compute budget.

Methodology: TIGER

TIGER (Traceable Inference with Graph-Based Evidence Routing) is an inference-time framework designed to mitigate hallucinations by redesigning the feedback mechanism. It operates on a frozen backbone model and consists of three core components:

Independent Atomic Projection:
Instead of jointly processing the input and output, TIGER extracts them independently into structured fact graphs:
- Observation Graph ( $G_X$ ): Extracted from the input ( $X$ ) using a prompt that enumerates observable facts as triples $(e_1, rel, e_2)$ . This graph is computed once and remains fixed.
- Claim Graph ( $G_{Y_t}$ ): Extracted from the current output ( $Y_t$ ) using the same atomic extraction prompt.
  This separation prevents hallucinated claims in the output from influencing the extraction of facts from the input.
Deterministic Fact-Level Risk Ranking:
TIGER computes a risk score $r(f)$ for each claim $f$ in $G_{Y_t}$ based on its relationship with $G_X$ :
- Support Score ( $s(f)$ ): Measures the similarity between a claim and the best-matching fact in $G_X$ . To handle extraction omissions, this score is propagated along coreference edges in $G_{Y_t}$ with geometric decay.
- Conflict Score ( $c(f)$ ): Measures the strongest contradiction between a claim and $G_X$ , defined by topic consistency and conclusion divergence.
- Risk Calculation: $r(f) = (1 - s(f)) + \lambda \cdot c(f)$ .
  Because the scoring is deterministic, the system can rank claims and select a specific subset (top- $\lceil \alpha N \rceil$ ) for repair, enabling fact-level scheduling.
Localized Refinement:
The model performs iterative repair where only the selected high-risk claims are targeted. The refinement step ( $Y_{t+1} = \Phi(P_{refine}, X, Y_t, F_t)$ ) uses the raw input $X$ and the current output $Y_t$ to locally repair the flagged spans, while keeping the backbone frozen.

Theoretical Guarantee:
The paper provides a convergence analysis (Theorem 3.1) showing that under mild assumptions (bounded graph size, per-fact repair progress, bounded side effects), the expected total risk decreases geometrically to an explicit asymptotic bound.

Key Contributions

Identification of Feedback Failure Modes: The authors demonstrate that joint conditioning in feedback generation allows hallucinated claims to bias input interpretation, leading to the reinforcement of errors.
TIGER Framework: A novel inference-time framework that replaces free-form joint feedback with independent atomic projection and deterministic risk ranking. This makes feedback explicit, rankable, and suitable for localized repair.
Comprehensive Evaluation: TIGER is evaluated across four cross-modal generation paths:
- Image $\to$ Text (COCO, AMBER)
- Image+Text $\to$ Text (MMHal-Bench)
- Audio $\to$ Text (Clotho)
- Video $\to$ Text (VideoHallucer)
  The framework uses Qwen2.5-Omni as the primary backbone, with validation on LLaVA-1.5 and proprietary models.

Experimental Results

Hallucination Reduction: TIGER consistently reduces unsupported claims (measured by metrics like CHAIR, AEHR, and HallucRate) across all modalities. For instance, on COCO with Qwen2.5-Omni, it reduced CHAIRs from 0.070 to 0.050 and CHAIRi from 0.069 to 0.035.
Quality Preservation: Unlike some baselines that trade fluency for accuracy, TIGER preserves or improves task quality (e.g., BERTScore increased from 0.588 to 0.643 on COCO).
Ablation Studies:
- Iterative repair alone (without independent extraction) worsened performance by amplifying hallucinations.
- Adding atomic projection restored performance.
- Adding deterministic risk ranking yielded the best results, confirming that all three components are necessary.
Mechanism Analysis: Experiments showed that atomic projection reduces the mention rate of absent objects in feedback (from 5.0% in joint feedback to 2.0% in TIGER). Additionally, the refinement step successfully recovers correct facts that were missed during the initial extraction of the input graph.
Real-World Case Study: In a multi-source crisis reporting scenario (CrisisFACTS), TIGER improved precision from 0.65 to 0.96 and F1 from 0.66 to 0.74, demonstrating its ability to ground summaries in noisy, multi-source evidence.

Significance and Claims

The paper claims that TIGER offers a robust solution to the "feedback bias" problem in iterative multimodal repair. By decoupling input observation from output claim extraction and introducing a deterministic, rankable risk metric, TIGER enables precise, localized repairs without requiring model fine-tuning. The authors assert that this approach generalizes across different backbones and modalities, effectively reducing hallucinations while maintaining the utility of the generated content. The work suggests that structured, graph-based evidence routing is a viable path toward more faithful multimodal generation in real-world, noisy settings.

Limitations Noted:
The authors acknowledge that TIGER focuses on localized factual triples, which may be less direct for abstract, causal, or subjective content. Additionally, the iterative nature adds inference-time computation, suggesting a need for future work on more efficient scheduling strategies for real-time deployment.