Technical Summary: VIHD - Visual Intervention-based Hallucination Detection

Problem Statement

Medical Multimodal Large Language Models (MLLMs) have demonstrated significant potential in assisting clinical diagnosis through Visual Question Answering (VQA) and automated report generation. However, a critical limitation remains: these models frequently generate hallucinated responses—outputs that are linguistically coherent but contradict the visual evidence provided. Such errors pose substantial risks to clinical decision-making.

Existing hallucination detection methods generally fall into three categories:

1. Supervised Detection: Requires labor-intensive annotated datasets and struggles to generalize to new domains or architectures.
2. External Verification: Relies on auxiliary models or knowledge bases, introducing significant computational overhead and dependency on external tools.
3. Introspective Detection: Self-contained methods that estimate uncertainty or logical consistency without extra training. While efficient, current introspective approaches often rely on heuristic and context-agnostic external perturbations. They treat MLLMs as black boxes, failing to probe the internal causal dependencies between generated tokens and specific visual evidence during the decoding process.

Methodology: VIHD Framework

The authors propose VIHD (Visual Intervention-based Hallucination Detection), a training-free framework designed to probe the fidelity of cross-modal reasoning through targeted, fine-grained visual interventions. The method operates in three distinct stages:

1. Visual Dependency Probing (VDP)

To identify where the model relies most heavily on visual evidence, VIHD analyzes the attention mechanisms within the LLM decoder.

• Mechanism: It calculates the attention weights between generated tokens and input visual tokens across all decoder layers and attention heads.
• Localization: It aggregates these weights to quantify layer-wise visual dependency ($s^{(l)}$). The method then identifies a contiguous window of $w$ layers $[l^*, l^*+w-1]$ exhibiting the strongest cumulative visual dependency. This ensures interventions are applied where cross-modal attention is most prominent.

2. Visual Intervention Decoding (VID)

Once the visually dominant layers are identified, VIHD performs a targeted intervention to observe how the model's output changes when specific visual evidence is suppressed.

• Mechanism: During the decoding process within the identified window, the algorithm masks the top-$\rho$ fraction of visual tokens that have the highest attention weights relative to the current generated token.
• Outcome: This produces an "intervened response" ($r'$) conditioned on a perturbed visual representation. By comparing the original response ($r$) with the intervened response ($r'$), the method reveals the model's dependency on specific visual tokens.

3. Calibrated Semantic Entropy (CSE)

To quantify hallucination severity, VIHD computes a metric based on the semantic distribution of the responses.

• Sampling: The model performs $M$ independent sampling runs to generate two sets of responses: normal responses $\{r^{(i)}\}$ and visual-intervened responses $\{r'^{(i)}\}$.
• Clustering: These responses are clustered into semantic-equivalence classes using a pre-trained model (DeBERTa-Large-MNLI), yielding a normal semantic distribution $P(s)$ and an intervened distribution $P(s')$.
• Adaptive Fusion: Recognizing that intervention impacts vary by query granularity, VIHD employs an adaptive fusion strategy:
  • If the distributions align closely (cosine similarity $\ge \tau$), it uses complementary fusion to reinforce shared semantics.
  • If they diverge, it uses contrastive fusion to suppress masking-induced hallucinations.
• Metric: The final Calibrated Semantic Entropy (CSE) is calculated from this fused distribution. A higher CSE indicates significant semantic dispersion between the original and intervened outputs, serving as a robust signal for hallucination.

Key Contributions

1. VIHD Framework: Introduction of a training-free method that utilizes fine-grained visual interventions to probe internal causal dependencies, moving beyond black-box heuristic perturbations.
2. Adaptive Pipeline: Design of a three-stage pipeline comprising Visual Dependency Probing (to locate critical layers), Visual Intervention Decoding (to mask high-attention tokens), and Calibrated Semantic Entropy (to quantify severity via adaptive distribution fusion).
3. Empirical Superiority: Extensive validation demonstrating that VIHD consistently outperforms state-of-the-art introspective methods across diverse medical VQA benchmarks and model architectures.

Experimental Results

The authors evaluated VIHD on three medical VQA benchmarks (VQA-RAD, SLAKE, and VQA-Med-2019) using two medical MLLMs (Hulu-4B and Lingshu-7B). Hallucination labels were generated using the GREEN model, and performance was measured via AUC (Area Under the ROC Curve) and AUG (Area under the GREEN Curve).

• Performance Gains: VIHD consistently outperformed seven competing methods (including token uncertainty, consistency-based, and semantic uncertainty baselines).
  • On Hulu-4B, VIHD improved the average AUC by 8.57% across all questions and achieved a substantial 12% AUC increase on the VQA-RAD dataset.
  • On Lingshu-7B, it maintained an advantage with an average AUC improvement of 3.22% overall.
• Efficiency: The inference time for VIHD was approximately 11.59 seconds per sample, comparable to the leading baseline VASE (11.55s/sample), indicating that performance gains were achieved without additional computational cost.
• Ablation Studies:
  • Removing any component (VDP, VID, or CSE) resulted in performance degradation, validating the necessity of the full pipeline.
  • High-attention masking was found to be superior to random or low-attention masking, confirming that targeting salient visual tokens is crucial for effective calibration.
  • A sliding window width of $w = L/2$ and a masking ratio of $\rho = 10\%$ were identified as optimal hyperparameters.

Significance and Claims

The paper claims that VIHD underscores the importance of fine-grained visual dependency for hallucination detection. By shifting from external, heuristic perturbations to internal, targeted interventions based on the model's own attention mechanisms, VIHD provides a more reliable signal for detecting when a model is hallucinating.

The authors position VIHD as a deployment-friendly, training-free solution that does not require annotated hallucination datasets or external verification tools. The consistent improvements across different model capacities and dataset characteristics suggest that the method effectively addresses the core issue of MLLMs generating linguistically plausible but visually unsupported medical claims. The code is made available to facilitate further research in this domain.