Technical Summary: SENTINEL

Problem Statement

Multimodal Large Language Models (MLLMs) have advanced cross-modal understanding but suffer from object hallucinations, where models generate text that is semantically coherent yet factually inconsistent with the visual input. Existing mitigation strategies face significant limitations:

• Decoding-based methods (e.g., contrastive decoding) often incur prohibitive computational costs and increased inference latency.
• Preference alignment methods frequently rely on expensive external resources, such as proprietary Large Language Models (LLMs) for rewriting or human annotators for data curation. Furthermore, these methods often introduce distribution mismatches between the training data (rewritten by external models) and the target model's original output distribution, which can harm generalization capabilities.

Methodology: SENTINEL

The authors propose SENTINEL (Sentence-level Early iNtervention Through IN-domain prEference Learning), a framework designed to mitigate hallucinations by intervening at the earliest stage of their occurrence without relying on external rewriting or human annotation. The method operates through six key steps:

1. In-domain Candidate Bootstrapping:

   • The current MLLM generates multiple candidate responses via sampling.
   • Objects mentioned in these outputs are extracted using a Scene-GraphParser.
   • Cross-Validation: Two open-vocabulary object detectors (GroundingDINO and YOLO World) are used to cross-check the existence of these objects in the input image.
   • Objects are classified as factual (both detectors confirm presence), hallucinated (both confirm absence), or uncertain. Sentences containing hallucinated objects are flagged as negative samples; those with only factual objects are positive samples. Uncertain objects are ignored to ensure data quality.

2. Context-aware Preference Data Generation:

   • The framework constructs preference pairs $(y_w^+, y_l)$ where $y_w^+$ is a non-hallucinated sentence and $y_l$ is a hallucinated one.
   • Crucially, the positive sample $y_w^+$ is selected to be context-coherent, meaning the objects described in the sentence are explicitly referenced or logically connected to the preceding context. This contrasts with context-agnostic samples, which the authors find less effective for generalization.

3. Iterative Contextual Bootstrapping (ICB):

   • To enhance robustness across diverse scenarios, the framework iteratively extends the generation context. After identifying a non-hallucinated positive sample, it is appended to the context $c$ for the next iteration. This allows the model to learn from progressively complex and varied contexts while maintaining a hallucination-free foundation.

4. Context-aware Preference Learning (C-DPO):

   • The model is trained using a modified Direct Preference Optimization (DPO) loss, termed C-DPO.
   • The input to the policy model is augmented with the context $c$: $x' = [v, q, c]$.
   • The loss function maximizes the likelihood of generating the context-coherent positive sample $y_w^+$ while minimizing the hallucinated negative sample $y_l$.
   • Key Design: The context $c$ is excluded from the loss computation (masked). Gradients are derived solely from the discrimination between $y_w^+$ and $y_l$, ensuring the model learns to distinguish hallucinations at the sentence level without being biased by the context tokens during optimization.

Key Contributions

1. Early Intervention Insight: The authors demonstrate through analysis that hallucination intensity escalates with text length and that intervening at the first occurrence of a hallucination significantly prevents its propagation in subsequent outputs.
2. Efficient, In-domain Framework: SENTINEL eliminates dependency on external proprietary models or human annotators. By using the model's own outputs and lightweight open-vocabulary detectors, it constructs high-quality, style-consistent preference data.
3. Model-Agnostic Performance: The method achieves state-of-the-art results on hallucination benchmarks while preserving or improving general capabilities, demonstrating that in-domain preference learning is superior to rewriting-based approaches.

Experimental Results

Experiments were conducted on LLaVA-v1.5 (7B and 13B) models across multiple benchmarks:

• Hallucination Mitigation:

  • On Object HalBench, SENTINEL reduced the response-level hallucination rate from 52.7% (baseline) to 4.3% for the 7B model, outperforming the previous SOTA (TPO) by a significant margin.
  • On AMBER, hallucinations were reduced by approximately 92% (Object HalBench) and 65% (AMBER) compared to the original model.
  • The method effectively reduced hallucinations across all six categories (Existence, Attribute, State, Relation, Action, Number).

• General Capabilities:

  • Unlike many mitigation methods that degrade general performance, SENTINEL maintained or improved scores on VQAv2, TextVQA, ScienceQA, and MM-Vet.
  • For instance, on MM-Vet, the 7B model improved from 31.0 to 32.6, and the 13B model improved from 36.0 to 36.2.

• Ablation Studies:

  • Rewriting vs. In-domain: Training on rewritten data (via GPT-4) resulted in performance degradation, confirming the importance of preserving the model's intrinsic distribution.
  • Context Coherence: Using context-coherent positive samples ($y_w^+$) yielded better results than context-agnostic samples.
  • Cross-checking: Using two detectors for validation significantly outperformed single-detector approaches.
  • Scalability: Performance improved as the training data scale increased, demonstrating the method's efficiency in utilizing larger datasets without costly annotation.

Significance and Claims

The paper claims that SENTINEL offers a scalable, efficient, and model-agnostic solution for enhancing the reliability of MLLMs. Its primary significance lies in shifting the paradigm from expensive, distribution-shifting external interventions to in-domain, sentence-level early intervention. By focusing on the precise moment hallucinations emerge and leveraging the model's own distribution, SENTINEL successfully curbs the propagation of errors without compromising the model's general reasoning or language generation capabilities.

The authors note a limitation: the current framework lacks spatiotemporal reasoning capabilities, meaning it may not fully address hallucination issues in video-based MLLMs requiring long-term reasoning. However, for static image understanding, it establishes a new standard for efficient hallucination mitigation.