Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward

1. Problem Statement

Vision-Language Models (VLMs) have achieved significant success but suffer from critical trustworthiness issues:

• Text-Over-Reliance: Models often prioritize textual cues over visual evidence, leading to answers derived from language priors rather than image content.
• Hallucination: Models generate ungrounded or fabricated details not supported by the visual input.
• Unfaithful Reasoning: Even when models generate Chain-of-Thought (CoT) explanations, the reasoning process often does not align with the visual evidence used to reach the final answer. The "thinking" may be a post-hoc rationalization rather than a faithful reflection of the decision-making process.

Existing interpretability methods (e.g., Grad-CAM, Integrated Gradients) are often computationally expensive (requiring backpropagation or multiple forward passes) and are not optimized for the sequential prediction nature of VLMs.

2. Methodology: Saliency-R1

The authors propose Saliency-R1, a framework that enhances VLM reasoning by aligning the model's internal attention (saliency) with human-annotated ground truth regions using Reinforcement Learning.

A. Efficient Token-Level Saliency Map Generation

Instead of using gradient-based methods, the authors introduce a logits decomposition technique based on the Transformer architecture:

• Direct Contribution: They decompose the final logits of a generated token into the direct contributions of all previous context tokens.
• Visual Token Isolation: By isolating the contributions from visual tokens and rearranging them according to image patch order, they generate a saliency map.
• Efficiency: This method requires no additional forward or backward passes. It leverages existing attention weights and KV caches, making it computationally efficient and suitable for real-time integration into training pipelines.
• ReLU Application: Negative contributions are ignored (via ReLU) to focus on positive influence.

B. Holistic Aggregation via Attention Rollout

To trace how visual information flows through the reasoning process to the final answer:

• Bottleneck Mechanism: The framework treats "thinking tokens" (CoT) as a bottleneck.
• Attention Rollout: It computes the attention weight from visual tokens to thinking tokens ($A_{vt}$) and from thinking tokens to answer tokens ($A_{ta}$). The holistic attention from visual to answer tokens is calculated as $\tilde{A}_{va} = A_{vt} \times A_{ta}$.
• Visualization: This reveals which image regions drive both the intermediate reasoning steps and the final conclusion, filtering out irrelevant background noise.

C. Training via Group Relative Policy Optimization (GRPO)

The core innovation is using the alignment between the generated saliency map and human annotations as a reward signal in a Reinforcement Learning (RL) framework.

• Reward Function ($R_{overall}$):
  1. $R_{accuracy}$: Binary reward for correct answers (evaluated by an LLM judge).
  2. $R_{format}$: Reward for correct output structure (e.g., using <thought> tags).
  3. $R_{saliency}$: The Alignment Score, calculated as the ratio of saliency mass falling within human-annotated bounding boxes (critical regions) to the total saliency mass over the whole image.
• Optimization: The model is fine-tuned using GRPO to maximize this composite reward. This encourages the model to concentrate its attention on task-relevant visual regions during the reasoning process.

3. Key Contributions

1. Novel Saliency Technique: A lightweight, attention-based saliency map generation method that highlights token-wise important regions without extra computational overhead (no backpropagation).
2. Reasoning Trace Visualization: An attention rollout mechanism that visualizes the flow of visual information through the "thinking" bottleneck to the final answer, exposing unfaithful reasoning paths.
3. Saliency-Alignment Reward: The first work to use saliency map alignment with human annotations as a reward signal during the post-training (RL) phase, effectively steering the model to "look" at the right parts of the image.
4. Comprehensive Evaluation: Demonstrated improvements in faithfulness, interpretability, and overall VQA performance across diverse benchmarks.

4. Experimental Results

The authors evaluated Saliency-R1 on Qwen2.5-VL (3B and 7B) models across 10 benchmarks (e.g., MMMU-Pro, MMBench, POPE, ChartQA).

• Faithfulness: On deletion/insertion tests (measuring causal impact of highlighted regions), Saliency-R1 outperformed or matched SOTA methods like Grad-CAM, ATTN-LRP, and TAM, proving its saliency maps are more faithful to the model's actual decision process.
• Performance:
  • General VQA: Saliency-R1 (7B) achieved State-of-the-Art (SOTA) performance, outperforming both general open-source models and specialized reasoning models.
  • Hallucination Reduction: Significant improvements on POPE (object hallucination) and ChartQA (OCR/visual reasoning), with the 7B model outperforming the base model by 4.2% on ChartQA.
  • Comparison: It surpassed commercial models (GPT-4o, Claude-3.5 Sonnet) on specific benchmarks like MME and ScienceQA.
• Interpretability:
  • Quantitative: The "Pointing Game" (PG) and Energy-PG metrics improved significantly (e.g., +14.14% PG accuracy) compared to the base model.
  • Qualitative: Visualizations showed that while base models often attended to irrelevant backgrounds, Saliency-R1 focused tightly on critical regions (e.g., the tail of a bird, specific text in a chart), resulting in more concise and accurate reasoning traces.
• Ablation Studies:
  • Removing the saliency reward (training only with standard GRPO) resulted in lower performance, confirming the necessity of the alignment constraint.
  • Using only the saliency reward (without accuracy reward) yielded competitive results, suggesting the alignment signal is a strong proxy for correctness.

5. Significance

• Trustworthy AI: Saliency-R1 addresses the "black box" nature of VLMs by ensuring that the reasoning process is visually grounded, reducing hallucinations and increasing reliability in safety-critical domains (e.g., medical diagnosis, autonomous driving).
• Efficiency: By eliminating the need for expensive gradient calculations or multiple forward passes for interpretability, the method is scalable and can be integrated into standard RL training pipelines.
• Paradigm Shift: It moves beyond simple "answer correctness" to optimize the process of reasoning, forcing models to align their internal attention mechanisms with human expectations of visual relevance.

In conclusion, Saliency-R1 provides a robust, efficient, and effective framework for making VLMs more interpretable and faithful, bridging the gap between high-level reasoning capabilities and grounded visual understanding.