More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models

1. Problem Statement: The Dual Nature of Reasoning

The paper investigates a critical paradox in Vision-Language Models (VLMs): while reasoning capabilities (often trained via Reinforcement Learning like GRPO) significantly improve performance on complex logical tasks, they can inadvertently degrade performance on tasks requiring basic visual perception.

• The Phenomenon: The authors identify a "dual nature" of reasoning. Initially, reasoning steps enhance logical inference. However, as the reasoning length increases, the model's perceptual grounding (ability to accurately interpret visual details) deteriorates.
• The Consequence: In many cases, "thinking too much" leads to lower accuracy than "thinking less" or answering directly. The model may correctly identify visual details in the early stages of reasoning but hallucinate or misread them in later stages.
• Root Cause: Through error analysis and attention tracking, the authors attribute this to Visual Forgetting. As the model generates longer reasoning chains, its attention to visual tokens diminishes significantly, causing the model to rely increasingly on its internal textual history rather than the input image.

2. Methodology: Vision-Anchored Policy Optimization (VAPO)

To address visual forgetting, the authors propose VAPO, a policy gradient algorithm designed to explicitly steer the reasoning process toward visually grounded trajectories. Unlike standard RL methods that only reward the final answer, VAPO introduces intermediate supervision.

Key Components of VAPO:

1. Visual Claims Generation:
   • For each training example, a set of factual claims about the image is generated (e.g., by GPT-5).
   • These claims are balanced (50% correct, 50% incorrect) and independent of the specific question to ensure they test pure visual perception.
2. Visual Anchors:
   • During the reasoning rollout, these claims are inserted at specific points (anchors) along the reasoning trajectory.
   • At each anchor, the model is prompted to judge the validity of the claim (Yes/No) based only on the visual input, interrupting the flow to assess perceptual capability.
3. Perception Reward ($R_{perc}$):
   • A new reward signal is introduced based on the model's accuracy in judging these visual claims.
   • Late-Emphasis Weighting: Since visual forgetting is most severe in later reasoning steps, the reward calculation applies higher weights to anchors appearing later in the sequence ($w_k = \exp(\beta \cdot a_k/T)$).
4. Final Objective:
   • The total reward combines standard accuracy/format rewards with the perception reward:
     $$R_i = R_{acc} + R_{fmt} + \gamma \cdot \mathbb{1}[R_{acc}=1] \cdot R_{perc}$$
   • The accuracy gate ($\mathbb{1}[R_{acc}=1]$) ensures the model only receives perception rewards if the final answer is correct, preventing "reward hacking" where the model might focus on perception but fail the main task.

3. Key Contributions

• Discovery of the Dual Nature: The paper provides empirical evidence that reasoning is not universally beneficial; it creates a trade-off where extended reasoning weakens perceptual grounding, leading to a "no-free-lunch" scenario in multimodal settings.
• Identification of Visual Forgetting: The authors quantify the decline in visual attention during reasoning and demonstrate that "visual forgetting" is the primary bottleneck, accounting for over 50% of errors in reasoning-heavy benchmarks.
• VAPO Algorithm: They propose a simple yet effective training method that integrates visual checks directly into the RL loop, forcing the model to maintain visual attention throughout the reasoning process.
• State-of-the-Art Performance: The resulting model, VAPO-Thinker-7B, establishes new SOTA results across diverse benchmarks, proving that strengthening visual grounding enhances overall reasoning efficacy.

4. Experimental Results

The authors evaluated VAPO-Thinker-7B (based on Qwen2.5-VL-7B) against strong baselines (including R1-OneVision, VLAA-Thinker, Vision-R1, and proprietary models like GPT-5-Thinking).

• Mathematical Benchmarks: Achieved an average improvement of 2% over the previous best open-source models (e.g., 49.1% $\to$ 51.1% average on MathVerse, MathVista, etc.).
• General-Purpose Benchmarks: Demonstrated a more significant gain of 3.2% on vision-intensive tasks (e.g., MMStar, HallusionBench), reaching 63.1% average accuracy (vs. 59.9% for the best baseline).
• Comparison with Inference-Time Remedies: VAPO outperformed inference-time strategies like "Visual Replay" (re-inserting the image) and "Focus Prompt" (asking the model to look back), proving that training-time intervention is necessary to fix the underlying behavioral deficiency.
• Ablation Studies:
  • Increasing the number of anchors ($K$) improves performance up to a saturation point ($K=20$).
  • The late-emphasis weight ($\beta$) is crucial; setting $\beta=1.5$ yielded the best results, confirming that later reasoning steps are where visual forgetting is most critical.
  • The method is robust across different base models (tested on Qwen3-VL) and claim generators (works with open-source models like LLaVA).

5. Significance

This work fundamentally challenges the prevailing assumption in the AI community that "more reasoning is always better." It highlights a specific failure mode in multimodal reasoning where the model loses sight of the visual input.

• Paradigm Shift: It moves the focus from simply scaling reasoning length to ensuring visual grounding throughout the reasoning process.
• Practical Impact: VAPO offers a computationally efficient solution (minimal overhead compared to standard GRPO) that significantly boosts the reliability of VLMs, particularly for tasks requiring fine-grained visual understanding (e.g., chart reading, object counting, navigation).
• Future Direction: The paper suggests that future reasoning models must be designed with mechanisms to prevent modality imbalance, ensuring that visual cues remain active even during complex cognitive processes.