Bad Seeing or Bad Thinking? Rewarding Perception for Vision-Language Reasoning

` tags, these blocks handle planning, hypothesis formation, and logical deduction based on the grounded evidence.

This transformation turns the VLM's "black box" perception into a transparent, interpretable sequence, isolating the locus of error.

2.2. Perception Verification (PV): The "Blindfolded Reasoner" Test

To supervise the perception blocks without ground-truth visual descriptions, the authors introduce Perception Verification.

• Mechanism: A strong, text-only Large Language Model (LLM) acts as a "reasoning oracle." It is fed the user's question and the VLM's generated perception text (<recognition> blocks) but is withheld the original image.
• Logic: If the text-only proxy can correctly solve the question using only the VLM's descriptions, the perception block is deemed a "sufficient statistic" for the task.
• Reward: A binary reward ($R_P \in \{0, 1\}$) is assigned. A success ($1$) indicates the VLM successfully extracted the necessary visual facts; a failure ($0$) implies "bad seeing." This approach operationalizes the Information Bottleneck principle, rewarding minimal yet sufficient visual representations.

2.3. Structured Verbal Verification (SVV)

To address the high variance of standard LLM judges and the brittleness of rule-based verifiers for the final outcome, the authors propose Structured Verbal Verification.

• Mechanism: Instead of asking an LLM to subjectively "judge" an answer, the verifier is provided with a structured natural language algorithm (a verification protocol).
• Execution: The LLM is instructed to "execute" this protocol step-by-step (e.g., identifying answer types, extracting boxed content, performing type-specific semantic comparisons).
• Benefit: This shifts the verifier's role from subjective estimation to structured execution, significantly reducing variance and providing a reliable reward signal ($R_O$) for free-form answers.

2.4. Modality-Aware Credit Assignment (MoCA)

The core innovation lies in how rewards are routed during reinforcement learning (specifically Group Relative Policy Optimization, GRPO).

• The Problem: In standard GRPO, a failed outcome ($R_O=0$) results in a negative advantage signal applied uniformly across the entire trajectory, potentially punishing correct perception steps if the subsequent reasoning failed.
• The Solution: MoCA uses the binary perception reward ($R_P$) as a gate to distinguish failure modes:
  • "Bad Thinking" ($R_O=0, R_P=1$): The perception was correct, but reasoning failed. The framework applies a protection term to perception tokens, preventing the model from "unlearning" valid visual grounding capabilities.
  • "Bad Seeing" ($R_O=0, R_P=0$): Both perception and reasoning failed. The framework amplifies the penalty for perception tokens to drive improvement in visual extraction.

3. Key Contributions

1. Decoupled Architecture: The paper shifts VLM training from holistic outcome supervision to a decoupled, modality-aware framework, transforming the VLM into an interpretable, modular reasoner.
2. Perception Verification via Proxy: A novel "blindfolded reasoning" paradigm that validates visual fidelity by testing if a text-only reasoner can solve a task using only the model's descriptions.
3. Structured Verbal Verification: A method to internalize the benefits of agentic "thinking" without external computational overhead, providing robust, low-variance evaluation for free-form responses.
4. MoCA Mechanism: A credit assignment strategy that resolves the "seesaw effect" by distinguishing between perception and reasoning errors, allowing simultaneous optimization of both capabilities.

4. Experimental Results

The authors evaluated MoCA on a comprehensive suite of benchmarks spanning perception-intensive, reasoning-intensive, and rich-modality tasks.

• Performance Gains: The MoCA model (based on a 7B parameter base) demonstrated significant improvements across all categories compared to its base model and other state-of-the-art (SOTA) open-source and proprietary models.
  • Perception-Centric: Achieved 86.6 on V* and 87.0 on InfoVQA, outperforming specialized perception models like DeepEyes.
  • Reasoning-Centric: Scored 73.8 on MathVista and 54.8 on EMMA, surpassing many larger models.
  • Rich Modalities: Achieved a new state-of-the-art of 58.3 on DUDE (Document Understanding).
• Ablation Studies:
  • Removing Perception Verification (PV) caused significant drops in perception-intensive tasks (e.g., -6.9 on V*), confirming that targeted perception supervision is a critical bottleneck.
  • Removing MoCA logic (using simple reward shaping) resulted in performance drops, validating the necessity of distinguishing "bad seeing" from "bad thinking" to prevent unlearning.
  • Replacing SVV with standard LLM prompting led to unstable training and lower performance due to high reward variance.
• Reliability: Human evaluation studies confirmed that the "blindfolded reasoner" proxy achieves substantial agreement (86.31% accuracy) with human judgments on perceptual sufficiency, validating its reliability as a reward signal.

5. Significance and Claims

The paper claims to fundamentally rethink VLM training by addressing the root cause of the perception-reasoning trade-off: the lack of modality-aware feedback. By externalizing perception and directly rewarding it, the authors demonstrate that a single VLM can achieve simultaneous performance gains across a wide task spectrum without the heavy engineering burden of external agentic frameworks.

The work posits that the "seesaw effect" is not an inevitable limitation of VLMs but a consequence of ambiguous credit assignment. The proposed MoCA framework offers a scalable, lightweight path to robust perception-reasoning synergy, internalizing the "thinking with images" paradigm directly within the model's autoregressive generation process. The authors emphasize that their approach is orthogonal to existing external tool-based frameworks, offering a distinct and efficient alternative for advancing generalizable multimodal intelligence.