Improving Visual Reasoning with Iterative Evidence Refinement

1. Problem Statement

Vision-Language Models (VLMs) have shown strong reasoning capabilities via Chain-of-Thought (CoT), but their reasoning pipelines are largely text-centric. In standard inference, the image is encoded into a fixed set of visual tokens that act as static context. As the model generates text autoregressively, the conditioning shifts toward the growing history of text tokens, causing the model to "forget" or underutilize specific visual evidence.

Existing solutions attempt to fix this by using external tools (e.g., zooming, cropping, or navigating) to generate new image views and re-encode them during inference. However, these approaches suffer from two critical drawbacks:

1. Disruption of Reasoning Continuity: External tools append new image views to the input rather than integrating them directly into the CoT reasoning chain at the specific step where they are needed.
2. Inefficiency: Re-encoding images and managing external tool calls introduces significant inference overhead and requires complex training pipelines with large datasets.

The authors hypothesize that VLMs already possess sufficient visual information within their internal hidden state embeddings. The bottleneck is not the lack of data, but the model's inability to selectively retrieve and reuse relevant visual evidence during the reasoning process without external intervention.

2. Methodology: SIEVE Framework

The authors propose SIEVE (Self-revisit framework for Iterative Evidence Refinement), an end-to-end framework that enables VLMs to "re-visit" visual evidence by retrieving and injecting region-level embeddings directly into the reasoning chain.

A. Self-Guided Visual Evidence Discovery

Instead of manual annotation, SIEVE automatically identifies critical visual regions using a two-stage process:

1. Textual Anchor Identification: The model analyzes its own predictive dynamics using gradient saliency. It identifies input tokens that have the highest gradient magnitude regarding the next predicted token. These tokens (e.g., specific objects or attributes) serve as "textual anchors."
2. Visual Grounding: The system computes cross-modal similarity between these textual anchors and image patch representations within the model's intermediate transformer layers (which are found to capture the best balance of semantic abstraction and spatial fidelity). It selects the top-k image patches with the highest affinity to the anchors, aggregates their embeddings, and caches them as visual evidence snapshots.

B. Reinforcement Learning (RL) Training

SIEVE employs a Visually-Grounded Reinforcement Learning pipeline to teach the model when and how to use these cached embeddings.

• State Definition: The state includes the original image, the generation history, and any previously inserted visual evidence embeddings.
• Action Space: The model chooses between:
  1. Generating the next text token.
  2. Triggering an embedding insertion (injecting a cached region embedding into the context).
  3. Terminating with a final answer.
• Reward Function: A trajectory-level reward is designed with four components:
  • Result Reward ($R_{res}$): Accuracy of the final answer.
  • Format Reward ($R_{format}$): Ensures structured output and explicit embedding selection steps.
  • Embedding Reward ($R_{emb}$): A bonus for correctly using visual evidence when it aids the solution.
  • Action Reward ($R_{act}$): Encourages "thought richness" and prevents the model from outputting empty reasoning tags or collapsing into non-committal behaviors.

C. Inference Process

During inference, if the model determines it needs more visual grounding, it retrieves the pre-computed, task-relevant region embeddings and injects them directly into the latent space of the reasoning chain. This allows subsequent reasoning steps to condition on fine-grained visual cues without re-encoding the image or calling external tools.

3. Key Contributions

1. Paradigm Shift: Proposes a tool-free framework that leverages internal representations (hidden state embeddings) for visual revisiting, eliminating the need for external image manipulation tools (zoom/crop) and redundant re-encoding.
2. Saliency-Based Mechanism: Introduces a self-guided method to identify and cache critical visual evidence based on gradient saliency and cross-modal similarity, requiring no manual annotations.
3. Data Efficiency: Demonstrates that the model can learn to effectively retrieve and insert visual evidence using a very small training dataset (approx. 1,500 samples).
4. Performance Gains: Validates the approach across multiple model scales (4B and 8B parameters) and diverse benchmarks.

4. Experimental Results

The authors evaluated SIEVE on Qwen3-VL (4B and 8B) models across several benchmarks:

• High-Resolution Reasoning (V Bench & HR-Bench):*
  • SIEVE achieved consistent improvements over vanilla models and tool-augmented baselines (like ZoomEye, DyFo).
  • The 4B variant showed a 10.06% improvement over the vanilla model on V* Bench.
  • Average improvement across benchmarks was 8%.
• Generalization:
  • Perception (MME-Real-Lite): +5.05% (4B) and +5.48% (8B) gains.
  • Reasoning (MathVista, LogicVista, WeMath): Significant gains, particularly on WeMath (+11.3% for 8B).
  • Hallucination (HallusionBench): Improved factual accuracy, indicating better grounding.
• Ablation Studies:
  • Random vs. Selected Embeddings: Replacing SIEVE's selected embeddings with random ones caused a substantial performance drop, proving that the semantic relevance of the retrieved evidence is crucial.
  • Layer Selection: Middle transformer layers yielded the highest "Information Hit Ratio" (IHR) for grounding, validating the choice of intermediate layers for feature extraction.
  • Action Rewards: Removing action rewards led to training instability and reward collapse, confirming their necessity for stable policy learning.

5. Significance

The paper challenges the prevailing assumption that complex visual reasoning requires external tool orchestration. By demonstrating that internal multimodal representations contain sufficient information for fine-grained reasoning, SIEVE offers a more efficient, scalable, and lightweight solution. It preserves the continuity of the reasoning chain, reduces inference latency by avoiding re-encoding, and achieves state-of-the-art results with minimal training data. This suggests a future direction where VLMs can dynamically refine their understanding of visual content through internal "self-revisiting" mechanisms rather than external agent loops.