Technical Summary: ReGround

Problem Statement

Vision-Language Models (VLMs) frequently suffer from visual grounding decay during multi-step reasoning. As reasoning chains lengthen, the model's reliance on language priors increases while its attention to image tokens diminishes. The authors identify a consistent signature of this degradation: across 2,510 re-examined samples, attention entropy over image tokens typically decreases significantly during the first reasoning round (Round 1) and fails to recover without intervention.

While prior work has documented this decay, a critical gap remains: simply re-injecting visual evidence does not guarantee correction. The paper posits that diagnostic quality is the determining factor. Without targeted self-diagnosis, visual re-examination can lead to "harmful second-guessing," where the model overrides correct answers or fails to recover from errors. Conversely, effective re-examination requires both image re-injection and a meaningful, targeted diagnostic cue.

Methodology: The ReGround Framework

ReGround is a two-stage, tool-free framework designed to teach VLMs to autonomously diagnose grounding failures and selectively re-examine visual evidence. It requires no architectural modifications or external tools (e.g., cropping or zooming tools).

Stage 1: Cold-Start Supervised Fine-Tuning (SFT)

The framework constructs a specialized training dataset using a sequential Re-examination Trigger and Visual Self-Diagnosis:

1. Re-examination Trigger Policy: Not all samples require re-examination. The system filters initial responses through three steps:

   • Is Correct? If the answer is wrong, re-examination is triggered unconditionally.
   • Is Grounded? If the answer is correct, a judge model (Qwen2.5-VL-72B) evaluates if the reasoning is truly grounded using four criteria: visual attribute coverage, alternative elimination, absence of hedging, and visual anchoring. Correct but ungrounded answers trigger re-examination.
   • Is Verifiable? A stochastic trigger ($\epsilon$) is applied to correct, grounded answers to teach the model that re-examination can also serve as verification.
   • No-ReGround: Samples passing all checks without a stochastic trigger are retained as single-turn trajectories to prevent over-triggering.

2. Visual Self-Diagnosis and Re-injection: Once triggered, the system generates a diagnostic cue inside a <reground> tag. Three modes of information re-injection are used:

   • Correction: Identifies likely points of visual misperception to redirect attention.
   • Grounding: Forces the model to anchor speculative language to concrete spatial coordinates or elements without revealing the answer.
   • Verification: Re-injects the image holistically to double-check the logical chain.

3. Capability Bootstrapping: During data construction, a stronger model variant (e.g., 72B) generates the diagnostic cues. The policy model (e.g., 7B) learns to generate these cues autonomously during SFT. The stronger variant is discarded after training.

4. Trajectory Formatting: Data is formatted into a multi-turn template: Initial Reasoning $\rightarrow$ <reground> (Diagnosis + Re-injection) $\rightarrow$ Revised Reasoning $\rightarrow$ Final Answer.

Stage 2: Reinforcement Learning (GRPO)

The policy is refined using Group Relative Policy Optimization (GRPO) with an asymmetric reward function to calibrate the triggering behavior:

• Re-grounding Reward ($R_{reg}$): Rewards the structural presence of a valid <reground> span.
• Accuracy Reward ($R_{acc}$): Rewards correct final answers but includes a negative interaction term. This penalizes the marginal benefit of correctness if the model triggered a re-examination unnecessarily, discouraging blind guessing.
• Formatting Penalty ($R_{form}$): Penalizes syntax errors.

This reward shaping creates an asymmetric distribution: the highest reward goes to successful recovery (re-grounding + correct), while "blind failure" (no re-grounding + incorrect) is penalized. This encourages the model to learn when to re-examine and how to use the re-examination effectively.

Key Contributions

1. Empirical Evidence of Decay and Recovery: The authors demonstrate that attention entropy narrows in 98.0% of reasoning steps but recovers after image re-injection in 94.4% of cases.
2. Diagnostic Quality as the Bottleneck: The paper establishes that generic re-examination can degrade performance, whereas targeted self-diagnosis recovers 79–90% of the gains provided by an oracle (stronger model) cue.
3. Triggering Policy Analysis: The study reveals that over-triggering damages correct answers, while conservative triggering forfeits recoverable gains. A balanced policy is essential.
4. A Practical, Tool-Free Framework: ReGround improves performance across multiple benchmarks without external tools or architectural changes, relying solely on image re-injection and self-diagnosis.

Experimental Results

Experiments were conducted on eight benchmarks (including MathVista, MathVision, MathVerse, HallusionBench, MMBench, etc.) using two VLM backbones (Qwen2.5-VL-7B and Qwen3-VL-8B).

• Performance Gains: On Qwen2.5-VL-7B, ReGround achieved the best performance on five of eight benchmarks, with significant improvements on math-reasoning tasks (e.g., +3.8 on MathVista, +4.5 on MathVerse) and a substantial +6.8 gain on HallusionBench (targeting visual hallucination).
• Cross-Backbone Transfer: Applying the same training recipe to Qwen3-VL-8B yielded consistent gains across all eight benchmarks without hyperparameter re-tuning.
• Efficiency: ReGround incurs moderate inference overhead (1.62× total tokens, 1.76× latency) compared to the baseline but outperforms tool-augmented baselines like Thyme in both speed and accuracy.
• Ablation Studies:
  • Mechanism: Visual re-injection is essential; text-only reflection yields negligible gains.
  • Data Composition: Removing "No-ReGround" samples causes the trigger rate to spike to ~85% and degrades accuracy, confirming the need for negative examples.
  • Cue Quality: Generic cues (e.g., "look again") can hurt performance. Targeted self-diagnosis is the key variable.

Significance and Claims

The paper claims that diagnostic quality, rather than the mere act of re-examining visual evidence, is the central factor in successful visual self-correction. ReGround demonstrates that VLMs can learn to autonomously identify when they have lost visual grounding and what specific evidence to revisit, provided they are trained with high-quality diagnostic cues and a balanced triggering policy.

The authors position ReGround as a complementary, tool-free mechanism that addresses the specific failure mode of "visual grounding drift" in multi-step reasoning. By isolating the impact of diagnostic guidance from the re-examination format, the work suggests that future improvements in grounded reasoning should focus on enhancing the model's ability to self-diagnose and target specific visual evidence, rather than solely relying on external tools or larger model scales.