Technical Summary: Improving Reasoning in Vision-Language Models via Perception Verified Self-Training

Problem Statement

Achieving human-like reasoning in Vision-Language Models (VLMs) is a significant challenge. While Chain-of-Thought (CoT) rationales generated by human annotators or proprietary models (e.g., GPT-4V) have improved reasoning, these approaches are costly and difficult to scale. Self-training, which leverages a model's own outputs as supervision, offers a scalable alternative. However, existing self-training methods for VLMs (e.g., STaR, R3V) typically filter training samples based solely on the correctness of the final answer. This strategy fails to verify the intermediate reasoning steps, leading to two critical failure modes:

1. Visual Hallucinations: Rationales describe objects, attributes, or scenes that are not present in the image.
2. Language Shortcuts: Predictions rely on textual priors or statistical correlations rather than true visual grounding.

Consequently, models trained on such data may learn to produce correct answers for the wrong reasons, propagating errors and hallucinations during fine-tuning.

Methodology

The authors propose a Perception Verified Self-Training framework designed to enforce visually grounded reasoning through a structured, two-stage curriculum learning strategy.

1. Disentangled Perception and Reasoning

The framework employs a structured Caption-Reasoning-Conclusion (CRC) template. Instead of generating a monolithic rationale, the model is prompted to output:

• Caption: A descriptive interpretation of the visual content.
• Reasoning: Logical deduction based on the caption.
• Conclusion: The final answer.
  This structure allows the perception component (caption) to be verified independently of the reasoning and conclusion.

2. PerceptEval: Unsupervised Caption Verification

Since ground-truth captions are unavailable for self-training, the authors introduce PerceptEval, an unsupervised module to evaluate caption quality. It utilizes two complementary signals:

• OCR-Based Text Agreement: For images containing embedded text, PaddleOCR extracts visible text. The semantic similarity between the generated caption and the OCR-extracted text is computed using Sentence-Transformers.
• Visual Agreement: The similarity between the generated caption and the image is measured using FG-CLIP, which focuses on visual element descriptions.
• Domain-Aware Joint Evaluation: To handle varying ratios of text-to-visual content, the framework adaptively adjusts similarity thresholds based on the text area coverage ratio. A caption is accepted only if it satisfies both thresholds.

3. Data Partitioning and Curriculum Learning

Based on the dual verification of caption quality (via PerceptEval) and answer correctness, the dataset is partitioned into three subsets:

• Easy: Correct caption AND correct conclusion.
• Medium: Correct caption BUT incorrect conclusion.
• Hard: Incorrect caption (discarded).

The training proceeds in two stages:

• Stage 1 (Easy Samples): The model is fine-tuned exclusively on "Easy" samples. This establishes a baseline of perceptually grounded reasoning.
• Stage 2 (Caption-Guided Reasoning Enhancement): The model attempts to solve "Medium" samples. For these cases, the verified caption from the initial generation is injected as auxiliary context to guide the model in regenerating the reasoning process. Only regenerated samples that yield the correct conclusion are retained. These enhanced samples are then merged with the "Easy" set for further fine-tuning.

This staged approach ensures that the model masters visual perception before tackling complex reasoning tasks, preventing the model from being overwhelmed by difficult examples early in training.

Key Contributions

1. Perception Verified Self-Training Framework: A novel pipeline that jointly enhances visual perception and reasoning in VLMs by enforcing a disentangled generation process.
2. PerceptEval: An unsupervised metric for evaluating generated image captions, ensuring both visual and textual elements are faithfully captured without ground-truth supervision.
3. Caption-Guided Reasoning Enhancement: A mechanism that uses verified captions as context to correct reasoning errors in "medium-difficulty" samples, effectively turning failure cases into training data.
4. Two-Stage Curriculum Learning: A strategy that stabilizes iterative self-training by gradually incorporating medium-difficulty samples, allowing the model to build upon simpler, verified examples.

Experimental Results

The framework was evaluated across four domains (Commonsense, Natural Science, Language Science, Social Science) using the M3CoT dataset and tested on LLaVA-v1.5 (7B and 13B) and Qwen2-VL-7B models.

• Performance Gains: The proposed method achieved improvements of up to 16% over standard self-training baselines (STaR and R3V). For instance, on LLaVA-v1.5-7B, the method achieved 64.93% accuracy in Language Science and 72.74% in Commonsense, significantly outperforming STaR (48.82% and 64.98%) and R3V (50.24% and 65.49%).
• Generalization: The method demonstrated robustness on Out-of-Distribution (OOD) tasks, achieving the highest accuracy (36.48%) on the MMMU Science validation split compared to other self-training methods.
• Qualitative Improvements: Qualitative analysis showed that while baselines often produced hallucinated descriptions (e.g., identifying non-existent objects) or relied on language shortcuts, the proposed method generated rationales that were both visually detailed and logically coherent.
• Token Efficiency: The method produced moderate-length outputs (69.2–93.9 tokens), balancing explicit reasoning with efficiency, unlike R3V which produced significantly longer, often verbose responses.

Significance and Claims

The paper claims that the proposed framework offers a scalable and cost-effective solution for advancing multimodal reasoning without the need for manually annotated CoT rationales. By explicitly separating perception from reasoning and rigorously filtering training data based on visual grounding, the framework mitigates the propagation of visual hallucinations and language shortcuts. The authors argue that this approach is critical for developing VLMs that rely on visual evidence rather than textual priors, providing a reliable path toward more accurate and trustworthy multimodal reasoning.