Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models

1. Problem Statement

While Large Language Models (LLMs) have achieved remarkable success in long-chain reasoning (e.g., Chain-of-Thought, OpenAI o1), extending these capabilities to Multimodal Large Language Models (MLLMs) remains a critical bottleneck. The paper identifies three primary challenges:

• Data Scarcity: There is a lack of high-quality, structured, long-chain reasoning datasets for visual and video domains. Collecting and annotating such data is expensive and labor-intensive.
• Training Limitations: Existing training pipelines often fail to effectively teach MLLMs to perform detailed, step-by-step visual reasoning. Directly applying Chain-of-Thought (CoT) often yields sub-optimal results, and current off-policy reinforcement learning methods (like standard DPO) struggle with stability and long-horizon reasoning.
• Static vs. Dynamic: Most existing solutions focus on static images. Extending reasoning to video requires handling complex spatial-temporal dynamics, object tracking, and action sequences, which current models struggle to master without specialized architectures.

2. Methodology

The authors propose a unified framework evolving from Insight-V (image-centric) to Insight-V++ (spatial-temporal), built on three core pillars:

A. Scalable Data Generation Pipeline

To overcome data scarcity, the authors designed an autonomous pipeline that generates structured, long-chain reasoning data without human intervention:

• Progressive Generation: A reasoning generator produces JSON-structured reasoning trajectories with intermediate steps and actions (Continue/Summary).
• Multi-Granularity Assessment: A scoring agent (using strong models like Qwen2-VL) evaluates the generated paths based on step-by-step accuracy and detail, filtering out low-quality data.
• Video Extension (Insight-V++): For video, the pipeline introduces in-context scoring. It uses "golden" examples of high-quality video reasoning (curated by human annotators and verified by top-tier models) to guide the scoring agent, ensuring the generated video reasoning trajectories are coherent and temporally accurate.

B. Dual-Agent Architecture

Instead of a single model attempting to reason and answer simultaneously, the system decomposes the task into two specialized agents:

1. Reasoning Agent: Dedicated to generating detailed, step-by-step analytical chains. It is trained to explore diverse reasoning paths.
2. Summary Agent: Dedicated to critically evaluating the reasoning path, identifying flaws, and synthesizing the final answer. It is trained on both high-quality and flawed reasoning paths to develop robustness against errors.

C. Advanced Training Paradigms

The paper introduces distinct training strategies for Insight-V and Insight-V++:

• Insight-V (Image): Uses Iterative Direct Preference Optimization (DPO). The system generates preference pairs, trains the reasoning agent, and iteratively refines the data to align with human reasoning processes.
• Insight-V++ (Video & General): Moves beyond off-policy DPO to On-Policy Reinforcement Learning (RL) using two novel algorithms:
  • ST-GRPO (Spatial-Temporal Group Relative Policy Optimization): Specifically designed for the Reasoning Agent to master temporal alignment and complex spatial-temporal logic in video. It uses composite rewards including task accuracy, Intersection over Union (IoU) for temporal grounding, and video jigsaw reconstruction.
  • J-GRPO (Judgment Group Relative Policy Optimization): Designed for the Summary Agent to enhance its evaluative robustness. It uses a curriculum learning strategy, weighting judgment accuracy and final answer correctness differently based on the difficulty of the reasoning path.
• Self-Evolving Loop: A closed-loop mechanism where the Summary Agent provides feedback to the Reasoning Agent. The Reasoning Agent refines its paths based on this feedback, and the resulting high-quality data is used to retrain both agents, creating a continuous cycle of self-improvement without new human annotations.

3. Key Contributions

1. Unified Spatial-Temporal Data Pipeline: A scalable, automated pipeline that generates high-fidelity long-chain reasoning data for both images and videos, utilizing in-context scoring to ensure quality in the video domain.
2. Novel RL Algorithms (ST-GRPO & J-GRPO): Tailored reinforcement learning objectives that stabilize training for multi-agent systems. ST-GRPO enhances spatial-temporal reasoning, while J-GRPO improves the summary agent's ability to judge reasoning quality.
3. Closed-Loop Self-Evolution: A training strategy where the multi-agent system autonomously generates, evaluates, and refines its own training data, breaking the dependency on fixed human-annotated datasets.
4. Dual-Agent Decomposition: A robust architecture that separates reasoning generation from answer synthesis, preventing error compounding and improving interpretability.

4. Experimental Results

The framework was evaluated on extensive benchmarks using base models like LLaVA-NeXT and Qwen2.5-VL:

• Image Reasoning:
  • Insight-V improved average reasoning performance by 8.1% across six challenging benchmarks when integrated with LLaVA-NeXT.
  • Insight-V++ (on Qwen2.5-VL) achieved a +4.8% gain on general image reasoning and an impressive 53.9 average score on high-complexity tasks, outperforming all previous models built on the same base.
• Video Reasoning:
  • Insight-V++ achieved state-of-the-art results on six video benchmarks (e.g., VideoMME, VideoMMMU, VideoHolmes).
  • It delivered a +6.9% average improvement over strong baselines, significantly narrowing the gap with proprietary models like GPT-4o and Gemini-2.5-Pro, particularly in tasks requiring domain-specific knowledge (STEM) and temporal grounding.
• Ablation Studies: Confirmed that the multi-agent design outperforms single-model CoT, and that the self-evolving strategy provides greater gains on complex reasoning tasks compared to static training.

5. Significance

This work represents a significant leap forward in Visual Reasoning by addressing the fundamental bottlenecks of data scarcity and training instability in MLLMs.

• Scalability: The self-evolving, annotation-free data generation pipeline offers a scalable path to training models on increasingly complex reasoning tasks.
• Generalization: By unifying image and video reasoning under a single framework, Insight-V++ demonstrates that advanced reasoning capabilities can be transferred across modalities.
• Architectural Shift: The shift from single-model reasoning to a collaborative, self-correcting multi-agent system provides a new paradigm for achieving human-level visual understanding and problem-solving.
• Practical Impact: The ability to perform rigorous spatial-temporal reasoning makes this framework highly relevant for real-world applications such as autonomous driving, robotics, and scientific video analysis.