💬 NLP

From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models

This paper demonstrates that decoupling vision-language model post-training into distinct stages for visual perception, visual reasoning, and textual reasoning—specifically prioritizing perception optimization via reinforcement learning—significantly improves both perception and reasoning accuracy while reducing the need for excessive reasoning traces.

Original authors: Juncheng Wu, Hardy Chen, Haoqin Tu, Xianfeng Tang, Freda Shi, Hui Liu, Hanqing Lu, Cihang Xie, Yuyin Zhou

Published 2026-05-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Juncheng Wu, Hardy Chen, Haoqin Tu, Xianfeng Tang, Freda Shi, Hui Liu, Hanqing Lu, Cihang Xie, Yuyin Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex math problem, but the problem is written on a piece of paper with a smudge over a crucial number.

If you try to "think" harder about the math, you will just get a more confident, but completely wrong, answer. You might write a long, detailed essay explaining why the answer is 50, when the smudged number actually meant the answer should be 5.

This is exactly what the paper "From Seeing to Thinking" argues is happening with current AI models that look at images and try to reason about them (Vision-Language Models).

Here is the breakdown of their discovery and solution, using simple analogies:

The Problem: The "Confidently Wrong" AI

The paper finds that when these AI models fail at visual tasks (like geometry or reading a chart), it's rarely because they can't do the math or logic. It's because they can't "see" correctly in the first place.

The Analogy: Imagine a detective trying to solve a crime. If the detective misidentifies the suspect's shoe size (Perception Error), no amount of brilliant detective work (Reasoning) will solve the case. In fact, the more the detective "thinks" about the wrong shoe size, the more they will write a long, confusing report that leads to the wrong conclusion.
The Finding: The authors analyzed many failed AI attempts and found that 87% of the errors started with a simple visual mistake (like misreading a number or mixing up a shape). Once the AI made that mistake, "thinking longer" didn't fix it; it just made the AI double down on the error.

The Solution: A Three-Step Training Camp

Currently, most AI training mixes everything together: it teaches the model to see, read, and think all at once. The authors say this is like trying to teach someone to drive a car, fix the engine, and navigate a map all in the same lesson. It's too messy.

Instead, they propose a "Staged Training" approach, breaking the learning process into three distinct phases:

Stage 1: The Eyes (Visual Perception)
- Goal: Teach the AI to accurately describe what is in the picture without trying to solve a problem yet.
- The Method: They used a special type of training called RL (Reinforcement Learning). Think of this as a strict coach who gives the AI a "thumbs up" only if it correctly identifies a detail (like "there are 7 streetlamps") and a "thumbs down" if it guesses or hallucinates. They found this was much better than the old method of just showing the AI pictures with captions (SFT), which is like reading a storybook without being tested on the details.
Stage 2: The Brain (Textual Reasoning)
- Goal: Teach the AI how to think logically using words, but without any images.
- The Analogy: This is like teaching a student math problems on a whiteboard before giving them a diagram.
Stage 3: The Integration (Visual Reasoning)
- Goal: Now that the AI has "good eyes" and a "trained brain," teach it to combine them to solve visual puzzles.

Why This Order Matters

The paper discovered that the order of training is critical.

The Right Way: Build the eyes first, then the brain, then combine them.
The Wrong Way: If you try to teach the AI to solve complex visual puzzles before it has learned to see clearly, it gets confused. It's like trying to teach a child to play chess before they know how the pieces move. The paper showed that reversing this order actually made the AI worse at both seeing and thinking.

The Results: Smarter and Faster

When the authors trained their models using this new "Staged" method, the results were impressive:

Better Accuracy: The models got significantly better at visual math and real-world perception tests.
Shorter Thinking: This is the most surprising part. Because the models had "good eyes," they didn't need to "think" as long to get the right answer.
- The Analogy: A model with bad eyes keeps re-reading the problem, checking the image, and rewriting its thoughts because it's unsure. A model with good eyes sees the answer immediately and writes a short, clear solution.
- The Data: Their models achieved 1.5% higher accuracy while using 20% less "thinking time" (shorter text responses) compared to models trained the old, mixed way.

The Big Picture: A New Way to Learn

The authors also introduce a new concept called "Capability Curriculum."

Old Way: Train the AI on easy problems, then medium, then hard (Difficulty-based).
New Way: Train the AI on specific skills in a specific order (Perception $\to$ Logic $\to$ Reasoning).
The Magic: They found that these two methods work best when combined. It's like a school that organizes classes by subject (Math, then Science) and arranges the lessons from easy to hard within those subjects. Doing both gave the AI the best results of all.

In summary: To make AI better at "thinking" about images, we first need to make sure it can actually "see" them correctly. By separating these skills and training them in the right order, we get models that are not only smarter but also more efficient, avoiding the trap of over-thinking simple visual mistakes.

Technical Summary: From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models

1. Problem Statement

Recent advancements in Vision-Language Models (VLMs) have heavily emphasized long chain-of-thought (CoT) reasoning, often driven by reinforcement learning (RL) techniques. However, this work identifies a critical bottleneck: performance on complex visual tasks (e.g., visual mathematics, geometry, diagram understanding) is frequently limited not by reasoning capabilities, but by visual perception.

The authors observe that when a VLM makes an initial error in perceiving visual details (e.g., misidentifying a geometric angle or object count), subsequent reasoning steps rarely correct this mistake. Instead, the model compounds the error based on incorrect perceptual assumptions, leading to long, convoluted reasoning traces that fail to reach the correct conclusion. Conversely, when perception is accurate, reasoning tends to be concise and converges quickly. The paper argues that current post-training paradigms, which often merge perception and reasoning data or prioritize reasoning, fail to address this foundational deficit.

2. Methodology

The proposed solution is a staged post-training framework that decouples VLM capabilities into three distinct, sequential stages, each utilizing specialized datasets and training strategies:

2.1. Capability Decomposition

The training pipeline is divided into three stages:

Visual Perception: Focuses on recognizing fine-grained visual details and spatial relations without requiring multi-step reasoning.
Textual Reasoning: Focuses on logical inference using text-only inputs.
Visual Reasoning: Focuses on complex multimodal tasks requiring both accurate perception and multi-step reasoning.

2.2. Data Synthesis and Curation

Perception Data ( $D_{perc}$ ): Synthesized from the DOCCI dataset (fine-grained image captions). An LLM generates perception-focused questions based on captions. A filtering mechanism retains only samples where the base VLM fails to answer correctly using the image alone but succeeds when provided with the caption. This isolates samples where the limitation is purely perceptual.
Textual Reasoning Data ( $D_{text}$ ): Sourced from ORZ-Math-13k, containing challenging math problems solvable without visual inputs.
Visual Reasoning Data ( $D_{vis}$ ): Curated from multiple open-source sources (CLEVR-Math, GeoQA170K, Math PUMA, etc.), requiring both perception and reasoning.

2.3. Training Strategies

Staged Training: The model is trained sequentially ( $D_{perc} \rightarrow D_{text} \rightarrow D_{vis}$ $D_{p er c} \to D_{t e x t} \to D_{v i s}$ ) using Group Relative Policy Optimization (GRPO).
- Stage 1 (Perception): The authors compare Supervised Fine-Tuning (SFT) against Reinforcement Learning with Verifiable Rewards (RLVR). They find RLVR significantly more effective for perception, as it penalizes hallucinated interpretations more robustly than token-level SFT.
- Stages 2 & 3: Utilize GRPO to refine reasoning capabilities.
Vision Encoder Handling: In the perception stage, the vision encoder is active. In subsequent stages, strategies vary (e.g., freezing the encoder in textual reasoning stages), but the primary driver of success is the staged data order.
Curriculum Integration: The authors demonstrate that this capability-based staging is orthogonal to traditional difficulty-based curricula. Combining both (ordering data by difficulty within each capability stage) yields additive gains.

3. Key Contributions

Decoupling Perception and Reasoning: The paper establishes that visual perception is a distinct, fundamental capability that must be solidified before refining visual reasoning. Treating them as a merged task leads to suboptimal performance.
Staged Training Paradigm: The authors propose a specific training order (Perception $\rightarrow$ Textual Reasoning $\rightarrow$ Visual Reasoning) that consistently outperforms merged training. Reversing this order (training reasoning before perception) degrades performance.
RLVR for Perception: The work demonstrates that RLVR is superior to caption-based SFT for training visual perception, providing a stronger training signal that reduces hallucinations.
Efficiency Gains: Models trained with this approach achieve higher accuracy with significantly shorter reasoning traces (20.8% shorter), indicating that accurate perception reduces the need for excessive "thinking" to correct initial errors.
New Curriculum Dimension: The paper introduces "capability-dimension curriculum learning" as a complementary approach to difficulty-based curricula, showing that combining both dimensions yields further improvements.

4. Experimental Results

Experiments were conducted on Qwen2.5-VL-7B and Qwen3-VL-8B backbones, comparing staged training against merged training and various open-weight baselines (e.g., OneThinker, GThinker).

Performance Gains:
- Visual Math: Staged training improved WeMath accuracy by 5.2% (Qwen2.5-VL-7B) and 1.5% (Qwen3-VL-8B) over merged training. On MathVista, the staged Qwen3-VL-8B achieved 75.9%.
- Perception: The approach improved RealWorldQA accuracy by 3.7% (Qwen3-VL-8B) and maintained high performance on A-OKVQA and POPE, avoiding the "perceptual tax" often seen in reasoning-only training.
- Overall: The staged Qwen3-VL-8B achieved a record overall average of 65.8%, surpassing the base model and the reasoning-specialized OneThinker-8B.
Efficiency: Staged models produced reasoning traces 20.8% shorter than merged models while achieving higher accuracy.
Error Analysis: Using an external judge (Claude-Haiku-4.5), the authors found that staged training reduced the rate of perception errors in reasoning traces from 857 to 781 (out of 3044 samples) compared to merged training.
Generalization: The benefits of staged training were confirmed across different architectures, including InternVL3-8B and InternVL3.5-8B.

5. Significance and Claims

The paper claims that the dominant limiting factor for visual reasoning in current VLMs is visual perception, not reasoning capability itself. Consequently, "longer thinking" cannot compensate for incorrect perception.

The significance of this work lies in:

Paradigm Shift: Moving away from merged training toward a capability-decoupled, staged approach.
Methodological Insight: Proving that the order of training matters; perception must be the scaffold for reasoning.
Training Efficiency: Demonstrating that better perception leads to more efficient reasoning (shorter traces), reducing computational costs.
Curriculum Design: Establishing capability-based staging as a new, orthogonal dimension to difficulty-based curricula that can be combined for additive gains.

The authors conclude that integrating specialized perception data with a staged training paradigm is essential for developing advanced, general-purpose VLMs that can reliably solve complex visual tasks.