DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning

DeepEyes is a Large Vision-Language Model that leverages reinforcement learning to natively learn "thinking with images" through active perception, enabling it to strategically ground reasoning in visual information without requiring pre-collected reasoning data and achieving significant improvements in perception, reasoning, and grounding tasks.

The Gist

DeepEyes: Teaching AI to "Look Before It Leaps"

Imagine you are taking a difficult math test, but instead of just reading the question, you are allowed to use a magnifying glass to inspect the diagram, zoom in on tiny numbers, or even rotate the paper to see it from a different angle. Most current AI models are like students who are forced to answer only by reading the text, even if the answer is hidden in a tiny detail of a picture. They often guess based on what they think the text should say, rather than what the image actually shows.

DeepEyes is a new AI model that learns to stop guessing and start looking. It learns to "think with images" by actively zooming in, cropping, and examining parts of a picture just like a human would.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Blind" Genius

Current AI models (Vision-Language Models) are incredibly smart at reading and talking. They can describe a picture perfectly. But when it comes to reasoning about the picture, they often act like a person reading a map in the dark. They rely on their memory of what maps usually look like rather than actually looking at the specific details in front of them.

• The Analogy: Imagine a detective trying to solve a crime. A standard AI is like a detective who reads the police report and guesses who the culprit is based on the description. DeepEyes is the detective who puts on a magnifying glass, walks over to the window, and actually looks for the fingerprints.

2. The Solution: Active Perception (The "Magnifying Glass" Skill)

DeepEyes doesn't just stare at the whole image. It has a special ability called Active Perception.

• How it works: As the AI thinks through a problem, it can decide, "Wait, I can't see the clock clearly from here. I need to zoom in on the bookshelf."
• The Magic: It generates a "crop" (a zoomed-in picture) of that specific area, adds it to its memory, and continues thinking. It can do this multiple times, building a chain of thought that mixes text and zoomed-in images.

3. How It Learned: The "Video Game" Method

Usually, to teach an AI to do something complex, humans have to write out thousands of examples of "Step 1: Look here, Step 2: Say this." This is slow and expensive.

DeepEyes learned differently. The researchers used Reinforcement Learning, which is like training a dog or a video game character:

• No Cheat Sheets: They didn't give the AI a pre-written script.
• The Reward System: They let the AI try to solve problems on its own.
  • If it guessed the answer without looking, it got a low score.
  • If it guessed correctly after zooming in and checking the details, it got a bonus reward.
• The Result: Over time, the AI realized that "looking closer" was the key to winning. It learned to use its "magnifying glass" instinctively, just like a human learns to squint when reading small print.

4. The Evolution: From Clumsy to Master

The paper describes a fascinating journey of how the AI learned:

1. Stage 1 (The Clumsy Explorer): At first, the AI was confused. It would zoom in randomly, like a toddler pressing every button on a remote control. It didn't know where to look.
2. Stage 2 (The Over-Enthusiast): Then, it realized zooming helped! But it went too far. It would zoom in on everything, asking for too many pictures, wasting time.
3. Stage 3 (The Master Detective): Finally, it became efficient. It learned to look at the whole picture, realize exactly which tiny detail was missing, zoom in once on that specific spot, and solve the problem. It learned to be strategic.

5. Why This Matters

This isn't just about solving math problems or finding clocks in living rooms. It changes how AI interacts with the world:

• Fewer Hallucinations: AI often "hallucinates" (makes things up) because it relies on text patterns. By forcing the AI to look at the visual evidence, it stops making things up.
• Better Reasoning: It can handle complex charts, tiny text in high-resolution photos, and tricky logic puzzles that require comparing different parts of an image.
• Human-Like Thinking: It mimics how humans solve visual problems: we don't just stare; we scan, we focus, and we verify.

In a Nutshell

DeepEyes is an AI that learned to stop guessing and start investigating. Instead of being a passive reader of images, it became an active explorer, using a "magnifying glass" to find the truth hidden in the details. And the best part? It taught itself how to do this through trial and error, without needing humans to write out a manual for every single step.

Technical Summary

1. Problem Statement

Current Large Vision-Language Models (VLMs) excel at multimodal understanding but struggle to deeply integrate visual information into their reasoning processes.

• The Limitation: Most existing models rely primarily on text-based Chain-of-Thought (CoT) reasoning. While they can process images, their "thought process" remains confined to the language modality, often leading to hallucinations, poor performance on high-resolution tasks, and an inability to perform "active perception" (e.g., zooming in to inspect details) dynamically.
• The Gap: Human cognition naturally combines vision and reasoning by sequentially fixating on visual details to make decisions. Previous attempts to mimic this using predefined workflows or external tools suffer from suboptimal performance due to modular decoupling and the need for extensive, hard-to-collect supervised fine-tuning (SFT) data for cold starts.
• Goal: To create a model that natively learns to "think with images" by dynamically interleaving visual perception (zooming/cropping) with textual reasoning, without relying on external specialized models or pre-collected reasoning datasets.

2. Methodology: DeepEyes

DeepEyes is a unified multimodal large language model trained end-to-end using Reinforcement Learning (RL) to incentivize Active Perception.

Core Mechanism: Interleaved Multi-modal Chain-of-Thought (iMCoT)

Instead of a static pipeline, DeepEyes operates in an agentic framework where the model autonomously decides, at each reasoning step, whether to:

1. Generate a text-based reasoning step.
2. Answer the question directly.
3. Perform Active Perception: Generate bounding box coordinates to crop a specific region of the original image, append the cropped image to the context, and continue reasoning.

This creates an iMCoT trajectory where text and visual observations are seamlessly interleaved.

Training Strategy

• No Cold-Start SFT: Unlike previous methods requiring SFT on reasoning trajectories, DeepEyes is trained directly from question-answer pairs using Group Relative Policy Optimization (GRPO).
• Reward Design: The model is guided by a composite reward function:
  • $R_{acc}$: Accuracy reward (correct final answer).
  • $R_{format}$: Format reward (structured output).
  • $R_{tool}$ (Conditional Bonus): A crucial component that grants a bonus only if the model answers correctly AND has performed at least one active perception step. This incentivizes the model to use visual grounding only when it leads to a correct solution, preventing unnecessary tool calls.
• Data Curation: To bootstrap RL without SFT, the authors curated a dataset combining:
  • V*: For fine-grained visual grounding.
  • ArxivQA: For chart and diagram diversity.
  • ThinkLite-VL: For challenging reasoning tasks.
  • Filtering: A multi-stage filter removes trivial or overly difficult samples and retains only those solvable via active perception with ground-truth regions.

3. Key Contributions

1. Native "Thinking with Images": DeepEyes demonstrates that the ability to dynamically zoom and reason can emerge natively through end-to-end RL, eliminating the need for external tools or specialized SFT data.
2. Active Perception Mechanism: The paper introduces a tailored data selection and reward strategy that successfully incentivizes the model to strategically ground its reasoning in visual information.
3. Discovery of RL Training Dynamics: The authors observe a distinct three-stage evolution in the model's behavior during training:
   • Stage 1 (Exploration): Ineffective, random zooming.
   • Stage 2 (High-Frequency Engagement): Over-querying the environment to boost rewards.
   • Stage 3 (Efficient Exploitation): Selective, precise zooming only when necessary, mirroring human cognitive efficiency.
4. Emergent Reasoning Patterns: The model naturally develops human-like reasoning patterns, including:
   • Visual Search: Scanning regions for small/hard-to-recognize objects.
   • Visual Comparison: Iteratively zooming to compare different parts of an image.
   • Visual Confirmation: Zooming to resolve uncertainty.
   • Hallucination Mitigation: Using visual evidence to override linguistic priors.

4. Experimental Results

DeepEyes (based on a 7B parameter Qwen2.5-VL backbone) was evaluated against state-of-the-art proprietary (GPT-4o, o3) and open-source models.

• High-Resolution Benchmarks:
  • V* Bench: Achieved 90.1% accuracy, a +18.9% improvement over the baseline Qwen2.5-VL (7B).
  • HR-Bench-8K: Improved by 7.3% over the baseline.
  • Significance: Proves that simple RL can unlock high-resolution reasoning without complex pipelines.
• General Perception & Reasoning (MME-RealWorld-Lite): Outperformed both 7B and 32B versions of Qwen2.5-VL, demonstrating superior real-world capability.
• Grounding & Hallucination:
  • Improved grounding accuracy on refCOCO/ReasonSeg.
  • Significantly reduced hallucinations by forcing the model to re-ground assumptions with visual evidence (verified via relevancy maps).
• Mathematical Reasoning: Showed consistent gains on benchmarks like MathVista and WeMath, indicating that visual grounding aids abstract reasoning.
• Zero-Shot Generalization: The model successfully adapted to a new "rotate" tool via system prompt alone, showing extensibility.

5. Significance and Impact

• Paradigm Shift: DeepEyes moves beyond static "image-to-text" processing toward dynamic "image-in-the-loop" reasoning, bridging the gap between human visual cognition and AI.
• Efficiency: By removing the need for cold-start SFT and external tool dependencies, the approach is more scalable and easier to deploy.
• Interpretability: The emergence of distinct training stages and reasoning patterns (search, compare, confirm) provides a transparent view into how RL shapes multimodal reasoning, offering a blueprint for future interpretable AI systems.
• Scalability: The framework scales effectively with model size (7B to 32B), with larger models exhibiting more sophisticated emergent behaviors and longer, more accurate reasoning chains.

In conclusion, DeepEyes establishes that active perception is not just a tool but a fundamental reasoning capability that can be learned end-to-end, significantly advancing the state of multimodal reasoning and reducing hallucination in high-resolution visual tasks.