Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm

1. Problem Statement

Current reasoning paradigms in Artificial Intelligence, specifically "Thinking with Text" (Chain-of-Thought in LLMs) and "Thinking with Images" (outputting images in CoT for VLMs), face inherent limitations:

• Static Constraints: Images capture single moments and fail to represent dynamic processes, temporal changes, or continuous transformations (e.g., drawing a path, simulating light reflection).
• Modality Separation: Text and vision are often treated as distinct modalities, hindering a unified framework for multimodal understanding and generation.
• Lack of Dynamic Reasoning: Existing models struggle to visualize dynamic processes or perform "mental simulation" required for complex spatial and logical tasks.

The authors propose "Thinking with Video" as a new paradigm, leveraging video generation models (specifically Sora-2) to use video frames as a unified medium for reasoning, enabling dynamic visualization and natural alignment with human cognitive processes involving imagination.

2. Methodology

A. The Benchmark: VideoThinkBench

To evaluate this paradigm, the authors constructed VideoThinkBench, a comprehensive benchmark covering 4,149 test samples across two main categories:

1. Vision-Centric Tasks: Solved primarily through reasoning about visual elements via drawing and imagination. These are further divided into a hierarchy of reasoning abilities:
   • Geometric Intuition: Eyeballing Puzzles (Point, Line, Shape tasks like ray intersection, reflection, and shape completion).
   • Visual Pattern Induction: Visual Puzzles (Symmetry, Gradient, Compositionality).
   • Abstract Rule Induction: ARC-AGI-2 (few-shot abstract grid transformations).
   • Spatial Planning & Search: Mazes (Square, Hexagon, Circle).
2. Text-Centric Tasks: Solved through text-based reasoning but adapted for video generation. The model must display the written solution process and speak the answer within the generated video.
   • Text-Only: GSM8K, MATH-500, AIME, MMLU, GPQA.
   • Multimodal: MathVista, MathVision, MMBench, MMMU.

B. Evaluation Protocol

• Input: A text prompt and a reference image containing the problem.
• Output: A generated video showing the solution process (e.g., writing on a whiteboard, drawing lines) and an audio track stating the final answer.
• Metrics:
  • Vision-Centric: Automated evaluation based on the final frame (Last Frame), majority vote across sampled frames (Major Frame), and audio transcription.
  • Text-Centric: Evaluation of the last video frame (visual text) and audio transcription using an LLM-as-a-Judge (GPT-4o) to verify correctness against ground truth.

C. Analysis Experiments

The authors conducted several experiments to understand the source of Sora-2's capabilities:

• Few-Shot Learning: Testing performance on ARC-AGI-2 with varying numbers of examples (1-shot vs. few-shot).
• Self-Consistency: Aggregating results from multiple generation attempts (Major Frame voting) to mitigate noise.
• Data Leakage Check: Creating derived math problems with different numerical values to ensure performance wasn't due to memorization.
• Prompt Rewriter Analysis: Using the open-source Wan 2.5 model (with a toggleable prompt rewriter) to isolate whether text reasoning comes from the video generator or an internal text-processing module.

3. Key Contributions

1. Proposal of "Thinking with Video": A new reasoning paradigm where video generation models act as unified multimodal reasoners, capable of dynamic simulation (drawing, movement) and embedding text within visual frames.
2. VideoThinkBench: The first systematic benchmark designed to evaluate video generation models on both vision-centric (spatial/dynamic) and text-centric (logical/mathematical) reasoning tasks, featuring verifiable and programmatically generated samples.
3. Systematic Comparison: A comprehensive evaluation of Sora-2 against State-of-the-Art (SOTA) Vision-Language Models (VLMs) like GPT-5, Claude Sonnet 4.5, and Gemini 2.5 Pro.
4. Insight into Mechanisms: Identification that video models can achieve high reasoning accuracy through self-consistency (test-time scaling) and that their text reasoning capabilities may stem from internal prompt rewriting mechanisms rather than the video generation component itself.

4. Key Results

A. Vision-Centric Performance

• Sora-2 vs. VLMs: Sora-2 is generally comparable to SOTA VLMs on vision-centric tasks.
• Eyeballing Puzzles: Sora-2 surpassed GPT-5 by 10% (40.2% vs. 29.7% using Major Frame evaluation), demonstrating superior spatial reasoning and the ability to "draw" solutions (e.g., tracing light paths).
• Visual Puzzles: Sora-2 showed competitive performance on symmetry tasks (comparable to Claude Sonnet 4.5) and demonstrated inductive reasoning on gradients and compositionality.
• ARC-AGI-2: Sora-2 achieved 1.3% accuracy (similar to VLMs), but manual analysis showed it often grasped the core rule (14% "Mostly Correct") even if it failed strict pixel-perfect execution.
• Mazes: Sora-2 solved 40% of square mazes but failed completely on hexagon and circle mazes, indicating limitations in complex spatial planning.

B. Text-Centric Performance

• Surprising Capability: Sora-2 achieved 98.9% accuracy on GSM8K and 92.0% on MATH-500 (via audio evaluation), rivaling GPT-5 and Claude.
• Multimodal Reasoning: Achieved 69.2% on MMMU and 75.7% on MathVista.
• Discrepancy: Audio accuracy was consistently higher than video (written text) accuracy, suggesting the model struggles to generate legible, coherent written text in video frames but can "think" the logic correctly.

C. Analysis Findings

• Few-Shot Learning: Sora-2 is a strong few-shot learner; performance on ARC-AGI-2 improved significantly when provided with more examples.
• Self-Consistency: Aggregating multiple generations (Major Frame voting) boosted performance from 56% to 90% on specific visual puzzles, revealing the potential of test-time scaling in video reasoning.
• Source of Text Reasoning: Experiments with Wan 2.5 revealed that disabling the prompt rewriter dropped accuracy to near zero. This suggests Sora-2's text reasoning likely originates from an internal prompt rewriter that translates complex problems into step-by-step visual instructions, rather than the video generation model performing the logic itself.

5. Significance

• Unified Paradigm: The paper argues that video generation models have the potential to unify multimodal understanding and generation, moving beyond the separation of text and vision.
• Dynamic Reasoning: It highlights the unique advantage of video models in handling dynamic processes (e.g., simulating physics, drawing paths) that static images cannot capture.
• Future Directions: The work suggests that Reinforcement Learning with Verifiable Rewards (RLVR) on video tasks could further enhance reasoning. It also proposes converting text corpora into video-form training data (simulating handwriting) to pretrain video models on world knowledge.
• Limitations: The study is limited by the closed-source nature of Sora-2, preventing deep internal mechanism analysis, and the current inability of video models to generate perfectly legible text without prompt rewriting assistance.

In conclusion, "Thinking with Video" represents a promising shift where video generation models serve not just as content creators, but as general-purpose multimodal reasoners capable of dynamic simulation and unified understanding.