💻 computer science

UniVR: Thinking in Visual Space for Unified Visual Reasoning

The paper introduces UniVR, a novel framework that leverages the VR-GRPO reinforcement learning paradigm to enable unified visual reasoning, physical dynamics understanding, and long-term planning directly from raw visual data, achieving significant performance gains on the newly constructed VR-X benchmark without relying on text or task-specific heuristics.

Original authors: Zhongwei Ren, Yunchao Wei, Yao Zhao, Weibo Gong, Xiao Liu, Anran Wang, Xiangtai Li, Xiaojie Jin

Published 2026-07-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhongwei Ren, Yunchao Wei, Yao Zhao, Weibo Gong, Xiao Liu, Anran Wang, Xiangtai Li, Xiaojie Jin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a robot how to tie a shoelace, fold a fitted sheet, or solve a tricky maze. For a long time, the smartest way to do this was to write a giant, detailed instruction manual in English (or any language) and hope the robot could read it, understand the physics, and then try to draw the steps.

But here's the twist: UniVR suggests that maybe we've been overthinking it. Instead of forcing the robot to translate the world into words first, why not let it just think in pictures?

The Problem with "Thinking in Words"

Current super-smart AI models are like brilliant librarians who have read every book in the world. They can tell you how to tie a knot in perfect grammar. But when they try to actually do it, they often trip over the physics. They might say, "Now, pull the string tight," but in their mental movie, the string might pass right through the table or the knot might magically untie itself.

The paper argues that relying on text to describe the real world is like trying to describe the taste of a strawberry using only math equations. You get the data, but you miss the messy, dynamic reality. Even the best models, when asked to plan a long sequence of actions (like cooking a meal or navigating a room), often produce videos where the laws of physics break down, the objects disappear, or the logic just doesn't hold up.

The Solution: UniVR and the "Visual Gym"

Enter UniVR. Think of this as a new training camp for AI where the students aren't allowed to speak; they can only learn by watching and doing.

Instead of reading a manual, UniVR learns directly from raw video demonstrations. It watches thousands of clips of people tying knots, folding clothes, or solving puzzles. It doesn't need a teacher to say, "Step 1: Grab the string." It just figures out the pattern by watching the visual flow.

To make sure the AI doesn't just guess randomly, the researchers built a special training method called VR-GRPO. Imagine a strict but fair coach who watches the AI practice.

The Global Coach: Checks if the robot actually finished the task (e.g., "Did the knot get tied?").
The Step-Focal Coach: This is the secret sauce. The Global Coach might miss small mistakes, like a hand slipping or a ball rolling the wrong way in the middle of the action. The Step-Focal Coach zooms in on those tricky moments. If the AI's "mental movie" gets wobbly or illogical in the middle, this coach catches it and says, "Whoa, that doesn't make sense physically! Try again."

This combination ensures the AI doesn't just get lucky at the end; it has to be logical and physically consistent every single step of the way.

The Big Test: VR-X

To see if this actually works, the team built a massive obstacle course called VR-X. It's like a giant video game level with 16 different types of challenges, from long, complex tasks like robotic arm manipulation and cooking, to quick brain teasers like visual puzzles and searching for objects.

The results? UniVR didn't just pass; it sprinted.

On this new benchmark, UniVR improved performance by up to 25% compared to previous methods.
Even though it's a relatively small model (with 34 billion parameters), it managed to beat the performance of much larger, text-heavy systems like Gemini 3 Pro combined with other tools on long-term planning tasks.
It didn't just get better at the tasks; it also got better at understanding images in general, scoring higher on standard vision tests like MMMU and MME.

What This Means (and What It Doesn't)

The paper is very clear about what this is and isn't.

It is NOT a magic wand that solves every AI problem. The authors explicitly state that current models still struggle with fine-grained physics if they rely only on text.
It IS a strong proof that AI can learn complex reasoning directly from visual data without needing a constant stream of text instructions.
The results are measured and proven on their specific benchmark (VR-X) and several existing public tests. The authors suggest that this "thinking in visual space" approach is a powerful way to build better world models, but they don't claim it's the final answer to all of artificial intelligence.

In short, UniVR shows that sometimes, the best way to learn how the world works isn't to read a book about it, but to jump in, watch closely, and figure out the rules by seeing how things move and change. It's a step toward AI that doesn't just talk about the world, but truly sees it.

Technical Summary: UniVR – Thinking in Visual Space for Unified Visual Reasoning

1. Problem Statement

Current artificial intelligence models predominantly acquire world knowledge and perform reasoning within a textual space. While Large Multimodal Models (LLMs) and Generative models have advanced significantly, they rely on text as an abstract representation of the world. This abstraction fails to fully capture the rich, continuous information of the real visual world, such as complex physical dynamics, fine-grained spatial relationships, and underlying physical laws.

Existing approaches to visual reasoning face two primary limitations:

Text-Bounded Reasoning: Models like Gemini-3 or GPT-5 generate textual reasoning chains (Chain-of-Thought) which are then converted into visual traces via generative models. This paradigm struggles to maintain logical coherence and physical consistency in long-horizon tasks because textual abstractions cannot seamlessly transfer reasoning proficiency to the visual domain.
Dependency on Dense Supervision: Unified generation models and video generation strategies often rely heavily on dense image-text pairs and text guidance for training and inference. This dependency constrains the scaling of visual reasoning and limits the model's ability to learn directly from raw visual data without linguistic mediation.

The core inquiry of this work is: How can models utilize raw visual data to boost their visual reasoning capabilities across diverse tasks without relying on dense text supervision?

2. Methodology

UniVR Architecture
UniVR is a unified next-token prediction framework designed to reason and plan directly within visual space. It adopts an autoregressive generation model (specifically Emu3.5 as the baseline) that tokenizes images, text, and video into a unified discrete vocabulary. The framework operates via a two-stage training pipeline:

Cold Initialization: Supervised Fine-Tuning (SFT) on a curated dataset of diverse visual tasks (query image, instruction, visual reasoning trajectory) to standardize the input format and provide visual reasoning priors.
Reinforcement Learning (RL): A novel RL paradigm called VR-GRPO (Visual Reasoning Group Relative Policy Optimization) is applied to refine the model's reasoning capabilities.

VR-GRPO: The Core Innovation
VR-GRPO is designed to optimize for logical coherence and physical consistency without requiring task-specific heuristics or image-text pairs. It introduces a dual-reward mechanism:

Global Reward ( $R_g$ ): A holistic assessment of the generated sequence regarding task completion and visual quality, provided by a Vision-Language Model (VLM) evaluator.
Step-Focal Reward ( $R_s$ ): A novel component addressing the limitation of global rewards, which often overlook intermediate physical violations or logical gaps in long sequences.
- Mechanism: The system calculates inter-trajectory variance using CLIP image encoders to identify time steps with maximum uncertainty (where reasoning paths diverge).
- Targeting: It extracts a specific sub-segment around these high-variance steps and prompts the VLM to perform a pairwise comparison to assess physical laws and trajectory progression.
- Integration: The final reasoning reward is calculated as $R_{reason} = R_g - \lambda|R_g - R_s|$ . This formulation penalizes the model if it achieves a good final state ( $R_g$ ) but fails to maintain procedural integrity in critical sub-steps ( $R_s$ ), effectively preventing "reasoning shortcuts."

VR-X Benchmark
To train and evaluate UniVR, the authors constructed VR-X, the first large-scale benchmark for diverse visual reasoning. It comprises 1.5M raw samples from 16 sources, curated into 310k cold-start training samples, 3k RL samples, and 1.8k benchmark evaluation samples. The benchmark covers:

Long-term Planning: Minute-scale tasks involving fine-grained dynamics (e.g., tying knots, folding clothes, robotic manipulation, cooking).
General Visual Reasoning: Cognitive skills including visual search, puzzles, spatial perception, and editing.
Evaluation Metrics:
- VLM Score: Automated assessment of task completion, procedural coherence, and image fidelity.
- JEPA Similarity: A metric using V-JEPA encoders to measure alignment with real-world physical dynamics, addressing the hallucination issues of text-dependent evaluators.

3. Key Results

Performance on VR-X
UniVR demonstrates significant improvements over state-of-the-art baselines on the VR-X benchmark:

Overall Performance: UniVR achieves a 25.2% improvement over the base Emu3.5 model and a 20.9% improvement in long-term planning tasks.
Comparison with Pipelines: Despite having only 34B parameters, UniVR approaches the performance of the Gemini 3 Pro + Nano Banana 2 pipeline and surpasses Gemini 3 in long-horizon manipulation tasks.
Physical Consistency: UniVR achieves a significantly lower JEPA score (indicating better alignment with physical laws) compared to text-based MLLMs and unified generation models, which often exhibit large feature distribution divergences from ground truth.

Generalization to Multimodal Understanding
The paper reports that enhancing visual reasoning via UniVR also boosts performance on standard multimodal understanding benchmarks (MMMU, MME, MMBench, MathVista, MM-Vet).

UniVR shows gains across all six evaluated metrics compared to the base Emu3.5, suggesting that native visual reasoning serves as a powerful complement to textual supervision.
Training solely on textual reasoning chains derived from VR-X yielded negligible gains, reinforcing the necessity of visual-space training.

Ablation Studies

Reward Design: Combining global and step-focal rewards yields synergistic effects. Using only global rewards led to degradation in long-term planning (reward hacking), while step-focal rewards alone improved all metrics.
Joint Training: VR-GRPO enables the model to successfully learn heterogeneous tasks (long-term planning vs. general reasoning) jointly, whereas cold-start models struggle to balance these disparate knowledge domains.
Compatibility: VR-GRPO is compatible with existing text-based RL methods and aesthetic rewards (e.g., HPSv3), though the visual reasoning reward is the primary driver of performance.

4. Significance and Claims

The paper claims that UniVR represents a fundamental shift in how AI models acquire world knowledge:

Language-Free Visual Reasoning: It is the first investigation to simultaneously learn complex reasoning, fine-grained physical dynamics, and long-term planning from pure visual demonstrations without relying on dense image-text pairs or task-specific rules.
Intrinsic World Modeling: The results underscore that raw visual data alone can support sophisticated, long-horizon policy learning, challenging the notion that text is a necessary intermediary for complex reasoning.
Unified Capability: The framework successfully unifies heterogeneous tasks (from robotic manipulation to spatial puzzles) under a single visual protocol, overcoming the limitations of previous models that were confined to short-horizon dynamics or narrow single-task settings.
Broader Impact: The findings suggest that strengthening visual reasoning confers broader benefits to multimodal understanding, positioning vision-native computation as a vital complement to textual abstraction for building more grounded and efficient world models.

The authors emphasize that all code, data (VR-X), and models are open-sourced to facilitate further research in this domain.