💻 computer science

WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform

This paper introduces WorldArena 2.0, an expanded benchmark that systematically evaluates embodied world models across three new dimensions—visuotactile modality, interactive functionality for policy optimization, and diverse real-world and simulated platforms—to address the limitations of existing vision-only and simulator-based assessments.

Original authors: Yu Shang, Yinzhou Tang, Yiding Ma, Zhuohang Li, Lei Jin, Weikang Su, Xin Jin, Zhaolu Wang, Ziyou Wang, Xin Zhang, Haisheng Su, Weizhen He, Wei Wu, Haoyi Duan, Gordon Wetzstein, Xihui Liu, Dhruv Shah

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu Shang, Yinzhou Tang, Yiding Ma, Zhuohang Li, Lei Jin, Weikang Su, Xin Jin, Zhaolu Wang, Ziyou Wang, Xin Zhang, Haisheng Su, Weizhen He, Wei Wu, Haoyi Duan, Gordon Wetzstein, Xihui Liu, Dhruv Shah, Zhaoxiang Zhang, Zhibo Chen, Jun Zhu, Yonghong Tian, Tat-Seng Chua, Wenwu Zhu, Chen Gao, Yong Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to do chores, like pouring water or wiping a table. To do this safely and effectively, the robot needs a "World Model." Think of a World Model as the robot's internal dream machine. It's a simulation inside the robot's brain where it can "imagine" what will happen next if it moves its arm a certain way, without actually risking a broken vase or a spilled drink.

For a long time, scientists have been testing these dream machines, but their tests were a bit like judging a pilot's skills only by looking at a flat, 2D drawing of a plane. They only checked if the robot could "see" the future clearly, ignoring other senses, how well it could actually learn to fly, and whether it could handle real wind and rain.

WorldArena 2.0 is a brand new, much tougher "flight simulator" for testing these robot brains. The researchers from Tsinghua University and many other top institutions upgraded the test in three major ways:

1. Adding a "Sense of Touch" (Modality)

The Old Way: Previously, the robot's dream machine only had eyes. It could predict what a video of a moving object would look like, but it couldn't "feel" anything.
The New Way: WorldArena 2.0 gives the robot eyes and hands. It tests if the robot can predict not just the video, but also the tactile feedback (the feeling of pressure, slip, or texture).

The Analogy: Imagine trying to learn to juggle by only watching a video of someone else juggling. You might know where the balls look like they are going, but you won't know how hard to catch them. WorldArena 2.0 forces the robot to imagine the feeling of the ball hitting its hand, which is crucial for tasks like inserting a USB cable (where you need to feel the "click") or lifting a bottle without crushing it.

2. Turning the Dream Machine into a "Training Gym" (Functionality)

The Old Way: Before, researchers would ask the robot to "dream" a future, and then they would check if the dream looked realistic. It was a one-way street: the robot dreamed, and the humans graded the picture.
The New Way: Now, the robot's dream machine acts as a virtual training gym. The robot can practice its skills inside the dream, make mistakes, learn from them, and get better, all without touching the real world.

The Analogy: Instead of just watching a movie of a soccer player, the robot is now playing a video game where it can actually practice kicking the ball. The test checks: "If we let the robot train inside this dream for 10 hours, will it become a better player when it steps onto the real field?"

3. Testing in the "Real World" (Platform)

The Old Way: Most tests happened entirely inside a computer simulation. It was like testing a car only on a perfectly smooth, flat track in a video game.
The New Way: WorldArena 2.0 tests the robots in three different environments:

RoboTwin 2.0: A complex computer simulation with random obstacles.
LIBERO: A structured simulation that tests specific types of learning.
AgileX ALOHA: A real physical robot in a real room.

The Analogy: It's the difference between testing a car in a video game, on a test track, and finally driving it on a bumpy, rainy city street. The researchers found that while many robots were great at the video game level, they often struggled when they had to drive on the real, messy street. This highlights a big gap between "dreaming" perfectly and "doing" perfectly.

What Did They Find?

The researchers tested 12 different robot "dream machines" using these new rules.

The Good News: Some models learned to "feel" the world quite well. For example, one model (Wan 2.2) was so good at predicting tactile sensations that it could successfully perform a delicate task (inserting an HDMI cable) 100% of the time in the simulation.
The Bad News: There is still a huge gap between the simulation and reality. A robot that looks like a genius in the computer simulation often fails when asked to pour water on a real table. The "dream" isn't quite accurate enough to handle the messy physics of the real world yet.

The Bottom Line

WorldArena 2.0 is a more honest and comprehensive report card for robot intelligence. It stops us from just admiring pretty robot videos and starts asking the hard questions: Can the robot feel? Can it learn inside its own mind? And can it actually do the job when it's out in the real world?

The paper concludes that while we are making progress, we still have a long way to go before robots can reliably learn from their own "dreams" and apply that learning to real-life tasks.

Technical Summary: WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform

Problem Statement

World models have become a central paradigm for embodied intelligence, enabling agents to predict action-conditioned futures and reason about environmental dynamics. However, existing benchmarks for embodied world models (EWMs) suffer from three critical limitations that hinder the assessment of increasingly comprehensive models:

Modality Limitation: Prevailing benchmarks are confined to vision-only settings, overlooking the multimodal nature of embodied interactions where tactile feedback is essential for resolving contact-rich dynamics and physical friction.
Functionality Limitation: Downstream evaluations are largely restricted to open-loop planning or static policy assessment. They rarely investigate the capacity of a world model to serve as an interactive reinforcement learning (RL) environment that supports iterative policy improvement through imagined rollouts.
Platform Limitation: Benchmark results are obtained almost exclusively in simulation. It remains unclear whether strong performance in controlled virtual environments translates to real-world deployment, leaving the sim-to-real gap largely unexamined.

Methodology

To address these limitations, the authors introduce WorldArena 2.0, an expanded benchmark that systematically broadens EWM evaluation along three coordinated dimensions: modality, functionality, and platform.

1. Modality Extension: Visuotactile Evaluation

WorldArena 2.0 extends evaluation from vision-only to visuotactile modalities.

Standardized Pipeline: Built upon the UniVTAC simulator, the authors propose a standardized pipeline to upgrade vision-only world models into visuotactile models without architectural surgery. This involves three modules:
- A Tactile VAE that encodes tactile deformation maps and aligns them into the video latent space.
- A Visuotactile Two-Stream World Model that performs synchronized denoising for both video and tactile prediction.
- An Action Diffusion Head that infers future actions based on predicted visuotactile latents.
Evaluation Metrics: Models are assessed on tactile prediction quality (PSNR, SSIM) and functional success rates in downstream manipulation tasks (e.g., Insert HDMI, Lift Bottle).

2. Functionality Extension: World Models as RL Environments

The benchmark moves beyond offline prediction to evaluate world models as online interactive RL environments.

Framework: The authors formalize a closed-loop RL pipeline where the world model approximates the environment transition kernel ( $\hat{P}_\phi$ $\hat{P}_{ϕ}$ ). The framework comprises four components:
- World Model Environment: Predicts the next observation given current state and action.
- Reward Model: Predicts immediate rewards (tested with Proxy-based, VLM-based, and Similarity-based designs).
- Policy Model: Generates actions conditioned on observations.
- Optimization Module: Updates policy parameters (e.g., via GRPO) to maximize expected return.
Process: The world model replaces the simulator in traditional RL settings. Policies are trained via iterative interaction with the world model, and the resulting policies are deployed to measure success rates in real-world or high-fidelity simulation tasks.

3. Platform Extension: Cross-Embodiment Sim-to-Real

WorldArena 2.0 introduces a graded evaluation pipeline spanning three distinct platforms to assess cross-embodiment generalization:

RoboTwin 2.0: A scalable bimanual simulation environment with extensive domain randomization (tasks: Adjust Bottle, Click Bell).
LIBERO: A structured benchmark for single-arm manipulation with procedural knowledge shifts (task: Turn on the Stove).
AgileX Split-Type ALOHA: A real-world robotic testbed equipped with PiPER 6-DoF arms (tasks: Pour Water, Wipe Table).
Evaluation Protocols: Models are tested as Embodied Data Engines (generating synthetic data for policy training) and Embodied Action Planners (directly predicting closed-loop action sequences).

Key Results

The authors conducted extensive experiments on 12 state-of-the-art embodied world models across the defined platforms.

Visuotactile Performance:
- General-purpose world models (e.g., Wan2.2) demonstrated superior tactile prediction quality compared to specialized embodied models, achieving 100% success on the "Insert HDMI" task.
- However, long-horizon tasks like "Lift Bottle" revealed limitations; while a baseline ACT model achieved 80% success, world models dropped to 0%, indicating current models struggle with sustained force control over long horizons despite improved tactile perception.
RL Environment Utility:
- Policies trained using world models (e.g., WoVR, Ctrl-World) achieved performance comparable to ground-truth simulator training on short-horizon tasks (Click Bell) but showed varying results on long-horizon tasks.
- Among reward models, Proxy-based reward models (using ResNet backbones) proved most robust, outperforming VLM-based and similarity-based approaches which suffered from lack of fine-tuning or dependence on prediction accuracy.
Cross-Platform Correlations:
- Perceptual Quality: Visual quality, motion quality, and physics adherence showed strong correlations across simulation platforms (RoboTwin and LIBERO).
- Sim-to-Real Gap: A significant disconnect was observed between simulation and real-world performance. While simulation rankings correlated moderately, task success rates dropped drastically in real-world evaluations. Only a few models achieved non-zero success rates on real-world tasks, and all remained far below practical deployment requirements.
- Functional Transfer: Task success correlated positively between simulators but dropped significantly when compared to real-world performance, confirming that simulation-based metrics are not reliable proxies for real-world deployability.

Significance and Claims

The paper claims that WorldArena 2.0 provides a rigorous, standardized testbed that tracks progress toward truly general embodied world models. Its primary significance lies in:

Standardization: It pioneers standardized protocols for evaluating visuotactile world models and utilizing world models as online RL environments, areas previously lacking consistent benchmarks.
Reality Check: By introducing real-world robotic evaluation, the benchmark exposes a substantial "sim-to-real usability gap," demonstrating that high-fidelity simulation performance does not guarantee real-world competence.
Comprehensive Assessment: It moves the field beyond visual realism to assess physical grounding, interactive utility, and cross-platform robustness, providing a more realistic foundation for developing embodied intelligence.

The authors conclude that while current models show promise in capturing multimodal dynamics and supporting policy optimization, significant challenges remain in bridging the gap between simulated training and real-world physical execution.