💬 NLP

MiMo-Embodied: X-Embodied Foundation Model Technical Report

The paper introduces MiMo-Embodied, the first open-source cross-embodied foundation model that achieves state-of-the-art performance in both autonomous driving and embodied AI by leveraging multi-stage learning, curated data, and CoT/RL fine-tuning to demonstrate strong positive transfer between these two domains.

Original authors: Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, Yuchen Zhang, Jing Wu, Jinghui Lu, Chenxu Dang, Jiayi Guan, Jianhua Wu

Published 2026-04-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, Yuchen Zhang, Jing Wu, Jinghui Lu, Chenxu Dang, Jiayi Guan, Jianhua Wu, Zhiyi Hou, Hanbing Li, Shumeng Xia, Mingliang Zhou, Yinan Zheng, Zihao Yue, Shuhao Gu, Hao Tian, Yuannan Shen, Jianwei Cui, Wen Zhang, Shaoqing Xu, Bing Wang, Haiyang Sun, Zeyu Zhu, Yuncheng Jiang, Zibin Guo, Chuhong Gong, Chaofan Zhang, Wenbo Ding, Kun Ma, Guang Chen, Rui Cai, Diyun Xiang, Heng Qu, Fuli Luo, Hangjun Ye, Long Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two very different students in a classroom.

Student A is a Robot Butler. They are great at navigating a messy kitchen, figuring out which part of a cup you can hold, and planning how to fold a shirt without wrinkling it. They are experts at "Embodied AI" (interacting with objects in a room).
Student B is a Self-Driving Car. They are experts at watching the road, predicting if a pedestrian will jaywalk, and deciding whether to brake or turn left at a busy intersection. They are experts at "Autonomous Driving."

Usually, schools teach these students separately. The Robot Butler doesn't know how to drive, and the Self-Driving Car doesn't know how to pick up a spoon.

Enter MiMo-Embodied.

This paper introduces a new "Super Student" created by Xiaomi. It is the first open-source model that successfully combines the brains of the Robot Butler and the Self-Driving Car into a single package. Think of it as a Swiss Army Knife that is just as good at opening a bottle of wine as it is at navigating a highway.

How Did They Build This Super Student?

The researchers didn't just mash the two together; they used a four-step training recipe to make sure the student learned everything without getting confused:

Step 1: The "Room" Class: First, they taught the model all about rooms, objects, and how to move around inside a house. They used data about robots picking things up and understanding where things are in 3D space.
Step 2: The "Road" Class: Next, they took that same student and put them in a driving simulator. They taught them about traffic lights, other cars, and how to predict what a truck ahead might do.
Step 3: The "Think Aloud" Class: This is the secret sauce. Instead of just giving the answer, the model was taught to think out loud (Chain-of-Thought). Before saying "Turn left," it learns to say, "I see a red light, so I must stop. I see a car turning, so I must wait." This helps the model understand why it's making a decision.
Step 4: The "Coach" Class: Finally, they used a reinforcement learning technique (like a coach giving points for good plays and penalties for bad ones). This fine-tuned the model to be extra precise, ensuring it doesn't just guess, but calculates the safest and most logical move.

What Can This Super Student Do?

The paper tested MiMo-Embodied on 29 different challenges (benchmarks) and found it was the best at almost all of them, beating both specialized robots and specialized cars, as well as big commercial AI models.

In the "Robot Butler" world (Embodied AI):

Affordance Prediction: If you show it a picture of a chair, it doesn't just see "chair." It sees "something you can sit on" and "something you can grab the arm of." It knows exactly where to touch an object to pick it up.
Task Planning: If you say, "I'm hungry, get me a snack," it can break that down: Go to the kitchen -> Open the fridge -> Find the apple -> Bring it here.
Spatial Understanding: It knows that the "window" is far away, the "table" is close, and the "cat" is under the chair. It can navigate a room without bumping into things.

In the "Self-Driving Car" world (Autonomous Driving):

Environmental Perception: It can look at a chaotic street scene and instantly spot a red traffic light, a pedestrian stepping off the curb, and a truck parked on the side.
Status Prediction: It can guess what other drivers are thinking. "That car is slowing down; they probably want to turn." "That pedestrian is waiting; they are about to cross."
Driving Planning: It doesn't just drive; it drives safely. It can plan a smooth lane change or a gentle stop, explaining its reasoning like a human driver would.

The Big Discovery: "Cross-Training" Works

The most exciting part of the paper is the discovery that learning one skill helps the other.

Usually, people thought a model trained on driving data would forget how to pick up a cup, and a model trained on robots would forget how to drive. But MiMo-Embodied proved that these skills actually reinforce each other.

Learning to predict where a car will go helps the model understand how objects move in a room.
Learning to figure out how to grab a handle helps the model understand the physical constraints of a steering wheel.

The Bottom Line

MiMo-Embodied is a "foundation model" that acts as a universal brain for physical machines. Whether the machine is a robot vacuum, a humanoid robot, or a self-driving car, this single model can understand the world, plan its actions, and execute them safely.

The paper claims this is a major step forward because it breaks down the wall between "indoor robots" and "outdoor cars," showing that a single, unified AI can handle the complexity of the entire physical world. They have even made the code and the model available for anyone to use and study.

Technical Summary: MiMo-Embodied

1. Problem Statement

The current landscape of Vision-Language Models (VLMs) is characterized by a significant domain gap between Embodied AI (indoor robotics, manipulation, navigation) and Autonomous Driving (outdoor traffic, vehicle control). Existing models are typically specialized for one domain, leading to:

Lack of Unified Models: No single open-source VLM bridges the capabilities required for both indoor physical interaction and outdoor driving.
Limited Cross-Domain Generalization: Specialized models fail to transfer spatial understanding and reasoning skills across diverse indoor and outdoor environments.
Incomplete Evaluation: Existing benchmarks assess capabilities in isolation, lacking a comprehensive framework to evaluate cross-embodiment performance.

MiMo-Embodied addresses these challenges by introducing the first open-source, cross-embodied foundation model that unifies Autonomous Driving and Embodied AI, aiming to achieve state-of-the-art (SOTA) performance in both domains simultaneously.

2. Methodology

2.1 Model Architecture

MiMo-Embodied is built upon the MiMo-VL architecture, consisting of three core components:

Vision Transformer (ViT): Utilizes the MiMo-VL encoder to process high-resolution inputs, including single images, multi-image sequences, and videos.
Projector: A Multi-Layer Perceptron (MLP) that maps visual tokens into a latent space aligned with the Large Language Model (LLM).
Large Language Model (LLM): The reasoning engine responsible for text understanding, logical inference, and generating coherent responses.

The model supports diverse input modalities (images, videos) and outputs text-based reasoning, action plans, and coordinate predictions.

2.2 Training Dataset Construction

The authors curated a massive, diverse dataset spanning three categories:

General Dataset: Derived from MiMo-VL's corpus, including high-res images, videos, and synthetic reasoning data to establish foundational visual-language understanding.
Embodied AI Dataset: Focused on three core capabilities:
- Affordance Prediction: Sourced from PixMo-Points, RoboAfford, and RoboRefIt for fine-grained localization and interaction reasoning.
- Task Planning: Integrated Cosmos-Reason1, EgoPlan-IT, and RoboVQA for long-horizon reasoning and action sequencing.
- Spatial Understanding: Utilized SQA3D, VLM-3R, RefSpatial, and EmbSpatial for 3D spatial reasoning and navigation.
Autonomous Driving Dataset: Structured around three functional modules:
- Environmental Perception: General scene understanding and regional object localization (e.g., CODA-LM, DriveLM, nuScenes-QA).
- Status Prediction: Intent prediction and multi-agent interaction modeling.
- Driving Planning: Action decision-making and explainable reasoning (e.g., BDD-X, NuInstruct).

2.3 Four-Stage Training Strategy

To achieve synergistic learning without catastrophic forgetting, the team employed a progressive four-stage training strategy:

Stage 1 (Embodied AI SFT): Fine-tuning on general data + Embodied AI datasets to establish core affordance, planning, and spatial reasoning.
Stage 2 (Autonomous Driving SFT): Introducing mixed supervision from Autonomous Driving data to integrate cross-domain understanding (perception, prediction, planning).
Stage 3 (Chain-of-Thought SFT): Enhancing reasoning transparency by training on generated CoT rationales, forcing the model to break down complex problems into sequential logical steps.
Stage 4 (Reinforcement Learning Fine-Tuning): Applying Group Relative Policy Optimization (GRPO) to refine performance on specific tasks. This stage uses task-specific reward signals (e.g., exact match for multiple-choice, IoU for spatial grounding) to optimize correctness and reliability.

3. Key Contributions

First Unified Cross-Embodied Model: MiMo-Embodied is the first open-source VLM to successfully integrate and excel in both Autonomous Driving and Embodied AI tasks.
Comprehensive Benchmarking: The authors established a rigorous evaluation framework covering 17 Embodied AI benchmarks and 12 Autonomous Driving benchmarks, providing a holistic view of cross-embodiment capabilities.
Novel Training Paradigm: The proposed multi-stage training strategy (General $\to$ Specialized $\to$ CoT $\to$ RL) demonstrates that positive transfer exists between indoor and outdoor embodied tasks, allowing mutual reinforcement rather than interference.
Open-Source Release: The model, code, and training data strategies are publicly available, fostering further research in unified embodied intelligence.

4. Results

4.1 Quantitative Performance

MiMo-Embodied (7B parameters) achieved State-of-the-Art (SOTA) performance across the board, outperforming:

General VLMs: Including GPT-4o, Claude-Sonnet-4, Gemini-2.5-Pro, and Qwen-VL-Max.
Specialized Models: Surpassing dedicated Embodied models (e.g., RoboBrain-2.0, VeBrain) and Autonomous Driving models (e.g., DriveLMM-o1, RoboTron-Drive).

Key Metrics:

Embodied AI: Top scores on 17 benchmarks, particularly in Affordance Prediction (e.g., 65.50 on Part-Afford, 69.81 on RoboAfford-Eval) and Spatial Understanding (88.82 on CV-Bench).
Autonomous Driving: Leading performance on 12 benchmarks, including Perception (58.55 on CODA-LM, 76.14 on DRAMA) and Planning (83.58 on NuInstruct, 80.99 on DriveAction).
NAVSIM Benchmark: In trajectory planning, MiMo-Embodied achieved a Predictive Driver Model Score (PDMS) of 91.0, outperforming larger models like InternVL3-8B and ReCogDrive.

4.2 Qualitative Performance

Real-World Navigation: Successfully executed complex household navigation tasks (e.g., "find the vacuum cleaner in the canteen") with precise spatial localization, outperforming baselines in identifying target objects and handling plural queries.
Manipulation: Demonstrated superior affordance reasoning, correctly identifying graspable parts (handles, lids) and placement zones in cluttered scenes.
Driving Scenarios: Generated safe, interpretable driving decisions for complex scenarios like intersection turns, U-turns, and overtaking, providing explicit reasoning for safety-critical actions.

4.3 Ablation Studies

Experiments confirmed that the multi-stage training strategy is critical. Models trained only on single domains suffered performance drops in the other. Directly mixing data without staged learning led to slight performance degradation in driving tasks. The proposed progressive approach yielded the best balance, improving Embodied performance by 4% and Driving performance by 8.1% over a simple mixed-data baseline.

5. Significance

MiMo-Embodied represents a pivotal shift in embodied AI research by demonstrating that indoor and outdoor physical intelligence are not mutually exclusive but mutually reinforcing.

Unified Intelligence: It proves that a single foundation model can master the nuances of robot manipulation and vehicle navigation, suggesting a path toward general-purpose embodied agents.
Safety and Explainability: The integration of Chain-of-Thought and RL fine-tuning ensures that the model's decisions in safety-critical domains (like driving) are not only accurate but also interpretable and logically sound.
Community Impact: By open-sourcing the model and its training methodology, MiMo-Embodied lowers the barrier for developing unified embodied systems, accelerating progress in both robotics and autonomous driving industries.

In conclusion, MiMo-Embodied sets a new standard for cross-embodied foundation models, validating that a unified approach to physical world interaction yields superior generalization and performance compared to specialized silos.