💻 computer science

GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

The paper introduces GigaBrain-0.7, an embodied foundation model that leverages a novel three-system architecture and is trained on over 37,000 hours of heterogeneous data to achieve superior zero-shot generalization, instruction following, and task success rates across diverse robot embodiments compared to prior state-of-the-art models.

Original authors: GigaBrain Team, Angen Ye, Axiang Sun, Can Jin, Chenxi Cheng, Chong Shi, Dengke Shang, Dingqian Zhang, Guan Huang, Guangqiang Wang, Guangqing Ding, Guo Li, Hangcong Li, Hengyu Zhong, Hongtao Lu, Jianbo

Published 2026-08-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: GigaBrain Team, Angen Ye, Axiang Sun, Can Jin, Chenxi Cheng, Chong Shi, Dengke Shang, Dingqian Zhang, Guan Huang, Guangqiang Wang, Guangqing Ding, Guo Li, Hangcong Li, Hengyu Zhong, Hongtao Lu, Jianbo Qin, Jiming Mao, Jing Zhu, Jindi Lv, Jingzhi Cui, Junjie Xie, Junyi Bao, Kai Liu, Lei Yuan, Limin Long, Lv Feng, Mingming Yu, Peng Li, Pengfei Yi, Qi Li, Qianli Zhang, Qingfang Li, Qitang Hu, Rui Zhang, Shaoyan Sun, Shibo Sun, Shiying Duan, Tenghui Chen, Tianze Liu, Weijie Ke, Wenyao Xue, Xiaofeng Wang, Xiaoyu Tian, Xinyu Liu, Xinze Chen, Yang Wang, Yankai Wang, Yejun Zeng, Yifan Li, Yifei Nie, Yilong Li, Yilong Liu, Yongchao Feng, Yumeng Wang, Yun Ye, Zhichao Liu, Ziheng He, Zonghai Yang, Zheng Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where a robot can look at a cluttered kitchen table, understand a spoken request to "put the red apple in the bowl," and then figure out exactly how to move its arm to do it without knocking anything over. This is the promise of embodied artificial intelligence: machines that do not just process information but interact with the physical world. For years, researchers have tried to teach robots by showing them thousands of videos of human hands moving objects, hoping the machine would learn the rules of physics and cause-and-effect. However, a major hurdle has remained. Robots are built differently; one might have two arms, another a single gripper, and a third might roll on wheels. Data collected from one type of robot often confuses another. Furthermore, simply watching a video does not teach a machine how to predict what will happen next if it makes a mistake, or how to recover when a task goes wrong. The question has been whether we can build a single, smart brain that understands language, sees the world, and controls any kind of robot body, all while learning from a massive mix of different experiences.

A team of researchers has now presented a significant step forward with a system called GigaBrain-0.7. This is not just a single program but a coordinated system designed to handle the complexity of real-world interaction. Instead of trying to force a robot to learn everything in one chaotic burst, the researchers organized the learning process into three distinct but connected parts that work together. The first part acts as the hands and reflexes, taking immediate control of the robot's motors to execute actions. The second part acts as the planner and observer, looking at the scene, understanding the language instruction, and breaking a big goal like "clean the table" into smaller, manageable steps like "pick up the cup" and "move it to the sink." The third part is the most novel addition: it acts as a predictor and judge. It imagines what the scene will look like a few seconds from now if the robot continues its current path, and it assigns a score to that future to decide if the robot is making progress or heading toward a failure.

To train this system, the researchers did not rely on a single source of data. They gathered a massive library of over 37,000 hours of experience. This collection included videos from real robots working in factories and homes, recordings of human hands manipulating objects from a first-person perspective, and data generated by computer simulations. They also used artificial intelligence to create new training scenarios, effectively expanding the variety of situations the robot could learn from. By feeding this diverse mix of data into the system, the researchers allowed the model to learn general principles of movement and interaction rather than just memorizing specific tricks for one specific robot. The system was trained to handle 16 different types of robot bodies, from dual-armed machines to humanoid figures, teaching it to adapt its understanding of "left" and "right" or "grasp" and "release" based on the specific body it was controlling.

The results of this approach were tested on real robots in both industrial and household settings. When asked to perform tasks it had never seen before, the system showed a remarkable ability to generalize. In one test, a robot was asked to fold a piece of clothing that was thrown in a messy pile, a task that requires the machine to constantly adjust its grip as the fabric shifts and changes shape. Without needing to be retrained for that specific shirt, the system successfully manipulated the deformable object. In another scenario, the robot was asked to pick up a specific colored spoon from a group of mixed utensils, demonstrating that it could understand subtle language instructions and apply them to new objects. The system also proved capable of handling long sequences of actions, such as organizing a desk or sweeping rice, where the robot had to maintain a sense of the overall goal while executing many small movements.

A key finding was that the system's ability to predict the future and evaluate its own progress made a tangible difference in success rates. When the researchers removed the predictive part of the system, the robot struggled more with complex tasks, often getting stuck in loops of repetitive motion or failing to recover from minor errors. With the predictive component active, the robot could anticipate that a certain movement would lead to a collision or a drop, and it would adjust its course before the mistake happened. This capability allowed the robot to complete difficult tasks, such as wrapping a gift or sorting cubes, with much higher reliability than previous models. The researchers found that as they increased the amount of training data, the robot's performance improved consistently, suggesting that the system was truly learning from the sheer volume and variety of its experiences.

The study also highlighted the importance of how different types of data are combined. The system performed best when it learned from a mix of real robot trials, human demonstrations, and simulated environments. Each source provided a different kind of lesson: real robots taught the system about the physical constraints of specific machines, human videos showed it how people naturally interact with objects, and simulations allowed it to practice rare or dangerous scenarios safely. By blending these sources, the system developed a robust understanding of how the world works that could be applied across different robot bodies. The researchers noted that while the system is not perfect and still struggles with some of the most complex physical interactions, it represents a shift from building specialized robots for single tasks to creating a general-purpose intelligence that can adapt to new challenges.

Ultimately, GigaBrain-0.7 demonstrates that scaling up the amount and diversity of training data, combined with a structured architecture that separates planning, acting, and predicting, can lead to robots that are more capable and adaptable. The system does not just follow a script; it understands the context of a task, anticipates the consequences of its actions, and learns from its own experience to improve over time. While there is still work to be done before such systems can handle every possible situation in a home or factory, this research provides a clear path forward. It suggests that the future of robotics lies not in building better hardware alone, but in creating smarter, more flexible software that can learn from the vast and varied experiences of the physical world.

Technical Summary: GigaBrain-0.7

Problem Statement

Vision-Language-Action (VLA) models have emerged as a dominant paradigm for generalist embodied agents, demonstrating strong capabilities in structured settings. However, significant open questions remain regarding whether current VLA systems can effectively scale to substantially larger and more heterogeneous data regimes, achieve broader generalization across diverse robot embodiments, and benefit from more effective architectural designs. Existing systems often face challenges in coordinating understanding, planning, prediction, action execution, and experience-driven improvement across the full learning lifecycle. Furthermore, robot data varies substantially across embodiments and action spaces; without appropriate alignment, increased data diversity may introduce interference rather than transferable knowledge. Most current VLAs remain centered on reactive observation-to-action prediction with limited mechanisms for anticipating future states or evaluating behavioral progress.

Methodology

Three-System Architecture

GigaBrain-0.7 introduces a unified embodied foundation model organized around a three-system architecture designed to coordinate understanding, prediction, action, and continual improvement:

System 1 (Action & Control): A Vision-Language-Action (VLA) model that generates continuous action chunks. It utilizes a Mixture-of-Transformers (MoT) design, coupling a PaliGemma2 (3B) vision-language backbone with a dedicated 0.5B Action Expert. System 1 employs Soft Knowledge Insulation (Soft KI), which attenuates rather than completely blocks action gradients entering the VLM backbone, allowing controlled adaptation to embodied control while preserving general vision-language capabilities. It supports both autoregressive discrete action tokens and continuous action generation via flow matching.
System 2 (Understanding & Planning): A PaliGemma2 (3B) Vision-Language Model responsible for interpreting current and historical visual observations. It performs temporal context modeling, tracks task progress, and decomposes long-horizon instructions into executable subtasks via hierarchical prompts and chain-of-thought reasoning.
System 3 (Prediction & Evaluation): A World Value Model (based on GigaWorld-1, 5B) that provides two complementary conditioning interfaces for System 1. It predicts future visual states (subgoal images) to represent anticipated physical outcomes and estimates scalar task-progress values. These values are converted into binary advantage conditions to guide action generation and experience-based learning.

Data Curation and Processing

The model is pretrained on a massive, heterogeneous corpus comprising 37,256.98 hours of embodied trajectory data and 271,976,674 vision-language samples. The trajectory data spans 16 robot morphologies and includes:

Real Robot Data (55.12%): 20,535 hours from 16 platforms (e.g., Maker H01, AgileX, Franka, UR5).
UMI Data (22.15%): 8,251 hours of near-hand human demonstrations.
EGO Data (7.68%): 2,862 hours of first-person human manipulation videos.
Simulation Data (3.90%): 1,453 hours from physics-based environments.
World Model-Generated Data (11.15%): 4,153 hours of synthetic trajectories expanding visual and contextual diversity.

A unified processing pipeline standardizes this data into the LeRobot v3.0 format, performs cross-embodiment state-action normalization, rewrites language instructions via LLM assistance, and applies multi-stage quality control (including extreme-value filtering and stationary segment removal).

Training Paradigm

GigaBrain-0.7 employs a one-stage pretraining framework that jointly optimizes vision-language understanding, hierarchical task prediction, and continuous action generation, eliminating the optimization fragmentation typical of multi-stage pipelines. The training lifecycle consists of four stages:

Joint VLA Pre-training: Systems 1 and 2 are trained jointly on the heterogeneous corpus using a unified objective combining next-token prediction (NTP) for language/discrete actions and flow matching (FM) for continuous actions.
World Model Pre-training: System 3 is extended from GigaWorld-1 through robot-centric video pre-training followed by value learning to predict future states and task progress.
World-Model-Powered VLA Post-training: System 3 is frozen, and its outputs (subgoal images and value-derived advantages) are used as additional conditioning signals for System 1 during task-specific fine-tuning.
Experience-Driven Reinforcement Learning: The policy is refined through an offline-to-online pipeline. Offline reinforcement learning (using Advantage-Weighted Regression) utilizes rollout data to improve error recovery. Online reinforcement learning incorporates human-in-the-loop (HIL) corrective feedback to further optimize performance on difficult states.

Key Contributions

Unified Three-System Architecture: A novel framework that decouples but coordinates understanding (System 2), prediction/evaluation (System 3), and action/control (System 1) through structured semantic, visual, and value-based interfaces.
Scale and Heterogeneity: Pretraining on over 37,000 hours of heterogeneous embodied data across 16 robot types, demonstrating that scaling diverse data sources (Real, UMI, EGO, Simulation, World Model) leads to emergent capabilities.
One-Stage Joint Optimization: A training recipe that jointly optimizes vision-language and action objectives in a single pass, utilizing Soft Knowledge Insulation to balance general capabilities with embodied control.
Predictive Conditioning: The integration of a World Value Model (System 3) that provides explicit future-state predictions and value-based advantage signals to guide action generation and improve robustness.
Open Release: All training code and pretrained model weights are released to the community.

Results

Scaling and Architecture Analysis

Data Scaling: Increasing the scale of pretraining data consistently reduces validation loss and improves real-robot success rates. Complex tasks like deformable object manipulation (e.g., clothes folding) show a sharper dependence on data scale compared to simpler tasks.
Data Composition: Combining robot, UMI, and EGO data yields the strongest performance, with human demonstrations providing complementary supervision that improves the quality of the learned pretraining prior.
Architecture: The PaliGemma2-based dual-stream MoT architecture outperforms alternatives (e.g., Last-Layer Cross Attention) in real-robot tasks, particularly on challenging deformable-object manipulation, despite higher inference costs.

Zero-Shot and Foundation Capabilities

Out-of-the-Box Performance: Without task-specific adaptation, GigaBrain-0.7 demonstrates strong zero-shot capabilities on both AgileX PiPER and Maker H01 platforms. It successfully follows language instructions and performs complex manipulation (e.g., folding unseen garments, sorting cubes) in both in-distribution and out-of-distribution (OOD) settings.
Generalization: The model exhibits target understanding, spatial grounding, and compositional generalization, handling unseen objects, randomized layouts, and novel task compositions.

Post-Training Performance

Language Following: After task-specific post-training, GigaBrain-0.7 achieves state-of-the-art or competitive performance against baselines like $\pi0.5$ , Xiaomi-Robotics-1, and G0.5. On Maker H01, it improves the average success rate from 69.6% (GigaBrain-0.1) to 84.2%.
Complex Manipulation: The model shows substantial improvements in complex, multi-stage tasks such as play-dough kneading, tableware arranging, and household item storage. On Maker H01, it achieves an average success rate of 74.1% across complex tasks, significantly outperforming previous GigaBrain models and other baselines.
System 3 Ablation: Ablation studies confirm that both subgoal image conditioning and value-derived advantage signals improve task success, task scores, and execution efficiency. The combination of both signals yields the best results, particularly on difficult tasks like gift wrapping.

Benchmark Evaluation

Embodied VLM: On the MiMo-Embodied benchmark, GigaBrain-0.7 achieves the highest overall score (0.4621 before MiMo data integration, rising to 0.5704 after) among compared models, demonstrating strong spatial understanding and affordance reasoning.
Simulation Benchmarks: In RoboTwin 2.0, GigaBrain-0.7 ranks first under the challenging domain-randomized Hard setting (67.9% success). On EBench, it achieves the highest success rate (33.30%) and task score (46.1) among publicly available VLA models.

Significance

The paper claims that GigaBrain-0.7 demonstrates that scaling heterogeneous embodied pretraining, when coordinated through a unified three-system architecture, leads to emergent embodied capabilities. The system achieves substantial improvements in foundation zero-shot capabilities, language-conditioned instruction following, and post-training task success rates compared to prior GigaBrain models and state-of-the-art baselines. By integrating prediction and evaluation (System 3) with action generation, the model moves beyond reactive control to anticipate future states and evaluate progress, enabling robust generalization across diverse robot embodiments and real-world scenarios. The work suggests that effective scaling depends not only on the volume of trajectories but on the integration of heterogeneous forms of physical experience (robot, human, simulation, and synthetic) into a common training representation.