💻 computer science

From Foundation to Application: Improving VLA Models in Practice

The paper introduces LingBot-VLA 2.0, a VLA foundation model that bridges the gap between laboratory research and real-world application by enhancing generalization through massive multi-embodiment pretraining, expanding action spaces to support whole-body manipulation, and improving temporal reasoning via predictive dynamics modeling.

Original authors: Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, Yibin Liu, Shuai Yang, Tianxiang Zhou, Kejia Zhang, Lei Zhou, Cheng Su, Nan Xue, Bin Tan

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wei Wu, Fangjing Wang, Fan Lu, He Sun, Shi Liu, Yunnan Wang, Yibin Yan, Yong Wang, Shuailei Ma, Xinyang Wang, Yibin Liu, Shuai Yang, Tianxiang Zhou, Kejia Zhang, Lei Zhou, Cheng Su, Nan Xue, Bin Tan, Han Zhang, Youchao Zhang, Fei Liao, Xing Zhu, Yujun Shen, Kecheng Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant robot student who has spent years studying in a perfectly controlled, quiet classroom (the "laboratory"). They are great at solving puzzles when the lights are bright, the table is empty, and the teacher gives them exact instructions. But the moment you take them into a busy, noisy kitchen to actually cook dinner, they freeze. They don't know how to handle a wobbly chair, a slippery spoon, or a moving floor.

This paper introduces LingBot-VLA 2.0, a major upgrade designed to take that "classroom robot" and turn it into a "kitchen-ready robot." The authors argue that to make robots useful in the real world, we need to fix three specific problems: variety, movement, and foresight.

Here is how they did it, explained through simple analogies:

1. The "Super-Student" Training Camp (Generalization)

The Problem: Previous robots were trained on very specific, limited data. It was like teaching a student to drive only on a single, empty track. They couldn't handle a different car or a rainy road.
The Solution: The team built a massive "training camp" with 60,000 hours of footage.

The Mix: They didn't just use one type of robot. They gathered data from 20 different robot bodies (some with one arm, some with two, some with wheels, some with legs).
The Human Touch: They also added 10,000 hours of videos filmed from a human's point of view (like a GoPro on your head), showing how humans naturally move their hands and bodies to do tasks.
The Result: By feeding the robot this "diet" of diverse experiences, it learned to be a generalist. It's no longer a specialist in one room; it's a jack-of-all-trades that can adapt to different robot bodies and messy environments.

2. The "Full-Body Dance" (Expanded Action Space)

The Problem: Most robots were trained to only move their arms, like a person sitting in a chair with their hands tied to a desk. They couldn't move their head to look around, twist their waist, roll on wheels, or use delicate fingers.
The Solution: LingBot-VLA 2.0 learned to move its whole body.

The Analogy: Imagine a pianist who was previously only allowed to press keys with their fingers. Now, they can also stand up, walk to the piano, adjust the bench, turn their head to read the sheet music, and use their toes to tap the rhythm.
The Capability: The new model controls the robot's head, waist, mobile base (wheels), and dexterous hands. This allows the robot to tackle complex jobs that require coordination, like walking up to a fridge, opening the door, and grabbing a bottle, all in one smooth motion.

3. The "Crystal Ball" (Predictive Dynamics)

The Problem: Old robots reacted to what they saw right now. If a ball started rolling, they waited until it hit them to react. They lacked "temporal reasoning"—the ability to think about what happens next.
The Solution: The team taught the robot to predict the future.

The Analogy: Instead of just looking at a chessboard and moving a piece, the robot is now playing a game where it has to guess what the board will look like three moves from before it even makes the move.
How it works: They used two "teachers" to help the robot learn this:
1. A Depth Teacher: Shows the robot how far away objects are (geometry).
2. A Video Teacher: Shows the robot how things move over time (causality).
The Result: The robot can now "imagine" the future state of a scene. It knows that if it pushes a cup, it will slide; if it opens a door, it will swing. This helps it plan ahead rather than just reacting.

The Proof: Passing the "Real-World Exam"

To test if these changes actually worked, the researchers put the robot through a series of difficult challenges called the GM-100 benchmark.

The Tasks: These weren't simple things like "pick up a block." They were complex, multi-step jobs like "sort snacks into containers," "pick toy bones out of a plate," or "take a bowl out of a microwave."
The Outcome: LingBot-VLA 2.0 significantly outperformed previous versions and other top models. It didn't just get the tasks right more often; it handled the messy, real-world variations much better. It also showed it could handle long, complicated sequences (like moving from one room to another to clean a stove) without getting lost.

Summary

In short, LingBot-VLA 2.0 is a robot brain that has been upgraded to be:

More adaptable (trained on 20 different robot types and human videos).
More mobile (can move its whole body, not just its arms).
More forward-thinking (can predict how the world will change in the next few seconds).

The paper claims this moves robot intelligence from "laboratory success" to "practical application," making them ready to actually help us in our homes and workplaces.

Technical Summary: LingBot-VLA 2.0

Problem Statement

Despite recent advancements in Vision-Language-Action (VLA) foundation models, a significant disparity remains between laboratory benchmarks and real-world robotic deployment. Current VLA systems often struggle with three primary limitations when transitioning to practical applications:

Limited Generalization: Existing models frequently fail to generalize across diverse robot embodiments (heterogeneous configurations) and data sources, often being trained on narrow task sets or specific hardware.
Restricted Action Spaces: Many systems are confined to standard dual-arm manipulation, lacking the capacity to control whole-body degrees of freedom (DoF) such as mobile bases, waists, heads, and dexterous hands, which are essential for complex real-world tasks.
Weak Temporal Reasoning: Current models often react to immediate observations rather than anticipating future scene evolution and action consequences, limiting their effectiveness in dynamic environments.

Methodology

The authors present LingBot-VLA 2.0, an improved VLA system designed to bridge the gap between foundation-level capabilities and practical deployment. The methodology focuses on three functional domains:

1. Large-Scale, Multi-Embodiment Data Curation

To enhance generalization, the authors redesigned the data processing pipeline to curate a pretraining corpus of approximately 60,000 hours:

Robot Data: ~50,000 hours of trajectories spanning 20 different robot configurations (including single-arm, dual-arm, half-humanoid, and full-humanoid platforms) with varying DoFs (arms, hands, waists, mobile bases).
Egocentric Data: ~10,000 hours of human videos.
Processing Pipeline: The pipeline employs rigorous filtering, including:
- Quality Control: Discarding episodes with trajectory discontinuities, abnormal velocities/accelerations (jerk), or static signals (>95% duration).
- Alignment Verification: Using URDF projection and human annotation to ensure consistency between video frames and state signals.
- Egocentric Reconstruction: For action-free human videos, the system uses SLAM for camera pose estimation and hand pose estimation (MANO parameters) to reconstruct continuous hand trajectories in the world coordinate frame.
Unified Representation: A 55-dimensional canonical vector representation is used to unify heterogeneous controls (arms, grippers, hands, waists, heads, mobility) into a single action space, with padding applied for missing dimensions.

2. Expanded Action Space and Architecture

The model is extended to support a broader action space beyond standard dual-arm setups, accommodating:

Whole-Body Control: Head (2D), Waist (4D), Mobile Base (3D), Dexterous Hands (6D/12D), and Arms (6D/7D).
Mixture-of-Experts (MoE) Architecture: The authors introduce a token-level, loss-free sparse MoE within the action expert.
- Structure: Replaces standard Feed-Forward Networks (FFNs) with a combination of one shared expert (for universal priors) and multiple routed experts (for specialized modeling).
- Routing: Utilizes a sigmoid-based routing mechanism (inspired by DeepSeek-V3) to allow tokens to independently activate multiple experts without softmax-induced competition.
- Load Balancing: Employs an auxiliary-loss-free strategy using routing correction biases to ensure balanced expert utilization without injecting explicit load-balancing losses into the primary objective.

3. Predictive Dynamics via Dual-Query Distillation

To improve temporal reasoning, the system formulates future prediction as a proxy task using a Dual-Query Distillation framework:

Dual Queries: Two learnable queries, $Q_t$ (current) and $Q_{t+T}$ (future), are appended to visual and textual tokens.
Teachers:
- LingBot-Depth: Provides geometric supervision by distilling depth representations for current and future frames.
- DINO-Video: A robotics-aware video representation model (built on DINOv3 with causal temporal attention) providing semantic priors and motion-aware visual features.
Objective: The model is trained to predict the depth and video features for both the current and future time steps, encouraging the VLA to reason about future scene evolution and action consequences.

Key Contributions

Data Scale and Diversity: The creation of a 60,000-hour pretraining dataset covering 20 diverse robot embodiments and egocentric human videos, processed through a novel pipeline that unifies heterogeneous data sources.
Whole-Body Action Modeling: The extension of VLA capabilities to control complex, whole-body degrees of freedom (head, waist, mobile base, dexterous hands) via a unified action representation.
Architectural Innovation: The implementation of a token-level, loss-free sparse MoE architecture that scales model capacity efficiently while maintaining balanced expert utilization.
Temporal Reasoning: The introduction of a dual-query distillation framework that leverages depth and video representation models to enhance the model's ability to predict future states and reason causally.

Experimental Results

The authors evaluated LingBot-VLA 2.0 on the GM-100 benchmark (9 bimanual manipulation tasks) and two long-horizon mobile manipulation tasks.

Bimanual Manipulation (GM-100):
- On the Agilex Cobot Magic platform, LingBot-VLA 2.0 achieved a progress score of 66.2% and success rate of 34.4%, outperforming LingBot-VLA 1.0 (58.2%/30.0%) and $\pi_0.5$ (59.1%/32.2%).
- On the Galaxea R1 Pro platform, it achieved 34.6% progress and 15.6% success, surpassing $\pi_0.5$ (27.4%/8.9%).
- Improvements were most significant in tasks requiring precise object grounding and goal-directed execution (e.g., "Retrieve keychain" reached 100% success on Agilex).
Long-Horizon Mobile Manipulation:
- Evaluated on Astribot S1 (refrigerator sorting) and Cobot Magic-ARX X5 (stove cleaning).
- LingBot-VLA 2.0 consistently outperformed $\pi_0.5$ in both in-domain and out-of-distribution (OOD) settings.
- In OOD settings (perturbed initial poses and unseen objects), LingBot-VLA 2.0 maintained a clear advantage in success rates, demonstrating robustness in completing full task trajectories.
Ablation Studies:
- Action Targets: Relative joint actions significantly outperformed absolute actions (55.0% vs. 33.7% success).
- Normalization: MeanStd normalization yielded the best results (55.0%) compared to MinMax or Q01-Q99, preserving the long-tailed structure of relative action distributions.
- Loss Function: L2 loss performed best overall, though L1 was superior for contact-rich tasks like "Squeeze Ketchup."

Significance and Claims

The paper claims that LingBot-VLA 2.0 represents a pragmatic step toward advancing VLA foundation models from laboratory success to real-world applicability. The authors argue that practical VLA systems must evolve beyond simple scaling of model and data size to address specific real-world demands:

Broader Embodiment Support: Handling heterogeneous robot configurations.
Richer Action Spaces: Enabling control over whole-body dynamics.
Predictive Understanding: Strengthening temporal reasoning to anticipate dynamic scene changes.

The results validate that jointly enhancing generalization, action-space coverage, and predictive dynamics modeling leads to consistent improvements in practical robotic capabilities. The work positions LingBot-VLA 2.0 not just as a model improvement, but as a framework for aligning foundation-level VLA capabilities with the complexities of real-world deployment.