Technical Summary: VENOM - Versatile Embodied Network for Omni-bodied Motion Tracking

Problem Statement

Achieving expert-level, expressive full-body motion tracking across multiple humanoid robots using solely demonstration data remains a significant challenge in robot learning. Current cross-embodiment approaches often decouple control into upper and lower body segments or rely on reinforcement learning (RL) with asymmetric actor-critic frameworks that require privileged information. Furthermore, there is a scarcity of large-scale, low-level control datasets for diverse humanoids, limiting the training of versatile foundation models. Existing models are often restricted to locomotion tasks or fail to generalize effectively across different embodiments without manual alignment of observation and action spaces.

Methodology

The VENOM Dataset

The authors curated a new multi-humanoid motion tracking dataset, the VENOM dataset, to address the lack of large-scale low-level control data.

• Source Data: The dataset was generated using the TWIST2 training framework (an RL-based motion tracking pipeline) to train expert trackers on five different humanoids: Unitree G1, Unitree H1, Unitree H1 v2, Fourier N1, and Booster T1 (29 DoF).
• Data Generation: Experts were trained on the entirety of the AMASS dataset. For each motion, experts generated one clean rollout and eight noisy rollouts. Noise was applied to reference motions in the observation space (root position, orientation, velocity, and joint positions).
• Scale: The dataset comprises approximately 7,705 hours of data, split into 6-second clips, containing states, actions, and rewards.

Model Architecture (VENOM)

VENOM is a GPT-based (Transformer) motion tracker designed for full-body control across multiple humanoids.

• Architecture: The core controller consists of 8 transformer layers with 8 attention heads and an embedding dimension of 768. It utilizes a unified architecture that does not decouple upper and lower body control.
• Input Processing: The model employs specific encoders to process reference motions:
  • Motion Encoder: Encodes 1-step reference motion.
  • History Encoder: Encodes 10 steps of reference motion history using 1-D convolutions.
  • Future Encoder: Encodes 1-step future reference motion using an MLP.
• Multi-Embodiment Handling: The model uses distinct token embedding layers and action heads for each humanoid. During training, inputs from all humanoids are concatenated along the batch dimension and fed into a shared transformer decoder. During inference, only the specific input module and action head corresponding to the target humanoid are active.
• Training Objective: The model is trained using supervised learning to minimize the Mean Squared Error (MSE) between predicted actions and expert actions. It is trained on both clean and noisy rollouts to improve robustness.

Baselines

The study compares VENOM against:

1. TWIST2 Baselines: MLP-based models trained on the VENOM dataset (either on a single robot or multiple robots) using the same supervised learning framework.
2. RL Experts: Asymmetric actor-critic models trained directly in the RL environment (TWIST2 framework), serving as the upper bound for performance.

Key Results

Scaling and Architecture

• Multi-Robot Generalization: When trained on multiple robots, the MLP-based TWIST2 baseline suffered from negative transfer, failing to stabilize robots while tracking reference motions. In contrast, the Transformer-based VENOM successfully generalized across different embodiments.
• Noise Robustness: The VENOM-noisy model (trained with noisy rollouts) demonstrated superior motion tracking capabilities compared to models trained only on clean data. The inclusion of noise exposed the policy to underexplored observation regions, enhancing robustness.
• Performance Metrics: On the Unitree G1, VENOM-noisy achieved a Root Mean Square (RMS) joint tracking error of 0.1052 radians, outperforming the multi-robot MLP baseline (0.1686) and matching the performance of the single-robot MLP baseline (0.1067).

Motion Tracking and Stability

• Joint Tracking: VENOM models consistently outperformed the multi-robot MLP baselines in joint tracking accuracy across various behavior categories (athletic, crouching, dancing, hopping, locomotion).
• Stability vs. Supervised Learning: While VENOM-noisy achieved high success rates (92–100%) and low drift compared to other supervised models, it did not fully match the motion stability of the RL experts. The RL experts achieved higher success rates and lower root error rates, suggesting a trade-off between motion tracking fidelity and stability when relying solely on supervised learning without reward feedback.
• Limitations: All models, including RL experts, struggled with highly dynamic motions involving flight phases (e.g., jumping, sprinting), often learning behaviors to maintain ground contact to ensure stability.

Significance and Claims

The paper positions VENOM as the first multi-humanoid full-body motion tracking model that operates without decoupling upper and lower body control. Its primary contributions are:

1. Dataset Curation: The creation of the VENOM dataset, a large-scale, multi-embodiment dataset generated via expert imitation, which serves as a foundation for training versatile motion trackers.
2. Foundation Model Viability: Demonstrating that a GPT-based foundation model trained via supervised learning on diverse, noisy data can achieve cross-embodiment full-body motion tracking capabilities that surpass traditional MLP baselines.
3. Supervised vs. RL Gap: Quantifying the gap between supervised learning and RL experts. The authors show that while supervised learning can recover much of an expert's capability, it currently involves a compromise in motion stability, particularly for highly dynamic tasks.

The authors conclude that while VENOM represents a significant step toward generalizable low-level control, future work should explore direct RL training for motion tracking, online retargeting policies to handle arbitrary morphologies, and improved prediction horizons for dynamic flight-phase motions.