Technical Summary: Now You See That: Learning End-to-End Humanoid Locomotion from Raw Pixels

1. Problem Statement

Achieving robust, vision-based locomotion for humanoid robots remains a significant challenge in embodied intelligence. Unlike quadrupedal robots, which possess inherent static stability that tolerates moderate control errors, humanoids operate in inherently unstable regimes requiring precise coordination between perception and action. The paper identifies two fundamental barriers to deploying such systems in the real world:

1. The Sim-to-Real Gap in Perception: Standard simulation pipelines fail to capture the structured imperfections of real-world depth sensors (e.g., stereo matching artifacts, calibration uncertainties, and depth-dependent noise). This perception noise severely degrades performance on fine-grained tasks requiring centimeter-level accuracy, such as traversing long staircases or navigating narrow gaps.
2. Conflicting Learning Objectives in Unified Control: Training a single policy to handle diverse terrains (e.g., stairs, gaps, rough terrain, platforms) is hindered by conflicting reward landscapes. Standard reinforcement learning with a single value function struggles to capture these heterogeneous dynamics, often necessitating the training of separate specialist policies, which incurs substantial overhead.

2. Methodology

The authors propose an end-to-end framework that addresses both perception transfer and unified terrain control through a two-stage training pipeline.

A. Realistic Depth Sensor Simulation

To bridge the sim-to-real gap, the authors introduce a comprehensive augmentation pipeline that sequentially applies eight operators to raw depth images. Unlike standard domain randomization, this pipeline explicitly models structured sensor artifacts:

• Stereo Depth Fusion: Simulates hole patterns in occluded and textureless regions by enforcing disparity consistency checks between left and right viewpoints.
• Depth-Dependent Noise: Models non-linear noise growth with distance using quadratic scaling.
• Structured Noise Patterns: Uses multi-octave Perlin noise to simulate spatially correlated environmental interference.
• Optical Distortions: Applies randomized convolution to model lens aberrations.
• Calibration Uncertainties: Randomizes depth scaling, camera intrinsics, and extrinsics to simulate manufacturing variations and drift.
• Preprocessing: Includes pixel failure simulation (dead/saturated pixels), depth clipping, and spatial cropping to match real sensor characteristics.

B. Privileged Reinforcement Learning (Teacher)

The first stage involves training a "privileged" policy using clean height scan observations (ray-cast elevation maps) rather than raw depth images. To handle diverse terrains, the authors employ:

• Terrain-Specific Reward Shaping: Terrains are partitioned into three categories (stairs/platforms, gap crossing, and rough terrain), each with specialized reward components.
• Multi-Critic and Multi-Discriminator Architecture: Instead of a single value function, the system utilizes $K=3$ dedicated critic networks and $K=3$ discriminator networks. Each pair is specialized for a specific terrain category, sharing a common backbone but maintaining separate output heads. This allows the policy to learn distinct motion priors and value landscapes for different dynamic regimes while operating within a unified architecture.
• Adversarial Motion Priors (AMP): Discriminators provide style rewards tailored to the motion characteristics required for each terrain type, guiding natural locomotion.

C. Vision-Aware Behavior Distillation (Student)

The second stage distills the privileged policy into a deployment policy that operates directly on noisy, augmented depth images. This process combines:

• Behavior Cloning: Minimizing the action discrepancy between the student (depth input) and the teacher (height scan input).
• Denoising Objective: Enforcing consistency between the latent representations of clean depth and augmented noisy depth to ensure robust feature extraction.
• Feature Regularization: Applying KL divergence regularization to prevent representation collapse by matching the encoder's feature distribution to a standard normal prior.

3. Key Contributions

The paper outlines four primary contributions:

1. Realistic Depth Sensor Simulation: A comprehensive augmentation pipeline that reproduces stereo matching artifacts, depth-dependent noise, optical distortions, and calibration uncertainties, facilitating seamless sim-to-real transfer.
2. Vision-Aware Behavior Distillation: A framework combining latent space shaping with noise-invariant auxiliary tasks (denoising and KL regularization) to transfer locomotion knowledge from privileged observations to noisy depth inputs.
3. Multi-Critic and Multi-Discriminator Terrain Learning: A unified policy architecture using terrain-specific reward shaping and dedicated networks to capture distinct dynamics and motion priors across diverse terrains without requiring separate specialist policies.
4. Cross-Platform Validation: Extensive real-world experiments on two humanoid robots with different stereo depth cameras, demonstrating successful traversal of extreme and fine-grained terrains without fine-tuning.

4. Experimental Results

The approach was validated on a custom humanoid robot equipped with an Orbbec Gemini 336L depth camera and evaluated on the Real-World Depth Transfer Benchmark (RDT-Bench), which uses CycleGAN-augmented depth images to simulate realistic perception conditions.

• Performance Metrics: The proposed method achieved a 98.9% average success rate across four terrain configurations (stairs, gaps, platforms, and mixed), with an average power consumption of $27.7 \times 10^1$ W and a Power Degradation Ratio (PDR) of only 5.8%.
• Comparisons:
  • Compared to "No Augmentation" (43.0% success, 70.9% PDR), the full pipeline demonstrated the critical importance of realistic sensor modeling.
  • Compared to "Humanoid Parkour Learning" (71.0% success, 30.9% PDR), the proposed method improved success rates by 27.9% and reduced power degradation by 25.1%.
  • Ablation studies confirmed that removing the multi-critic architecture or the denoising objectives significantly degraded performance, validating the necessity of both terrain-specific learning and noise-invariant feature extraction.
• Real-World Deployment: On physical hardware, the policy achieved a 97.8% overall success rate (88/90 trials) across outdoor stairs, platforms, extended staircases (30+ steps), and wide gaps. Notably, the policy ran entirely onboard at 50 Hz without any real-world fine-tuning.
• Cross-Platform Generalization: Preliminary results on a Unitree G1 robot with a different sensor (Intel RealSense D435i) showed a 100% success rate on stair ascending, suggesting the learned representations generalize beyond the specific training hardware.

5. Significance and Claims

The paper claims that its framework enables robust, end-to-end vision-based humanoid locomotion across a wide spectrum of challenging environments, from extreme parkour-style obstacles to fine-grained tasks like bidirectional long-term staircase traversal.

The authors emphasize that their approach successfully unifies the control of heterogeneous terrains within a single policy, overcoming the limitations of prior work that either relied on separate specialist policies or failed to handle the perception noise inherent in real-world depth sensors. By combining realistic sensor simulation with terrain-specific multi-critic learning and vision-aware distillation, the method achieves high success rates and energy efficiency without requiring real-world fine-tuning.

However, the authors modestly note a limitation: stair descent remains less robust (86.7% success) than ascent (100% success). They attribute this to the gravitational amplification of minor errors during descent and the occlusion of target footholds by step edges, which exacerbates stereo matching artifacts. This observation motivates future work on higher-resolution near-field depth sensing and predictive foot placement strategies.