Embodiment-Aware Generalist Specialist Distillation for Unified Humanoid Whole-Body Control

This paper introduces EAGLE, an iterative generalist-specialist distillation framework that enables a single unified reinforcement learning policy to robustly control diverse, heterogeneous humanoid robots across various skills without requiring per-robot reward tuning.

The Gist

Imagine you are trying to teach a group of very different people how to dance. You have a tall basketball player, a short gymnast, and a person with a very flexible spine. If you try to teach them all the exact same dance moves using the exact same instructions, the tall person might trip, the short person might not reach the steps, and the flexible person might twist in ways you didn't intend.

In the world of robotics, this is the problem with Humanoid Robots. Every robot (like Unitree H1, G1, or Fourier N1) has a different body shape, different joint counts, and different weight distributions. Usually, to make a robot walk, squat, or lean, engineers have to write a unique "brain" (a software policy) for each specific robot. It's like hiring a different dance instructor for every single dancer. This is slow, expensive, and doesn't scale.

This paper introduces EAGLE, a new method to create one single "Super Brain" that can control any of these different robots, no matter how different their bodies are.

Here is how EAGLE works, broken down into simple concepts:

1. The "Generalist" and the "Specialists"

Think of the EAGLE system as a master teacher and a group of apprentices.

• The Generalist (The Master): This is the main AI brain. It starts by learning from a simulation where it sees all the different robots at once. It tries to figure out the general rules of walking and balancing.
• The Specialists (The Apprentices): The system takes a copy of the Generalist and sends it to a specific robot (e.g., the tall one). This copy becomes a "Specialist." Because it only has to deal with one body type, it gets really, really good at moving that specific robot. It learns the tiny nuances, like "Oh, this robot's left leg is slightly heavier, so I need to push a bit harder."

2. The "Cyclic Distillation" (The Feedback Loop)

This is the magic sauce. Instead of stopping there, the system runs a loop:

1. Forking: The Generalist copies itself to become a Specialist for Robot A, Robot B, and Robot C.
2. Training: Each Specialist gets to practice only on its own robot until it becomes an expert.
3. Distillation (The Lesson): The Specialists then teach their new, specific tricks back to the Generalist. Imagine the tall basketball player telling the master teacher, "Hey, when I lean, I need to shift my weight differently than you thought."
4. Updating: The Generalist absorbs all these lessons from all the different robots. It becomes smarter and more adaptable.
5. Repeat: The cycle starts again. The Generalist is now better, so the new Specialists it creates are even better, and they teach it even more.

Eventually, the Generalist becomes so good that it can control any of the robots perfectly, without needing to be retrained for each new body type.

3. The "Universal Remote" (The Command Interface)

Usually, telling a robot to "walk" is easy, but telling it to "squat while leaning left" is hard because different robots have different ways of doing that.

EAGLE uses a Unified Command Interface. Think of this as a universal remote control with five buttons:

• Where to go (Forward/Backward/Left/Right)
• How fast to turn
• How high to stand (Squatting or standing tall)
• How much to lean (Leaning forward or backward)

No matter which robot you point the remote at, the Generalist brain translates those five simple buttons into the complex muscle movements that specific robot needs to perform. It's like a translator that speaks "Humanoid" fluently, regardless of the accent.

4. The Results: From Simulation to Reality

The researchers tested this on five different robots in a computer simulation and then took the same "Super Brain" to the real world to control four different physical robots.

• The Result: The EAGLE brain could make the robots walk, squat, and lean with incredible accuracy.
• The Comparison: Other methods (like trying to train one brain on all robots at once without this loop) were clumsy and made mistakes. EAGLE was precise and robust.
• Zero-Shot Transfer: This means they didn't have to tweak the code or retrain the brain when they switched from the computer simulation to the real metal robots. It just worked.

Why This Matters

Before this, if you wanted to deploy a fleet of 100 different robots (some big, some small, some with different arms), you would need 100 different software teams.

With EAGLE, you can have one software team that builds one brain. That brain can then be downloaded onto any robot in the fleet, and it will immediately know how to move that specific body safely and efficiently. It's a massive step toward making robots that can work together in a diverse world, just like humans of all shapes and sizes can dance together.

Technical Summary

1. Problem Statement

Current Reinforcement Learning (RL) approaches for Humanoid Whole-Body Control (WBC) face two major bottlenecks:

• Embodiment Specificity: Most policies are trained on a single robot platform. Due to differences in dynamics, Degrees of Freedom (DoFs), and kinematic topology, a policy trained on one robot (e.g., Unitree H1) cannot directly transfer to another (e.g., Fourier N1) without retraining and reward tuning.
• Limited Behavioral Richness: Existing cross-embodiment methods often support only low-dimensional velocity commands (walking). They struggle to execute complex, high-dimensional whole-body behaviors such as squatting, leaning, or torso pitching across diverse hardware.
• Data Scarcity: Unlike manipulation tasks where teleoperation can collect demonstrations, legged locomotion lacks a teleoperation pipeline for data collection, making imitation learning difficult.

The core challenge is to develop a single unified policy that can control multiple heterogeneous humanoids, support rich high-dimensional commands, and transfer effectively from simulation to the real world without per-robot reward tuning.

2. Methodology: EAGLE Framework

The authors propose EAGLE (Embodiment-Aware Generalist-specialist distillation for Unified Humanoid Whole-Body Control), an iterative framework that alternates between specializing on specific robots and generalizing knowledge back to a shared policy.

A. Unified Command and Observation Space

• High-Dimensional Command Interface: The system uses a 5-dimensional command vector $c_t = [v_x, v_y, \omega, h, p]^\top$, where:
  • $v_x, v_y, \omega$: Base linear and angular velocities (Task commands).
  • $h, p$: Base height offset and body pitch angle (Behavior commands).
  • This allows the policy to control walking, squatting, and leaning simultaneously.
• Embodiment-Aware Observation:
  • Input: Proprioception (joint positions/velocities, base angular velocity, gravity) over a time window, plus a gait clock.
  • Morphology Encoding: To help the network distinguish between robots, the critic receives privileged information about rigid body properties (mass, Center of Mass, inertia matrix) for the torso and feet. The actor is tasked with estimating these features, forcing the network to learn morphology-specific representations.
• Action Alignment: Since robots have different DoFs, the authors use zero-padding and fixed index mapping to embed all robot action spaces into a unified 32-dimensional vector. A permutation matrix maps specific joints to global indices, while a zero vector fills unused slots.

B. Iterative Generalist-Specialist Distillation Loop

The core innovation is a cyclic training process (Algorithm 1):

1. Specialize: The current generalist policy ($\pi_g$) is copied to create $N$ embodiment-specific specialists ($\pi_{s_i}$). Each specialist is fine-tuned exclusively on its specific robot.
2. Generalize (Distillation): The generalist $\pi_g$ is updated using data collected from all robots.
   • Action Relabeling: The generalist's proposed actions are relabeled with the actions of the corresponding specialists (DAgger-style).
   • Loss Function: The update minimizes a composite loss:
     $$L = L_{PPO} + \alpha \cdot L_a + \beta \cdot L_e$$
     • $L_{PPO}$: Standard RL exploration loss.
     • $L_a$: Action alignment loss (matching specialist outputs).
     • $L_e$: Representation alignment loss (matching hidden feature embeddings of the actor). This ensures the generalist learns the reasoning behind the specialist's actions, not just the outputs.

3. Key Contributions

1. Unified Control Framework: Introduced EAGLE, a distillation loop that produces a single policy capable of controlling heterogeneous humanoids (e.g., Unitree H1, G1, Fourier N1) without per-robot reward tuning.
2. Rich Behavioral Support: Demonstrated that a single policy can execute complex, high-dimensional behaviors (squatting, leaning, walking) across different morphologies, surpassing previous methods limited to velocity tracking.
3. Representation Learning: Showed that embedding morphology-aware observations and using representation-level distillation allows the network to learn distinct dynamics for different robots while sharing a unified architecture.
4. Sim2Real Validation: Successfully deployed the policy on four different real-world humanoid robots in a zero-shot manner, proving robustness against the sim-to-real gap.

4. Experimental Results

Experiments were conducted on 5 robots in simulation (Unitree H1, G1, Booster T1, Fourier N1, PNDbotics Adam) and 4 robots in the real world.

• Command Tracking Accuracy: EAGLE significantly outperformed baselines (PPO, COMPASS, Kickstarting) in tracking error for linear velocity, angular velocity, base height, and body pitch.
  • Example: On the Unitree H1, EAGLE achieved a linear velocity error ($E_{vx}$) of 0.061 m/s, compared to 0.108 m/s for standard PPO and 0.725 m/s for COMPASS.
• Ablation Studies:
  • Removing embodiment-aware observations caused a significant performance drop, confirming the necessity of morphology cues.
  • Iterative distillation (EAGLE w/ ID) consistently improved performance over single-round distillation.
  • The generalist policy achieved performance comparable to, or better than, policies trained exclusively on a single robot (single-robot PPO).
• Real-World Deployment: The policy successfully executed synchronous movements (walking, leaning, squatting) on Unitree H1, G1, Fourier N1, and Booster T1 without any real-world fine-tuning.

5. Significance

• Scalability: EAGLE offers a scalable solution for fleet-level humanoid control. Instead of training a unique model for every new robot design, a single "generalist" can be deployed across a heterogeneous fleet.
• Efficiency: It eliminates the need for costly per-robot reward tuning and data collection pipelines.
• Behavioral Complexity: It bridges the gap between simple locomotion and complex whole-body manipulation, enabling humanoids to perform dynamic tasks like squatting and leaning in diverse environments.
• Future Direction: The work paves the way for "Generalist" robots that can adapt to new hardware structures through distillation and randomization, moving closer to truly universal humanoid control.