💻 computer science

Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running

This paper presents a dynamic retargeting and control-guided reinforcement learning pipeline that enables a Unitree G1 humanoid robot to achieve high-speed, endurance-running up to 3.3 m/s with autonomous obstacle avoidance in real-world environments.

Original authors: Zachary Olkin, William D. Compton, Ryan M. Bena, Aaron D. Ames

Published 2026-03-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zachary Olkin, William D. Compton, Ryan M. Bena, Aaron D. Ames

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine teaching a robot to run like a human. It sounds simple, but for a machine made of metal and motors, running is like trying to balance a broomstick on your hand while sprinting through a hurricane. If the robot slips, it falls. If it tries to turn too fast, it crashes.

This paper is about a new "training camp" that teaches a humanoid robot (specifically a Unitree G1) to run fast, stay upright, and actually listen to its human boss when told to speed up, slow down, or dodge a tree.

Here is the story of how they did it, broken down into three simple steps:

1. The "Strict Coach" (Dynamic Retargeting)

Usually, when scientists want a robot to learn a human move, they just copy the human's video frame-by-frame. Think of this like a student trying to copy a teacher's handwriting by tracing over the letters. It looks okay, but if the robot tries to run at a different speed or on a different surface, the tracing falls apart because the robot's body is different from a human's.

The Innovation: Instead of just tracing, the authors acted like a strict physics coach. They took a video of a human running and ran it through a complex math simulation (optimization) that asked: "Is this physically possible for a robot?"

The Analogy: Imagine you have a recipe for a cake meant for a small oven. If you try to bake it in a giant industrial oven, it will burn. The "Strict Coach" rewrites the recipe specifically for the giant oven, adjusting the temperature and time so the cake comes out perfect every time.
The Result: They created a "library" of running styles. Instead of just one rigid video, they have a flexible set of instructions that the robot can actually execute without falling over, even at high speeds.

2. The "Smart Reward System" (Control-Guided RL)

Once the robot has the "Strict Coach's" instructions, it needs to learn how to follow them. This is done using Reinforcement Learning (RL), which is like training a dog with treats.

The Old Way (Mimic): The robot gets a treat every time it looks somewhat like the human video. It's like telling a dog, "Good boy, you're standing on your hind legs!" even if it's wobbling and about to fall.
The New Way (Control-Guided/CLF): The authors added a "safety guard" to the training. They didn't just reward the robot for looking like the human; they rewarded it for staying stable.
The Analogy: Imagine teaching a toddler to ride a bike.
- Mimic: "Great job pedaling!" (Even if they are about to crash).
- Control-Guided: "Great job pedaling, AND great job keeping your balance so you don't fall!"
  This "safety guard" (based on Control Lyapunov Functions, a fancy math concept for stability) ensures the robot learns to run safely while being fast.

3. The "GPS Navigator" (Autonomy Stack)

The final test was: Can this robot run on its own in the real world?
They connected the running robot to a "brain" that could see obstacles (using LiDAR, like a bat's sonar).

The Analogy: Think of the robot's legs as a race car engine (the running controller) and the "brain" as the driver. The driver tells the engine, "Go 20 mph," "Turn left," or "Brake!"
The Result: The robot didn't just run in a straight line. It ran outdoors for hundreds of meters, dodged obstacles in real-time, and even changed direction without tripping. It reached speeds of 3.3 meters per second (about 7.4 mph), which is a solid jog for a human.

Why This Matters

Before this, robots could either run fast but couldn't be controlled, or they could be controlled but moved like slow, clumsy zombies.

This paper bridges the gap. It shows that if you:

Clean up the human data so it makes sense for a robot (The Strict Coach).
Reward the robot for being stable, not just for looking cool (The Safety Guard).
Connect it to a navigation system (The GPS).

...you get a robot that can run like a human, listen to commands, and navigate a messy real-world environment without falling over. It's a huge step toward robots that can actually help us in the real world, rather than just looking cool in a lab.

1. Problem Statement

While Reinforcement Learning (RL) has enabled humanoid robots to perform highly dynamic motions (like running) by mimicking human data, current approaches face significant limitations in autonomy and controllability:

Single-Motion Limitation: Most "mimic" style RL controllers are restricted to playing back a single pre-defined motion clip. They struggle to generalize to steady-state locomotion or adapt to varying speeds and directions.
Velocity Tracking: Existing methods often fail to accurately track commanded velocities (forward, lateral, and yaw), making them difficult to integrate into layered autonomy stacks where a high-level planner issues velocity commands.
Feasibility: Directly retargeting human motion to a robot often results in dynamically infeasible trajectories due to the "embodiment gap" (differences in mass, inertia, and actuation) and lack of periodicity in raw human data.

The paper aims to bridge the gap between agile, human-like running and robust, controllable autonomy by creating a pipeline that generates optimized reference libraries and trains a controller capable of precise velocity tracking in real-world environments.

2. Methodology

The proposed framework consists of three main stages: Dynamic Retargeting, Reward Design, and RL Training/Deployment.

A. Dynamic Retargeting via Multi-Shooting Optimization

Instead of using raw kinematic retargeting, the authors generate a library of reference trajectories using multiple-shooting dynamic optimization with hard constraints.

Input: A single stride segment from human motion capture data (LAFAN dataset).
Process:
- The problem is formulated as a hybrid system with two domains: Single Support Phase (SSP) and Flight Phase (FLT).
- Hard Constraints: The optimizer enforces periodicity (ensuring the end state mirrors the start state), foot contact constraints (no slippage), and dynamic feasibility (Newton-Euler dynamics).
- Library Generation: By varying step length constraints while maintaining step time, the system generates a library of running gaits ranging from 1.2 m/s to 3.6 m/s from a single human data segment.
Output: A set of dynamically feasible, periodic reference trajectories represented as Bezier curves for the RL agent to track.

B. Reward Structure and RL Design

The paper investigates the impact of reward structures on tracking performance, comparing two primary approaches:

Mimic Rewards: Standard tracking rewards that penalize the difference between the robot's state (joints, base, end-effectors) and the reference trajectory.
Control-Guided (CLF-RL) Rewards: Inspired by Control Lyapunov Functions (CLFs), this reward embeds a Lyapunov stability condition. It encourages the system to satisfy a decrescent condition ( $\dot{V} + \lambda V < 0$ ), theoretically guaranteeing stability toward the reference.

Goal Conditioning:
To enable autonomous control, the authors introduce goal conditioning. The reward function includes terms that penalize the deviation between the robot's actual velocity and the commanded velocity (forward, lateral, and yaw). This allows the policy to deviate from the reference trajectory if necessary to obey high-level commands.

Training Setup:

Simulator: IsaacLab/IsaacSim with PPO (Proximal Policy Optimization).
Domain Randomization: Friction, mass, and motor gains are randomized to ensure robustness.
Observations: The policy observes base angular velocity, gravity, commanded velocity, joint states, and phase information. Crucially, the reference trajectory is not an observation, simplifying deployment.

3. Key Contributions

The paper makes three primary contributions (C1–C3):

C1: Optimized Reference Generation: A method to generate a library of dynamically feasible, periodic running references from a single human demonstration using constrained multi-shooting optimization. This outperforms purely kinematic retargeting and optimization without human data.
C2: Reward Structure Analysis: A systematic study showing that Goal-Conditioned CLF-RL combined with a library of dynamic optimized references yields superior performance. This combination achieves better reference tracking and, more importantly, accurate commanded velocity tracking compared to standard mimic rewards or single-trajectory approaches.
C3: Hardware Autonomy Integration: Successful deployment on a Unitree G1 humanoid robot. The controller achieves running speeds up to 3.3 m/s, traverses hundreds of meters outdoors, and integrates into a full autonomy stack for real-time obstacle avoidance.

4. Results

Simulation Ablation Studies

Reference Quality: Policies trained on dynamically optimized references significantly outperformed those using kinematically retargeted data or data generated without human priors.
Reward Efficacy: CLF-RL rewards consistently resulted in lower tracking errors (both reference and velocity) compared to standard Mimic rewards.
Velocity Tracking: The combination of Goal Conditioning + CLF Rewards + Dynamic Reference Library achieved the lowest velocity tracking error across all tested speeds (1.4 m/s to 3.2 m/s). Increasing the weight of goal conditioning on a single kinematic trajectory did not match the performance of the optimized library approach.

Hardware Experiments

Speed and Endurance: The robot achieved a top speed of 3.3 m/s on an indoor treadmill and ran continuously for over 250 meters on outdoor sidewalks.
Autonomy Stack: The running controller was integrated with a high-level planner using Model Predictive Control (MPC) and Control Barrier Functions (CBF).
- The system used LiDAR for real-time obstacle detection.
- The MPC layer generated velocity commands to avoid obstacles while maintaining a forward speed of ~2 m/s.
- The robot successfully performed collision avoidance in dynamic outdoor environments without falling.

5. Significance

This work represents a significant step toward fully autonomous humanoid locomotion.

Bridging the Gap: It solves the critical issue of making "mimic" RL controllers controllable. By moving from single-motion playback to goal-conditioned tracking of optimized libraries, the robot can act as a reliable component in a hierarchical autonomy stack.
Robustness: The use of Control Lyapunov Functions in the reward function provides a theoretical grounding for stability, resulting in a controller that is robust to disturbances and capable of high-speed operation in unstructured environments.
Practicality: The demonstration on a commercial robot (Unitree G1) in real-world outdoor settings validates the "sim-to-real" transfer capabilities of the proposed pipeline, moving humanoid running from controlled lab demonstrations to practical, long-duration autonomous navigation.