💻 computer science

HiWET: Hierarchical World-Frame End-Effector Tracking for Long-Horizon Humanoid Loco-Manipulation

This paper presents HiWET, a hierarchical reinforcement learning framework that enables precise and stable long-horizon humanoid loco-manipulation by reformulating control in the world frame to eliminate cumulative drift and incorporating a Kinematic Manifold Prior to ensure kinematically valid actions.

Original authors: Zhanxiang Cao, Liyun Yan, Yang Zhang, Sirui Chen, Jianming Ma, Tianyue Zhan, Shengcheng Fu, Yufei Jia, Cewu Lu, Yue Gao

Published 2026-02-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhanxiang Cao, Liyun Yan, Yang Zhang, Sirui Chen, Jianming Ma, Tianyue Zhan, Shengcheng Fu, Yufei Jia, Cewu Lu, Yue Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a humanoid robot as a very talented, but slightly clumsy, dancer who is also trying to juggle. The dancer needs to move their feet to get to a specific spot on the stage (locomotion) while simultaneously using their hands to draw a perfect heart shape in the air (manipulation).

The problem with most previous robot dancers is that they think in "body coordinates." If the dancer's feet slip or they stumble, their brain gets confused about where their hands are in the real world. It's like trying to draw a straight line on a piece of paper while someone is constantly shaking the table; the line ends up wobbly and off-target.

HiWET is a new "brain" for these robots that solves this by changing how they think. Here is how it works, broken down into simple concepts:

1. The Two-Brain System (Hierarchical Control)

Instead of one brain trying to do everything at once, HiWET uses a two-level team:

The "Captain" (High-Level Policy): This brain looks at the big picture. It lives in the "World Frame," meaning it knows exactly where the robot is on the stage and where the target (like a heart shape) is in the room. It doesn't worry about how the knees bend; it just says, "We need to move our feet to the left to reach that spot, and our hands need to go up." It sets the goals.
The "Dancer" (Low-Level Policy): This brain handles the physical details. It takes the Captain's goals and figures out exactly how to move every single joint in the legs and arms to achieve them without falling over. It runs very fast, making tiny adjustments to keep the robot balanced.

2. The "Kinematic Manifold Prior" (The KMP)

This is the paper's secret sauce. Imagine teaching a human to draw. If you ask them to draw a circle from scratch, they might struggle. But if you give them a stencil (a guide) and ask them to trace it while making small corrections, they do it perfectly.

HiWET uses a Kinematic Manifold Prior (KMP) which acts like that stencil.

Before the robot even starts learning, it is given a pre-trained "guide" that knows the basic, safe ways to move its arms and waist to reach a target.
The robot doesn't have to learn how to move its arm from zero; it just learns the tiny residual corrections (the small tweaks) needed to make the movement perfect and stable.
This stops the robot from trying to do impossible or dangerous moves, making it learn much faster and safer.

3. Fixing the "Shaky Table" Problem

When a robot walks, its body bounces and shifts. If the robot only looks at its own body to know where its hand is, that hand looks like it's wobbling wildly, even if it's steady in the room.

HiWET fixes this by keeping a constant eye on the World Frame. Even if the robot's feet slip or its body jerks, the "Captain" knows exactly where the target is in the room and tells the "Dancer" to compensate. It's like a tightrope walker who keeps their eyes fixed on the finish line, not on their own wobbling feet.

4. What They Tested

The researchers tested this on a real robot (the Unitree G1) and in a computer simulation.

The Test: They asked the robot to walk to different spots and draw complex shapes (like stars, hearts, and spirals) in the air with its hand.
The Result: The robot was able to draw these shapes very precisely (within about 12 millimeters of the target) while walking, squatting, and moving its feet.
The "Zero-Shot" Magic: They trained the robot entirely in a computer simulation and then put it on the real robot without any extra tuning. The robot worked immediately, proving the system is robust.

Summary

In short, HiWET is a new way to control humanoid robots that separates "where to go" from "how to move." By using a pre-trained guide to help with arm movements and keeping a strict focus on the real-world location of tasks (rather than just the robot's own body), it allows robots to walk and manipulate objects with a level of precision and stability that previous methods couldn't achieve.

Technical Summary: HiWET

Problem Statement

Humanoid loco-manipulation requires executing precise manipulation tasks while maintaining dynamic stability during base motion. Existing approaches face two primary limitations:

Body-Centric Formulation: Most command-driven methods operate in body-centric frames. This fails to inherently correct cumulative world-frame drift induced by legged locomotion, leading to degraded end-effector precision in the global frame.
Joint-Space Optimization: Whole-body motion imitation methods typically optimize joint-space errors rather than task-space precision, lacking theoretical guarantees for accurate end-effector tracking and often requiring costly dense reference trajectories.
Coupling Challenges: The tight coupling between upper and lower body dynamics means aggressive arm motions can destabilize gait, while locomotion impacts propagate to end-effectors. When tasks extend beyond the static reachable workspace, body-centric formulations do not explicitly address the need to actively transport the base to maintain reachability.

Methodology

The authors propose HiWET (Hierarchical World-Frame End-Effector Tracking), a hierarchical reinforcement learning (HRL) framework that decouples global reasoning from dynamic execution. The system is modeled as a Semi-Markov Decision Process (Semi-MDP) operating on two timescales.

1. Hierarchical Architecture

High-Level Policy (Commander, $\pi_H$ ): Operates at a lower frequency (every $K$ steps) in the world frame. It acts as a task planner, interpreting global end-effector targets and generating structured subgoals: base velocity ( $v_{des}^b$ ), body height ( $h_{des}$ ), and base-relative end-effector poses ( $bT_{des}$ ). It explicitly reasons about the geometric coupling between locomotion and manipulation to reshape the operational workspace.
Low-Level Policy (Tracker, $\pi_L$ ): Operates at high frequency to translate subgoals into joint-space commands. It ensures dynamic stability and physical feasibility. The humanoid is decomposed into upper-body (arms/waist) and lower-body (legs) subsystems.

2. Key Technical Components

Kinematic Manifold Prior (KMP): A pre-trained neural network that provides kinematically consistent joint references for the upper body based on desired end-effector poses and a waist regularization weight ( $\alpha$ $α$ ).
- The low-level policy learns residual corrections ( $\Delta q$ ) added to the KMP reference ( $\hat{q}$ ) rather than absolute joint targets. This reduces exploration dimensionality and mitigates kinematically invalid behaviors.
- The parameter $\alpha$ allows the high-level policy to modulate torso engagement, trading off between reachability (utilizing waist redundancy) and dynamic balance (stabilizing the Center of Mass).
State Estimator: A neural module trained to infer privileged state variables (e.g., base linear velocity, end-effector poses) from observation history, compensating for partial observability during deployment.
Importance Sampling (IS): To address the inefficiency of uniform Cartesian sampling (which often yields unreachable or ill-conditioned targets), the authors curate a prioritized dataset. This dataset is filtered by Inverse Kinematics (IK) reconstruction error and weighted by a manipulability index. The training command distribution is a mixture of uniform sampling and this curated prior.
Reward Shaping:
- Low-Level: Includes KMP reference tracking, knee-aware base height tracking (asymmetric weighting for flexion/extension margins), and regularization for smoothness.
- High-Level: Encourages workspace optimization (aligning base heading with targets), precise end-effector tracking in the world frame, and stability.

3. Training Procedure

The framework employs a two-stage training process:

Stage 1 (Tracker): The low-level policy is trained to follow base-relative end-effector commands using the KMP and importance sampling.
Stage 2 (Commander): The high-level policy is trained with the low-level policy frozen. A spatial curriculum is used, starting with targets near the robot's base and progressively expanding the range as tracking error decreases.

Key Results

The framework was validated on the Unitree G1 humanoid (29 DoF) in simulation and via zero-shot sim-to-real transfer.

Tracking Precision: In simulation, HiWET achieved a world-frame end-effector tracking error of 12.4 mm with low variance.
Ablation Studies:
- Removing the KMP increased hand error to 25.2 mm with 5 $\times$ higher variance, confirming the necessity of kinematic guidance.
- Removing the State Estimator degraded tracking by nearly 10 mm, highlighting the need for accurate feedback to compensate for locomotion oscillations.
- Removing Importance Sampling resulted in higher error (16.1 mm), demonstrating the value of focusing on functional workspace regions.
Comparison: HiWET outperformed the HOMIE baseline, reducing base linear velocity RMSE by approximately 20%.
Sim-to-Real: The low-level policy successfully transferred to the physical robot without fine-tuning, demonstrating stable locomotion under diverse manipulation commands.
Long-Horizon Tasks: The system successfully tracked complex geometric trajectories (stars, hearts, spirals) in the world frame, maintaining dynamic stability while navigating to target regions.

Significance and Claims

The paper claims that HiWET provides an effective and scalable solution for long-horizon humanoid loco-manipulation by explicitly addressing the geometric coupling between the base and the end-effectors.

World-Frame Consistency: By formulating the task as world-frame end-effector tracking, the controller actively compensates for base disturbances and reshapes the operational workspace through locomotion, a capability lacking in body-centric approaches.
Decoupled Control: The hierarchical separation of "where to go" (global planning) and "how to move" (dynamic execution) allows specialized optimization for both reachability and stability.
Efficiency: The integration of the Kinematic Manifold Prior via residual learning grounds the policy in valid manipulation manifolds, accelerating learning and improving tracking precision without sacrificing dynamic adaptability.

The authors conclude that this approach enables precise, stable end-effector tracking in long-horizon tasks and demonstrates robust zero-shot transfer to real-world hardware.