Technical Summary: Human2Humanoid

Problem Statement

Retargeting human motion to humanoid robots is a critical enabler for teleoperation, imitation learning, and human-robot interaction. However, this task faces two primary, coupled challenges:

1. Morphological Discrepancies: Significant differences exist between human and robot embodiments regarding skeletal topology, limb proportions, and degrees of freedom (DoFs). Simple geometric alignment often fails to preserve action semantics when body sizes and joint structures vary drastically.
2. Data Scarcity: High-quality, synchronized human-robot motion pairs are difficult to acquire and do not scale across different robot embodiments.
3. Physical Plausibility: Existing unpaired methods (often from computer graphics) prioritize visual plausibility but fail to ensure executable joint-space trajectories, stable ground contacts, or adherence to robot-specific kinematic limits. This leads to artifacts like foot skating, ground penetration, and body floating, which render motions unusable on real hardware.

Traditional optimization-based methods (e.g., inverse kinematics) are interpretable but sensitive to initial conditions and tuning, often producing semantically distorted poses under large morphology mismatches. Conversely, supervised learning methods require paired data that is rarely available, while existing unsupervised generative approaches lack the physical constraints necessary for robotic deployment.

Methodology

The paper proposes Human2Humanoid, an unsupervised motion retargeting framework designed to transfer human motions to humanoid robots without paired data. The framework is built upon a CycleGAN-style architecture but integrates specific mechanisms to address morphology and physics.

1. Network Architecture

• Skeleton-Aware Generators: Unlike methods that treat poses as flat vectors, the generators employ Skeleton-Aware Graph Convolutional Networks (GCNs). These utilize adjacency matrices to explicitly model the native topological structure of skeletal motions, capturing local joint correlations. The architecture includes encoders with SkeletonConv layers, a latent space, and decoders with unpooling operations to recover the target topology.
• Frame-wise Pose Discriminator: The discriminator operates on a frame-wise basis, evaluating realism at the joint level rather than collapsing to a single scalar score. It uses a dual-branch approach: a joint-wise branch for local realism and a full-body branch for pose coherence, combining them to score each joint independently.

2. Key Loss Functions and Constraints

To bridge the domain gap and ensure physical feasibility, the framework introduces three specific components:

• Morphology-Invariant End-Effector Consistency Loss ($L_{EE}$):
  To handle scale mismatches (e.g., human arms vs. compact robot arms), the method avoids aligning absolute Cartesian coordinates. Instead, it aligns end-effector trajectories relative to the T-pose of each embodiment. By normalizing displacements from the T-pose by the embodiment's body scale, the loss preserves motion semantics across different morphologies.
  $$L_{EE} = \mathbb{E} \left[ \frac{1}{T|E|} \sum_{t=1}^{T} \sum_{k \in E} \left\| \frac{FK_k(q_B, t) - FK_k(q_B^{Tpose})}{S_B} - \frac{FK_k(\hat{q}_A, t) - FK_k(q_A^{Tpose})}{S_A} \right\|_2^2 \right]$$

• Physics-Aware Feasibility Constraints:
  Explicit constraints are imposed during training to suppress artifacts common in unsupervised generation:

  • Foot Contact Constraint ($L_{con}$): Infers contact states from the source human motion (based on foot velocity) and penalizes target foot velocity when contact is expected, suppressing foot skating.
  • Foot Height Constraint ($L_{hei}$): Penalizes "floating" by ensuring that when the source foot is in contact, the target foot remains close to its nominal grounded height.
  • Joint Limit Constraints ($L_{lim}$): Penalizes predicted joint angles that exceed the robot's mechanical limits ($[q_{min}, q_{max}]$) to prevent hardware damage or emergency stops.

• Full Objective:
  The total generator loss combines the standard CycleGAN objectives (Adversarial, Cycle-consistency, Identity) with the proposed morphology and physics losses:
  $$L_G = \lambda_{GAN}L_{GAN} + \lambda_{cyc}L_{cyc} + \lambda_{idt}L_{idt} + \lambda_{EE}L_{EE} + \lambda_{con}L_{con} + \lambda_{hei}L_{hei} + \lambda_{lim}L_{lim}$$

Experimental Results

The method was validated on the Unitree G1 humanoid robot using unpaired data from the Motion-X dataset (human) and the PHUMA dataset (robot).

• Downstream Controllability: Evaluated using a fixed tracking policy in simulation. Human2Humanoid achieved the highest Success Rate (88.5%) and the lowest Tracking Error (0.12) among all baselines (including optimization-based methods PHC and GMR, and an industrial reference).
• Physical Feasibility: The method demonstrated superior suppression of contact artifacts. It achieved the lowest Ground Penetration (0.05 cm) and competitive Foot Skating rates (4.7%), outperforming optimization-based baselines which often suffered from joint-limit discontinuities or required per-motion tuning to avoid jitter.
• Ablation Study: Removing the morphology-invariant loss ($L_{EE}$) significantly degraded tracking success and increased error, confirming its role in semantic preservation. Removing physics constraints ($L_{con}, L_{hei}$) increased foot skating and ground penetration, validating their necessity for physical plausibility.

Key Contributions

1. Human2Humanoid Framework: An unpaired motion retargeting framework for heterogeneous humanoid robots that reduces dependence on robot-specific paired datasets.
2. Morphology-Invariant End-Effector Consistency Loss: A novel loss function that preserves motion semantics under large scale mismatches by aligning end-effector trajectories normalized relative to each embodiment's T-pose.
3. Physics-Aware Feasibility Constraints: The integration of explicit contact, height, and joint-limit constraints into the generative training process to mitigate deployment-critical artifacts like foot skating and ground penetration.

Significance and Claims

The paper claims that Human2Humanoid successfully bridges the gap between human motion priors and executable robot behaviors without requiring paired data. By combining unpaired learning with morphology-invariant semantic alignment and explicit physical priors, the framework achieves a robust balance between semantic fidelity (preserving the intent of the human motion) and physical executability (ensuring the motion is safe and stable for the robot).

The authors emphasize that while existing methods either rely on sensitive optimization or require expensive paired data construction, their approach offers a scalable solution that produces high-quality, trackable reference motions. The results suggest that the proposed method is particularly effective in handling the "embodiment gap," enabling humanoid robots to replicate complex human motions with higher reliability than current state-of-the-art baselines. Future work is noted to focus on generalizing to non-humanoid robots and end-to-end coupling with whole-body control policies.