💻 computer science

DynaRetarget: Dynamically-Feasible Retargeting using Sampling-Based Trajectory Optimization

This paper introduces DynaRetarget, a novel pipeline that leverages a Sampling-Based Trajectory Optimization (SBTO) framework to dynamically refine human motions into robust, dynamically feasible humanoid control policies, thereby enabling the generation of large-scale synthetic loco-manipulation datasets.

Original authors: Victor Dhedin, Ilyass Taouil, Shafeef Omar, Dian Yu, Kun Tao, Angela Dai, Majid Khadiv

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Victor Dhedin, Ilyass Taouil, Shafeef Omar, Dian Yu, Kun Tao, Angela Dai, Majid Khadiv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a talented human dancer who can perform a complex routine involving kicking, lifting, and pushing a heavy box. Now, you want a clumsy, heavy robot to copy this dance exactly.

The problem is that humans and robots are built very differently. If you simply tell the robot to copy the human's joint angles (a process called "retargeting"), the robot might try to kick the box with a foot that isn't there, or it might try to lift a box that is too heavy for its specific motor strength. The result is a "kinematic" plan that looks right on paper but is physically impossible to execute—the robot would fall over, slip, or crash.

This paper introduces DynaRetarget, a new system designed to fix these broken plans and turn them into real, working robot movements.

Here is how it works, using simple analogies:

1. The Problem: The "Bad Map"

Think of the initial robot plan as a map drawn by someone who has never seen the terrain. It shows the path, but it has holes in it (missing foot contacts) or leads off cliffs (impossible physics). If a robot tries to follow this map blindly, it fails.

Previous methods tried to fix this map in small steps. Imagine a hiker who only looks at the ground immediately in front of their feet. If the map says "jump," the hiker jumps. But if the jump is too far, they fall, and because they only looked one step ahead, they can't go back and change their decision to take a running start. They are "myopic" (short-sighted).

2. The Solution: The "Climbing Ladder" (SBTO)

The authors created a new method called Sampling-Based Trajectory Optimization (SBTO).

Instead of looking at just one step or trying to solve the whole 10-minute dance at once (which is too hard and confusing), SBTO acts like a climber building a ladder one rung at a time:

Step 1: It optimizes the first few seconds of the dance.
Step 2: Once the first few seconds are solid, it adds the next few seconds, using the first part as a stable base.
Step 3: It keeps adding time, refining the whole plan as it goes.

This is like fixing a long sentence by perfecting the first word, then the first phrase, then the first sentence, and so on. By the time it reaches the end of the dance, the entire sequence is physically consistent. It doesn't just look at the immediate next step; it keeps the whole future in mind, ensuring that the "kick" at the beginning sets up the "landing" at the end perfectly.

3. The Result: A "Polished" Performance

The system takes the rough, imperfect human-to-robot copy and smooths it out.

It fixes physics: If the robot was supposed to touch the ground but the math said it was floating, SBTO adjusts the joints so the foot actually hits the floor.
It handles heavy lifting: It figured out how to move a box that is heavier or shaped differently than the original human demonstration, without needing a new human to show it how.

4. The Payoff: Teaching the Robot to Learn

Once DynaRetarget creates this "perfect" physical plan, it feeds it to a robot brain (using Reinforcement Learning). Because the plan is physically realistic, the robot brain learns much faster.

The Analogy: Imagine teaching a student to drive. If you give them a map with a bridge that doesn't exist, they will crash and get confused. If you give them a map of a real road, they learn to drive smoothly and quickly.
The Paper's Claim: The authors tested this on hundreds of different moves (kicking, pushing, lifting). Their method succeeded nearly twice as often as the previous best methods. Furthermore, robots trained on these "perfect" plans learned to perform the tasks in the real world much better than robots trained on the "rough" plans.

Summary

DynaRetarget is a pipeline that takes a human's messy, imperfect motion data, uses a smart, step-by-step optimization ladder to fix the physics, and produces a flawless, real-world-ready instruction manual for a humanoid robot. It solves the problem of "short-sighted" planning and allows robots to learn complex, contact-heavy tasks (like kicking a ball or pushing a shelf) with high success rates.

Technical Summary: DynaRetarget

Problem Statement

Generating feasible loco-manipulation behaviors for humanoid robots is a complex challenge due to the underactuation of both the robot and the manipulated object, alongside intricate contact interactions. While traditional approaches rely on gradient-based trajectory optimization (TO) or deep reinforcement learning (RL), they face distinct limitations. Gradient-based TO often requires search-based augmentation for sufficient exploration, while RL struggles with exploration efficiency and frequently necessitates heavy reward shaping for individual motions.

A promising alternative involves retargeting human demonstrations to humanoid robots to alleviate exploration issues, using the resulting trajectories to train RL policies. However, existing retargeting methods primarily solve kinematic optimization problems. These approaches are susceptible to physical and geometric inconsistencies, such as foot skating, penetrations, and missing contacts, which are particularly detrimental for loco-manipulation tasks. Furthermore, recent sampling-based model predictive control (SBMPC) methods, while improving consistency, operate on short horizons. This myopic nature makes them prone to failure in long-horizon tasks, especially when the reference kinematic trajectories contain imperfections that cannot be corrected within a limited time window.

Methodology: DynaRetarget

The paper introduces DynaRetarget, a complete pipeline designed to retarget human motions to humanoid control policies. The core innovation is a novel Sampling-Based Trajectory Optimization (SBTO) framework that refines imperfect kinematic trajectories into dynamically feasible motions.

1. Pipeline Overview

The DynaRetarget pipeline operates in three stages:

Kinematic Retargeting: An inverse kinematics (IK) based retargeting process converts human-object demonstrations into kinematically feasible robot-object trajectories. This initial step often produces artifacts like missing contacts or geometric inconsistencies due to morphological differences.
Dynamic Refinement (SBTO): The kinematic trajectory serves as a reference for the SBTO module. SBTO refines the trajectory to ensure physical consistency, correcting artifacts such as missing contacts.
RL Policy Training: The refined, dynamically consistent trajectories are used to train an RL tracking policy in simulation using domain randomization. The learned policy is then transferred zero-shot to real-world humanoid hardware.

2. Sampling-Based Trajectory Optimization (SBTO)

Unlike standard SBMPC, which solves short-horizon problems in a receding fashion, SBTO incrementally advances the optimization horizon to solve the full long-horizon problem.

Incremental Horizon Expansion: SBTO optimizes control variables (interpolation knots) incrementally. It begins by optimizing the first knot, then the first two, and so on, until all knots are optimized. This process warm-starts larger-horizon problems using the solutions from shorter horizons.
Adaptive Convergence: The algorithm employs an adaptive criterion to determine when to increment the horizon. It monitors the diagonal values of the covariance matrix ( $\Sigma$ ) of the sampling distribution. An increment occurs only when the maximum diagonal value drops below a threshold ( $\sigma_{min}$ ), indicating sufficient convergence for the current subset of variables. This prevents premature convergence and allows the optimizer to escape poor local minima.
Efficiency Optimization (SBTO skip): To address computational costs in long-horizon problems, the authors propose an "SBTO skip" variant. This implementation caches initial trajectory segments that have already converged (variance below $\sigma_{skip}$ ) and skips their recomputation during subsequent roll-outs, significantly reducing simulation steps.
Cost Function: The optimization minimizes a cost function penalizing deviations in state position and velocity, tracking of torso/foot/hand poses, and contact-related terms (discouraging undesired collisions).

Key Contributions

Novel Optimization Framework: The paper introduces the first sampling-based trajectory optimization method capable of retargeting imperfect kinematic demonstrations into dynamically feasible humanoid loco-manipulation behaviors while considering the full problem horizon.
Superior Performance: The authors validate the approach on hundreds of motions (OmniRetarget dataset), demonstrating significantly higher retargeting success rates compared to state-of-the-art SBMPC baselines (e.g., SPIDER).
Robust Generalization: The framework generalizes across varying object properties (mass, size, geometry) using a single tracking objective, enabling the refinement of diverse demonstrations without retraining.
Downstream RL Benefits: The paper demonstrates that trajectories refined by SBTO significantly improve the training efficiency and robustness of downstream RL tracking policies, enabling zero-shot transfer to real hardware.

Results and Evaluation

The authors evaluated SBTO on 285 motions involving a G1 humanoid robot interacting with a box (picking, kicking, pushing, dragging).

Success Rate: SBTO achieved a 74.6% success rate (76.8% with the "skip" variant), nearly double the 37.9% success rate of the SBMPC baseline (SPIDER). Success was defined as maintaining object position error $<10$ cm and rotation error $<25^\circ$ .
Trajectory Quality: SBTO produced smoother trajectories compared to the baseline.
Computational Efficiency: While standard SBTO was computationally expensive, the "SBTO skip" variant reduced computational costs by a factor of three, making it more efficient than the baseline while maintaining higher performance.
Algorithm Analysis: Experiments showed that SBTO's incremental warm-starting process effectively mitigates convergence issues found in fixed-horizon TO. Furthermore, the "effective horizon" of SBTO (the duration over which early decision variables are actively refined) was found to be substantially longer than the per-increment look-ahead or the fixed horizon of SBMPC, explaining its ability to handle long-horizon tasks.
RL Transfer: RL policies trained on SBTO-refined trajectories converged significantly faster and achieved higher success rates (97.09%) in real-world transfer compared to policies trained on kinematically retargeted data alone (79.41%). The SBTO-trained policies successfully learned challenging behaviors, such as object manipulation using legs, without additional curriculum shaping.

Significance

The paper claims that DynaRetarget addresses a major bottleneck in real-world data collection for humanoid robots: the scarcity of high-quality, physically consistent loco-manipulation demonstrations. By enabling the robust retargeting of diverse human demonstrations into dynamically feasible motions, the framework opens the door to generating large-scale synthetic datasets. This capability facilitates the training of robust RL policies that can be directly deployed on real-world hardware, bridging the gap between simulation and reality for complex contact-rich tasks.