💻 computer science

Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking

Humanoid-DART is a self-supervised framework that combines diffusion-based trajectory generation with reinforcement learning to enable humanoid robots to learn diverse loco-manipulation skills from sparse demonstrations by automatically exploring the goal space with minimal expert supervision.

Original authors: Pranav Debbad, Kanish Thiagarajan, Victor Dhédin, Shafeef Omar, Majid Khadiv

Published 2026-06-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pranav Debbad, Kanish Thiagarajan, Victor Dhédin, Shafeef Omar, Majid Khadiv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine teaching a humanoid robot to do complex chores, like picking up a box and moving it to a specific spot, or kicking a ball. Usually, to teach a robot this, you need to record hundreds of hours of humans doing these tasks perfectly. But that's expensive, slow, and hard to do for every possible variation (what if the box is heavier? What if the target is further away?).

The paper introduces Humanoid-DART, a clever system that teaches a robot to master these tasks starting with just a tiny handful of examples (as few as four). It does this by acting like a creative coach who doesn't just repeat what they've seen, but imagines new ways to do things and then practices them until they work.

Here is how the system works, broken down into simple parts:

1. The Two-Team Strategy

Instead of trying to teach the robot everything in one giant brain, Humanoid-DART splits the job into two specialized teams that help each other:

The "Dreamer" (The Diffusion Model): This is the creative planner. Its job is to imagine a path for the robot to follow. It looks at the few examples it has and starts "dreaming up" new, slightly different paths to reach new goals. Think of it like an artist sketching new routes on a map. At first, these sketches might be physically impossible (like walking through a wall or sliding on the floor), but they are great at exploring where the robot could go.
The "Athlete" (The Reinforcement Learning Policy): This is the physical executor. Its job is to take the "Dreamer's" sketch and actually try to run it on the robot. It acts like a strict coach. If the Dreamer sketches a path where the robot's foot slides or it falls over, the Athlete says, "Nope, that won't work," and fixes the movement to make it physically possible.

2. The "Practice Loop" (The Curriculum)

The magic happens in how these two teams talk to each other over time. The system runs in cycles, like a training camp:

Pick a Goal: The system picks a target (e.g., "Move the box to this new spot").
Dream: The "Dreamer" creates a rough plan to get there.
Test: The "Athlete" tries to execute the plan in a physics simulator.
Filter & Relabel:
- If the robot falls or fails, the plan is thrown out.
- The Secret Sauce: If the robot almost makes it (a "near miss"), the system doesn't treat it as a failure. Instead, it says, "Okay, you didn't reach the intended spot, but you successfully reached this spot." It relabels the attempt as a success for the new spot it actually reached.
Learn: The "Dreamer" learns from these successful (or near-successful) attempts to get better at dreaming up plans for those specific spots. The "Athlete" gets better at executing them.
Repeat: The system picks new, slightly harder goals based on what it just learned, expanding its skills like a snowball rolling down a hill.

3. The "Dual-Brain" Architecture

To make the "Dreamer" really good at handling both walking and holding objects, the paper gives it a special two-part brain structure:

The Navigator: Focuses on the big picture (where the robot's feet are going).
The Localizer: Focuses on the details (how the hands and joints move relative to the object).
By separating these, the robot learns to coordinate its whole body without getting confused, ensuring that when it walks toward a box, its hand is ready to grab it.

4. The Results

The researchers tested this on a real robot (a Unitree G1) and in simulation. They started with just four basic examples of a robot pushing, kicking, handing off, or picking up a box.

The Outcome: The system didn't just copy those four examples. It learned to do these tasks for goals that were 4 to 5 times further away than the original examples.
Coverage: For the "pick-and-place" task, the robot learned to cover 96% of the possible target areas, whereas other methods only covered a tiny fraction.

Summary

Humanoid-DART is like a student who starts with a few textbook examples but learns to solve thousands of new problems by:

Imagining new solutions.
Trying them out and seeing what actually works.
Celebrating "near misses" as new learning opportunities.
Slowly building a massive library of skills without needing a human to record every single variation.

The paper concludes that this approach allows robots to learn complex, whole-body tasks from very sparse data, making the process of teaching robots much faster and more efficient than before.

Technical Summary: Humanoid-DART

Problem Statement

Humanoid loco-manipulation requires coordinated whole-body control and dynamic object interaction. While recent advances in imitation learning and reinforcement learning (RL) have enabled impressive locomotion skills by tracking expert demonstrations, scaling these approaches to contact-rich loco-manipulation remains a significant challenge. Unlike locomotion, loco-manipulation involves a large, continuous task space defined jointly by body motion, object states, and environmental interactions. Variations in object pose, transport configuration, grasp strategy, and interaction timing create a combinatorial space that is prohibitively expensive to cover with human demonstrations alone. Existing methods typically rely on large-scale human motion datasets or focus on tracking a single reference behavior, failing to robustly solve diverse loco-manipulation tasks under varying conditions with sparse initial data.

Methodology

The paper proposes Humanoid-DART, a self-supervised, iterative framework designed to bootstrap from a sparse set of initial demonstrations and progressively expand its behavioral repertoire. The pipeline alternates between two tightly coupled components: a goal-conditioned motion generator and a physics-based motion tracking policy.

1. Algorithm Overview

The system operates as an evolutionary curriculum over a skill distribution defined by an elite archive $E$ of successful trajectories.

Goal Sampling: At each iteration, the curriculum samples new goal states around the current distribution in $E$ , targeting both refinement of existing skills and exploration of underrepresented regions.
Kinematic Generation: A goal-conditioned diffusion model ( $\pi_\theta$ ) generates kinematic reference trajectories for the sampled goals. These trajectories may contain physical inconsistencies (e.g., foot sliding, penetrations).
Physics Evaluation: A motion tracking policy ( $\pi_\phi$ ), trained via RL, executes these kinematic references in a physics simulator. It acts as a dynamic filter, converting potentially infeasible plans into physically consistent rollouts.
Selection and Relabeling: The resulting trajectories are evaluated using a fitness function $F(\tau)$ that measures tracking fidelity, contact correctness, and stability. Trajectories exceeding a threshold $\tau_f$ are retained in the elite archive $E$ . Crucially, successful rollouts that do not perfectly match the sampled goal are relabelled with their achieved task metrics, converting "near-misses" into valid training signals.
Iterative Update: Both the diffusion generator and the tracking policy are retrained on the expanded archive, progressively increasing the diversity of feasible trajectories and task space coverage.

2. Key Architectural Components

Kinematic Motion Generator (Diffusion Transformer):
- Dual-Branch Backbone: The architecture separates the state into a global stream (root motion and object-relative features) and a local stream (body pose/joints). The local stream cross-attends to the global stream, decoupling coarse navigation from fine-grained pose synthesis while maintaining synchronization.
- Structured Partial Unmasking: During training, the model selectively reveals noised feature groups from future frames. This encourages the planner to infer missing motion components from partial observations, learning correlations between feature groups (e.g., between whole-body pose and object interaction) rather than treating them independently.
- Inference: Uses auto-regressive synthesis over overlapping segments with history inpainting (RePaint-style) to ensure temporal consistency.
Motion Tracking Policy:
- A DeepMimic-inspired asymmetric actor-critic controller trained with Proximal Policy Optimization (PPO).
- It observes a 5-step reference horizon, proprioceptive state, and object pose, with the critic receiving privileged simulation state.
- Training incorporates domain randomization (object mass, friction, pushes) to ensure sim-to-real transfer.
Curriculum and Goal Relabeling:
- Frontier Target Sampling: The task space is discretized into bins. Sampling weights are assigned based on the distance of bins from the current elite set: Refine (near existing elites), Explore (intermediate distance), and Frontier (far-away, unsolved regions). Exploration bins are weighted highest to drive coverage expansion.
- Goal Relabeling: Instead of discarding rollouts that miss the sampled goal, the system retroactively treats the achieved state as the intended goal, providing a dense learning signal even when the generator is undertrained.

Key Contributions

Humanoid-DART Pipeline: The first iterative trajectory augmentation pipeline for learning goal-conditioned loco-manipulation skills from sparse demonstrations.
Novel Design Choices:
- A hierarchical goal-conditioned Diffusion Transformer (DiT) with a dual-branch architecture and structured partial unmasking to improve trajectory synthesis in sparse-data regimes.
- An RL whole-body controller conditioned on object pose to track generated trajectories.
- A curriculum-based algorithm with a goal relabeling strategy that converts near-missed rollouts into valid training signals.
Empirical Validation: Demonstration of the framework on four loco-manipulation tasks (push, kick, hand-off, pick-and-place) using a Unitree G1 humanoid. The system achieves task-space coverage from as few as four base demonstrations and generalizes to goals significantly beyond the seed distribution.

Experimental Results

Task-Space Coverage: In the pick-and-place task, Humanoid-DART achieved 96.4% coverage of the target task space starting from only four base demonstrations (approx. 20 seconds of motion). This represents a growth from 0.1% to near-complete coverage over four generations.
Generalization: The system successfully generated and tracked trajectories to goals 4–5 times further away than the base demonstrations, demonstrating robust generalization.
Real-World Deployment: Generated trajectories were successfully deployed on a physical Unitree G1 humanoid for push, kick, and pick-and-place tasks, confirming dynamic feasibility on hardware.
Ablation Studies:
- Architecture: Removing the dual-branch structure (collapsing to a single flat DiT) caused the goal-reaching success rate to drop from 1.00 to 0.25. Structured partial unmasking improved path straightness and grasp consistency.
- Baselines: Compared to "Parameterised Motion" (which focuses on narrow high-quality regions) and "Hierarchical Diffusion + RL" (trained jointly without the evolutionary loop), Humanoid-DART demonstrated superior task-space coverage. The evolutionary loop was identified as the primary driver of performance.
- Initialization: Dynamically feasible (DF) seed trajectories led to faster convergence and higher coverage compared to kinematically feasible (KF) seeds, which often contained physical inconsistencies that the diffusion model amplified.

Significance and Claims

The paper claims that Humanoid-DART addresses the bottleneck of data scarcity in humanoid loco-manipulation by leveraging generative models to iteratively expand the set of trajectory references. By decoupling high-level motion generation from low-level control and utilizing a curriculum with goal relabeling, the framework enables the learning of robust, goal-conditioned policies with minimal expert supervision. The authors emphasize that this approach allows for the discovery of diverse behaviors and long-horizon skills that extend well beyond the scope of the initial seed demonstrations, offering a scalable path toward generalist humanoid manipulation without the need for massive human motion datasets.

Limitations and Future Work: The current evaluation is restricted to a single object geometry and flat terrain. The authors note that the pipeline inherits biases from seed demonstrations and relies on a hand-tuned fitness function. Future work aims to extend the framework to diverse objects, contact properties, and uneven terrain, and to replace the fitness function with an adaptive, learned criterion.