🤖 machine learning

HOIST: Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads

The paper presents HOIST, a framework that combines imitation learning from VR teleoperation with sample-efficient reinforcement learning to enable humanoid robots to accurately manipulate and place suspended, underactuated loads.

Original authors: Songyang Liu, Shunyu Yao, Dingyuan Huang, Shuai Li

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Songyang Liu, Shunyu Yao, Dingyuan Huang, Shuai Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to move a heavy, swinging chandelier that is hanging from the ceiling by a long rope. You can't grab the chandelier directly; you can only push it with your hands or body. If you push too hard, it swings wildly. If you stop too early, it keeps rolling forward. If you stop too late, it overshoots the spot where you want it to land. This is the tricky problem the paper HOIST tries to solve using a humanoid robot.

Here is a simple breakdown of what the researchers did and how they did it:

The Problem: The "Swinging Chandelier" Dilemma

In construction and warehouses, workers often have to guide heavy loads hanging from cranes. It's dangerous because if the load swings, it can hit people. Robots usually struggle with this because:

They can't grab the load: The robot has to push it while it's still hanging.
It's unpredictable: The load acts like a pendulum. It keeps moving even after the robot stops pushing.
Learning is hard: If you let a robot learn by trial and error (Reinforcement Learning) on a real robot, it might crash, break things, or hurt itself. If you just teach it by watching humans (Imitation Learning), it might copy the human's movements but fail to stop in the exact right spot because it doesn't understand the physics of the swing.

The Solution: HOIST (The "Smart Coach" Approach)

The team created a system called HOIST. Think of it as a three-step training program for the robot:

Step 1: The "Shadowing" Phase (Imitation)
First, a human operator wears a Virtual Reality (VR) headset and uses controllers to "ghost" the robot. The human guides the robot through the task of pushing the swinging load to a target. The robot watches and learns, "Okay, this is how a human moves their body to push this thing."

Analogy: This is like a dance student watching a master dancer and trying to copy their steps. The student gets the general flow right but might still trip on the final turn.

Step 2: The "Practice Run" Phase (Autonomous Rollouts)
Next, the robot tries the task on its own using what it learned from the human. It pushes the load, but because it's a robot, it might stop a little too early or a little too late. The system records these attempts.

Analogy: The student tries the dance routine alone. They get the rhythm, but they still miss the final pose by a few inches.

Step 3: The "Fine-Tuning" Phase (Sample-Efficient RL)
This is the magic part. The system doesn't make the robot start over from scratch. Instead, it looks at the "practice runs" and asks: "How can we tweak the very first thought or 'nudge' the robot makes to fix the final landing spot?" It uses a special math trick (called flow-matching) to adjust the robot's internal "steering" without changing the whole robot's brain.

Analogy: Imagine a coach watching the student's practice run and saying, "You're doing great, but if you just tilt your head slightly to the left at the very start, you'll land perfectly in the center." The robot learns this tiny adjustment.

The Results: How Well Did It Work?

The researchers tested this in a computer simulation and on a real robot.

Better than just copying: When they only used the "Shadowing" phase (Imitation), the robot was safe but often missed the target by a significant amount (about 22 cm off in the simulation).
Better than just adding more videos: They tried teaching the robot with even more human videos, but it didn't help much.
The Winner: By adding the "Fine-Tuning" phase (just 30 practice runs), the robot got much closer. In the simulation, it reduced the error by nearly 20 cm and stopped the swinging much more effectively.

The Catch (Limitations)

The paper admits one big weakness: The robot's "muscles" (the low-level controller that actually moves the joints) are fixed and haven't been retrained. The robot can only push with a certain amount of force. If the load is too heavy, the robot might not be strong enough to move it effectively, no matter how smart its "brain" is.

Summary

HOIST is a method that teaches a robot to push a swinging, hanging load by first copying a human, and then making tiny, smart adjustments based on its own practice runs. It's like teaching a robot to be a crane operator's assistant: it learns the moves from a human, then practices just enough to master the tricky art of stopping exactly where it needs to be without knocking the load over.

Technical Summary: HOIST – Humanoid Optimization with Imitation and Sample-efficient Tuning for Manipulating Suspended Loads

1. Problem Definition

The paper addresses the challenge of humanoid hoisting: using a humanoid robot to position an externally suspended, underactuated payload. Unlike standard pick-and-place tasks where the robot rigidly grasps an object, the payload in this scenario is suspended (e.g., by a crane or cable) and behaves as an oscillatory pendulum. The robot cannot directly actuate the payload; it can only influence the load's motion through whole-body locomotion, body posture, and intermittent contact forces (pushing).

Key difficulties include:

Underactuation and Dynamics: The payload continues to swing after contact ceases, leading to overshoot and residual oscillation.
Safety and Efficiency: Direct Reinforcement Learning (RL) from scratch on real humanoids is unsafe due to the risk of unstable motion and hard contacts. Conversely, pure Imitation Learning (IL) from demonstrations often fails to optimize final placement accuracy or correct for closed-loop execution errors.
Task Complexity: Success requires coordinated whole-body balance, precise contact timing, and specific stopping behaviors to dampen the payload's swing.

2. Methodology: The HOIST Framework

HOIST (Humanoid Optimized with Imitation and Sample-efficient Tuning) is a three-stage pipeline designed to bridge the gap between safe initialization and task-level optimization.

A. Teleoperation Data Collection

Human operators use VR headsets and controllers to teleoperate a humanoid robot through the hoisting sequence (approach, contact, push, stop).
Trajectories are recorded as state-action pairs, including visual observations, proprioceptive states, and planner commands.
These demonstrations provide a safe, initial behavior policy that avoids catastrophic failures during early deployment.

B. Supervised VLA Finetuning

A high-level Vision-Language-Action (VLA) policy is finetuned using the collected teleoperation data. The base model is adapted from GR00T N1.6.
Input Modalities: The policy processes multimodal inputs including onboard RGB/depth images, external side-view images (to capture payload-target relationships), proprioceptive states (joint/base/IMU), language instructions, and previous navigation commands.
Architecture Modifications:
- Navigation commands are added as an explicit modality to the vision-language model.
- Depth images and ego-velocity are added to the flow-matching-based action expert to condition on scene geometry and physical state.
- Only the final four layers of the VLM, the diffusion-transformer action expert, and new modality encoders are finetuned; the rest of the foundation model remains frozen to preserve general priors.
Output: The policy predicts a short-horizon chunk of relative command increments ( $\Delta U_t$ ) for hand targets, head pose, navigation, and base height. These are accumulated into full planner commands.

C. Fixed Whole-Body Execution

The high-level planner commands are executed by a fixed, decoupled whole-body controller (based on the GR00T WBC stack).
The low-level controller handles balance, joint stabilization, and contact forces. Crucially, the low-level controller is not finetuned; all task adaptation occurs at the high-level command generation stage.

D. Sample-Efficient RL Refinement

To correct systematic biases in the imitation-learned policy (e.g., consistent over/under-pushing), HOIST employs iterative batched RL.
Process:
1. The finetuned VLA policy is deployed to collect a small batch of autonomous rollouts.
2. These rollouts are labeled with hoisting-specific rewards (a weighted combination of translational placement error and raw angular error).
3. An actor-critic module is trained offline to predict an "initial-noise steering vector" for the flow-matching action expert. This actor deterministically outputs a correction vector to steer the frozen VLA policy's generation process.
4. The VLA policy itself remains frozen; only the noise-steering actor-critic is updated.
5. VLM features from the frozen policy are cached and reused to maximize efficiency.
This cycle repeats, allowing the system to refine placement accuracy and stopping behavior with minimal real-world data.

3. Key Contributions

Problem Formulation: The introduction of "humanoid hoisting" as a distinct whole-body loco-manipulation problem involving externally suspended, oscillatory payloads.
The HOIST Framework: A novel pipeline combining VR teleoperation, supervised VLA finetuning, fixed whole-body execution, and sample-efficient RL refinement (via noise steering) for suspended-payload manipulation.
Empirical Evaluation: Comprehensive experiments in simulation and on a real humanoid platform, demonstrating that RL refinement outperforms simply adding more demonstrations and analyzing the impact of input modalities and rollout counts.

4. Experimental Results

The paper evaluates HOIST against baselines including pure imitation (VLA-50, VLA-80) and human teleoperation.

Performance Gains:
- In simulation, HOIST (50 demos + 30 RL rollouts) reduced translational placement error by 19.9 cm and raw angular error by 3.56° compared to pure VLA rollouts (VLA-50).
- On the real platform, HOIST achieved a total Manhattan position error of 6.38 cm, outperforming VLA-80 (8.57 cm) which used 30 additional demonstrations but no RL.
RL vs. Additional Demonstrations: The results indicate that optimizing closed-loop placement rewards via RL is more effective for correcting final-position errors than simply increasing the number of teleoperated demonstrations.
Ablation Studies:
- Adding depth images and navigation command conditioning to the VLA input significantly improved the initial imitation-learned policy before RL refinement.
- Residual motion analysis (via IMU on the payload) showed that while HOIST yields higher accelerometer readings (indicating active correction), it reduces gyroscope responses (angular motion) by approximately half compared to imitation-only policies, suggesting better damping of residual swing.

5. Significance and Claims

The paper claims that HOIST demonstrates the potential of humanoids for underactuated material-handling tasks, a domain currently dominated by human workers due to safety hazards (struck-by, caught-between).

Safety and Efficiency: By starting from safe teleoperation data and using RL only for high-level command refinement, HOIST avoids the safety risks and sample inefficiency of training from scratch on real hardware.
Bias Correction: The work highlights that imitation learning alone suffers from deployment-time distribution shifts (where the robot's own errors alter the task dynamics). HOIST's RL refinement specifically targets these systematic biases to improve final accuracy.
Modest Limitations: The authors acknowledge that the system is currently limited by the fixed low-level whole-body controller, which cannot explicitly adapt forces to different payload weights. Consequently, the system struggles with heavier objects where larger, adaptive forces are required.

In summary, HOIST presents a practical approach to enabling humanoids to manipulate suspended loads by combining the safety of imitation learning with the optimization power of sample-efficient reinforcement learning, specifically tailored to the dynamics of underactuated, oscillatory systems.