💻 computer science

Learning Multi-Humanoid Pickup and Transport via Decentralized Object-Centric Control

This paper proposes a decentralized object-centric control framework that enables multi-humanoid teams to cooperatively pick up and transport objects of varying sizes using gripperless bimanual pinching, demonstrating that policies trained on single-robot tasks transfer effectively to multi-robot scenarios while further benefiting from explicit coordination training.

Original authors: Bikram Pandit, Mohitvishnu S. Gadde, Aayam Kumar Shrestha, Alan Fern

Published 2026-09-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bikram Pandit, Mohitvishnu S. Gadde, Aayam Kumar Shrestha, Alan Fern

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where robots can do more than just move in straight lines or push simple boxes. They can walk, balance, and carry heavy things, much like humans do. This field, known as humanoid robotics, has made great strides in teaching single machines to pick up objects and carry them from one place to another. However, a single robot has limits. It can only lift so much weight, and it can only reach so far. When the object is too big, too heavy, or shaped in a strange way, one robot simply cannot do the job alone. This is where the idea of teamwork comes in. Just as a group of people might gather to move a large piece of furniture, a team of robots could coordinate to carry a heavy load together. The challenge has been teaching these machines to work together without needing a central commander to tell each one exactly what to do at every second. If they cannot communicate with each other, how can they know when to lift, when to walk, and how to keep the object steady?

Researchers at Oregon State University have developed a new way to solve this problem, allowing teams of humanoids to cooperate without talking to one another. Their work focuses on a system where each robot is given a specific spot on the object it needs to hold. Instead of trying to understand the entire shape of the object or the position of every other robot, each machine only needs to know where its own hands should touch the load. This approach, which the authors call decentralized object-centric control, treats the object as the center of the action. Each robot is assigned a small region on the object's surface, like a specific patch on a large table, and learns to grab that spot using its two hands in a pinching motion. Once attached, the robot follows a simple instruction to move the object in a certain direction, such as forward, sideways, or up. Because every robot follows the same basic rule based on its own local spot, they naturally move together without needing to send messages back and forth.

The researchers trained their robots using a computer simulation, starting with just one robot learning to pick up and carry boxes of different sizes and weights. They taught the robot to approach an object, grab it with its hands, lift it, and walk while keeping it steady. Once the single robot mastered this, the researchers introduced a second robot to the simulation. They did not teach the robots to coordinate with each other directly. Instead, they let the second robot learn the same way, using its own assigned spot on the object. Surprisingly, the skills the robots learned when working alone transferred very well to working in a group. A policy trained on a single robot could handle a team of up to ten robots without any extra training. This suggests that the core difficulty of the task is not the coordination between robots, but rather the ability of each robot to hold its part of the load securely. When the robots work together, they share the weight, and the object becomes more stable. In tests with ten robots carrying a heavy load, the object tilted much less than when a single robot carried it, showing that the teamwork made the transport smoother and safer.

The system proved to be incredibly flexible. The researchers tested it with objects that were never seen during training, including a metal cabinet, a refrigerator, a wooden log, a bed, and a concrete slab. The robots successfully picked up and moved all of these items, even though they had never encountered these specific shapes before. They also tested the system with teams of varying sizes, from one robot to ten, and found that the performance remained high across the board. The robots could even pass an object from one to another. In these handover scenarios, one robot would hold the object while a second robot approached, grabbed its own spot, and then the first robot would let go. This happened smoothly whether the robots were standing still, walking, or even standing at different heights. The researchers also measured how much energy the robots used. They found that while adding a second robot initially cost a bit more energy due to the need for coordination, adding more robots beyond that did not significantly increase the energy cost per unit of weight moved. This means that larger teams can carry heavier loads without becoming inefficient.

To prove that this was not just a computer trick, the researchers took their system to the real world. They used two actual humanoid robots, called Digit V3, to perform the tasks. Without any new training or adjustments, the robots successfully picked up a cardboard box, carried it together, and passed it from one to the other. The robots relied on external sensors to see where the box was relative to their own bodies, but once they had that information, they executed the same decentralized control strategy they had learned in the simulation. This successful transfer from simulation to reality shows that the approach is robust enough for real-world use. The researchers note that their current system works best when all robots have the same role and simply hold different parts of the object. It does not yet handle situations where robots need to take on very different roles, such as one leading and the other following, or where the robots have grippers instead of flat hands. However, the results suggest that a simple, local way of thinking about the task can solve complex problems of teamwork. By focusing on the object and the specific spot each robot needs to hold, the system allows a group of machines to act as a single, coordinated unit, capable of moving the heavy and awkward things that a single robot could never manage alone.

Technical Summary: Learning Multi-Humanoid Pickup and Transport via Decentralized Object-Centric Control

Problem Statement

Recent advances in humanoid loco-manipulation have enabled single robots to pick up and transport objects of varying sizes and weights. However, many real-world payloads remain too large, heavy, or irregular for a single humanoid to handle. These tasks require teams of robots to coordinate whole-body motion while interacting through a shared load. Existing approaches to cooperative humanoid transport are sparse, often relying on analytic models, fixed team sizes, or specific leader-follower assumptions. Furthermore, prior learning-based methods have struggled to generalize across varying team sizes, attachment layouts, and object geometries without per-task redesign.

The core challenge addressed is developing a learning-based framework for multi-humanoid collaborative loco-manipulation that supports variable-sized teams, diverse object geometries, and seamless robot-to-robot handovers, all without requiring direct inter-robot communication or precise global pose commands.

Methodology

Decentralized Object-Centric Control (decMHT)

The authors introduce decMHT, a decentralized, object-centric control abstraction. This framework unifies single-robot pickup, cooperative multi-robot transport, and robot-to-robot handovers under a single policy.

1. Interface Design:
The system relies on two primary inputs provided to each robot:

Attachment Spec ( $A(r)$ ): A local geometric definition provided at the start of an episode. It specifies a local attachment frame relative to the robot's torso, two symmetric attachment points (left and right), and a frontal clearance distance. Crucially, it does not describe the full payload geometry, allowing the same representation to handle compact boxes, elongated bars, or irregular panels.
Object-Centric Command ( $C_{pl}$ ): A high-level command specifying the desired planar velocity, yaw rate, and height of a reference frame on the object (e.g., its centroid). This global command is transformed into a local command ( $C_{cf}^{(r)}$ ) for each robot using a fixed rigid transformation based on the robot's attachment offset.

2. Control Architecture:
The system employs a hierarchical decentralized architecture where each robot executes the same policy independently using only local observations:

Low-Level Locomotion Policy ( $\pi_l$ ): A pre-trained, frozen neural network that converts commanded base velocities, yaw rates, and torso heights into leg joint targets.
High-Level Manipulation Policy ( $\pi_h$ ): A recurrent policy (two-layer LSTM) that takes proprioceptive state, the attachment spec, the local command, and a mode flag as input. It outputs target upper-body joint positions and locomotion commands. The mode flag ( $m^{(r)}$ ) switches the robot between idle (maintain nominal pose) and track (approach, attach, and track the object).

3. Training Approach:
The high-level policy is trained using Independent Proximal Policy Optimization (IPPO) in simulation (IsaacLab with Agility Robotics Digit V3 models).

Curriculum: Training begins with single-robot pickup and transport tasks, progressing through stages of assisted contact formation, unassisted holding, height tracking, and full command tracking.
Fine-Tuning: The single-robot policy is fine-tuned on two-robot cooperative episodes to expose the controller to the coupled dynamics of shared payloads.
Generalization Strategy: The policy is never trained with $N \ge 3$ robots. Evaluation on larger teams (up to 10 robots) is performed zero-shot.
Domain Randomization: To ensure robustness and sim-to-real transfer, training includes randomization of sensor noise, friction, motor stiffness, and external perturbations.

Key Contributions

decMHT Abstraction: Introduction of a high-level, object-centric interface that supports variable team sizes, attachment layouts, and object geometries without per-task redesign.
Hierarchical Decentralized Control: Development of a control architecture and training curriculum enabling humanoids to coordinate through a shared object using only local observations and no direct communication.
Scalability and Zero-Shot Transfer: Demonstration in simulation that a single controller trained on at most two robots supports teams of up to ten humanoids across diverse object geometries and attachment configurations, including handovers across multiple elevation differences.
Sim-to-Real Validation: Successful deployment on two real Digit V3 humanoids performing collaborative object manipulation without additional real-world fine-tuning.

Experimental Results

Scalability and Performance

Team Size: The system was evaluated with teams of $N \in \{1, \dots, 10\}$ robots. Pickup success rates remained above 97% for all team sizes. Transport quality generally improved with team size; linear and height tracking errors decreased, and payload tilt was significantly reduced (RMS tilt dropped from 8.04° for one robot to 1.21° for ten robots).
Payload Mass: The policy generalized zero-shot to payloads up to 50 kg, far exceeding the training mass range (0.5–1.5 kg). Increasing team size allowed the system to lift heavier loads while maintaining stability.
Object Geometry: The framework generalized to unseen geometries (metal cabinet, fridge, wooden log, bed, concrete slab) without retraining. While pickup success was high across all objects, the "bed" (heavy, thin grasp regions) presented the most challenge, with an 85.2% success rate for decMHT compared to 44.1% for the single-robot-trained ablation.

Comparative Analysis

vs. Single-Robot Training (decMHT(1)): The single-robot-trained policy achieved comparable pickup rates but exhibited higher tracking errors, drop rates, and payload tilt during transport. This indicates that while the attachment abstraction captures the mechanics of lifting, multi-robot fine-tuning is necessary to learn the coupled dynamics required for stable transport.
vs. Centralized and Specialized Baselines: decMHT performed comparably to centralized baselines (trained on joint states) and object-specialized baselines (retrained per object), despite using a decentralized, model-free approach.
Cost of Transport (CoT): Energy efficiency (CoT) increased sharply from one to two robots but plateaued as the team grew larger. decMHT maintained lower CoT than the single-robot ablation for all multi-robot teams.

Handover Capabilities

The system successfully demonstrated robot-to-robot handovers in static, dynamic (moving object), and cross-level (different elevations) configurations. decMHT outperformed the single-robot ablation, particularly in dynamic scenarios where the receiving robot had to establish contact while the object was in motion.

Sim-to-Real Transfer

The approach was validated on two real Digit V3 humanoids. Using an external sensor to estimate the attachment frame pose, the robots successfully performed single-robot pickup, multi-robot cooperative transport, and handovers purely by toggling mode flags, with no policy modification or real-world fine-tuning.

Significance and Claims

The paper claims to be the first to demonstrate a learned, decentralized multi-humanoid loco-manipulation framework that generalizes across varying team sizes in robot-realistic simulation and deploys on real humanoid hardware.

The authors emphasize that the attachment-based abstraction captures the essential structure needed for coordination, allowing policies trained on single-robot tasks to transfer nontrivially to cooperative settings. However, they also note that explicit multi-robot training is beneficial for learning the specific coordination dynamics that improve transport stability and efficiency.

The work highlights that a common object-centric control interface can unify diverse manipulation modes (pickup, transport, handover) and scale to large teams without the need for complex global planning or direct robot-to-robot communication, addressing a significant gap in current cooperative humanoid robotics.