Multi-Quadruped Cooperative Object Transport: Learning Decentralized Pinch-Lift-Move

This paper presents a decentralized, communication-free learning framework for teams of quadruped robots to cooperatively transport ungraspable objects through physical contact alone, utilizing a hierarchical policy and a specialized reward formulation to achieve robust, scalable coordination without rigid mechanical coupling.

The Gist

Imagine you and three friends are trying to move a giant, awkward sofa across a room. The problem? The sofa has no handles, no hooks, and it's too heavy for any one of you to lift alone. You can't tie yourselves to it with ropes, and you can't talk to each other while you're doing it. You just have to stand around it, pinch it with your hands, lift it up, and walk in perfect sync without dropping it.

That is exactly the challenge this paper solves, but with robots instead of people.

Here is the story of how they taught a team of four-legged robots (like dogs) with arms to move heavy, ungraspable objects together.

The Problem: The "Silent Sofa" Challenge

Usually, when robots move things together, they use rigid metal bars to lock themselves to the object (like a forklift) or they use grippers to grab handles. But in the real world, many things—logs, furniture, irregular boxes—can't be grabbed or locked onto.

The researchers wanted to know: Can a team of robots coordinate to move these slippery, ungraspable objects just by pressing against them, without talking to each other or using any physical locks?

The Solution: "decPLM" (The Silent Dance)

They created a system called decPLM (Decentralized Pinch-Lift-Move). Think of it as teaching the robots a "silent dance."

1. The Pinch: The robots approach the object and press their arms against it.
2. The Lift: They all push up at the exact same time.
3. The Move: They walk forward, keeping the object steady, without ever letting go.

The magic is that they do this without a leader. There is no central computer telling Robot A to move left. There is no radio chatter. They just look at their own sensors and move.

The Secret Sauce: The "Constellation" Reward

How do you teach a robot to act like it's rigidly glued to an object when it's actually just pressing against it?

The researchers used a clever trick in the robot's "brain" (its training reward system). They invented something called a Constellation Reward.

• The Analogy: Imagine you are trying to keep a specific pattern of stars (a constellation) in the sky aligned with a pattern of lights on your shirt. If your shirt tilts, the stars look misaligned. If you move your body, the stars shift.
• The Robot's Job: The robot is trained to keep a specific pattern of points on its own body and arm perfectly aligned with a matching pattern on the object.
• The Result: Even though the robot isn't physically glued to the box, the reward system makes it behave as if it were. It creates an invisible "rigid connection" in the robot's mind. If the box starts to tilt, the robot feels like its "constellation" is broken and immediately corrects its posture to fix it.

The Training: From Two to Ten

One of the most surprising findings was how the robots learned.

• The "Small Class" Method: The researchers trained the robots using a team of just two robots.
• The "Big Class" Result: When they tested these same robots with 10 robots, it worked perfectly! The robots didn't need to be retrained. Because they learned the principles of the dance (how to pinch, lift, and sync) rather than memorizing specific steps for a specific number of friends, they could scale up effortlessly.

It's like teaching a child to ride a bike with training wheels; once they learn the balance, they can ride a tricycle, a bike, or even a unicycle without needing a new lesson.

Real-World Test: The "Sim-to-Real" Leap

Finally, they took the robots out of the computer simulation and into the real world. They used real Unitree Go2 robots (quadrupeds with arms) to move lightweight boxes.

• The Challenge: Real life is messy. The robots' joints weren't perfectly calibrated, the boxes were slightly squishy, and the robots couldn't generate as much force as the simulation promised.
• The Outcome: Despite these hurdles, the robots successfully performed the pinch-lift-move dance. They moved the boxes without dropping them, proving that this "silent coordination" works in the real world, not just in a video game.

Why Does This Matter?

This technology is a big step forward for:

• Disaster Relief: Imagine a team of robots moving heavy debris or logs after an earthquake without needing a human to program every single move.
• Construction: Moving heavy, awkward materials on a construction site where nothing has handles.
• Logistics: Moving furniture in warehouses where robots can't just "grab" things.

In short, the paper shows that with the right "dance moves" (the constellation reward) and a little bit of practice, a team of independent robots can work together as one giant, coordinated unit, even when they can't talk to each other and can't hold on tight.

Technical Summary

Here is a detailed technical summary of the paper "Multi-Quadruped Cooperative Object Transport: Learning Decentralized Pinch-Lift-Move" by Pandit, Shrestha, and Fern.

1. Problem Formulation

The paper addresses the challenge of decentralized cooperative transport of ungraspable objects (e.g., furniture, logs, boxes) using a team of $N$ quadruped robots equipped with arms.

• The Core Challenge: Unlike previous works where robots are rigidly coupled to objects via mechanical fixtures, this setting requires robots to manipulate objects solely through physical contact forces (pinching).
• Constraints:
  • No Communication: Robots cannot exchange state information or coordinate via explicit messaging.
  • No Centralized Control: There is no central planner dictating individual robot actions during execution.
  • No Rigid Coupling: Robots must maintain stable contact and synchronize lifting/moving without mechanical guarantees.
• Objective: A team of robots must coordinate to Pinch (establish contact), Lift (synchronize vertical motion), and Move (transport the payload along a trajectory) while maintaining force closure and stability.

2. Methodology: decPLM Framework

The authors propose decPLM (Decentralized Pinch-Lift-Move), a hierarchical control framework trained via Multi-Agent Reinforcement Learning (MARL).

A. Hierarchical Policy Architecture

To manage the high-dimensional control space, the system uses a two-level hierarchy:

1. Low-Level Policy ($\pi_b$): A pre-trained locomotion controller for the quadruped base. It takes proprioceptive state and desired base velocity commands, outputting joint targets for the legs. This layer is frozen during the main training.
2. High-Level Policy ($\pi_h$): A decentralized policy that coordinates task-level behavior.
   • Inputs: Local proprioception ($s$), contact frame commands ($C_{cf}$), and optionally the contact frame pose ($bT_{cf}$).
   • Outputs: Arm joint targets and base velocity commands.
   • Execution: Each robot runs the same policy independently using only local observations.

B. Training Paradigm

• Algorithm: Centralized Training and Decentralized Execution (CTDE) using Multi-Agent Proximal Policy Optimization (MAPPO).
• Asymmetric Critic: During training, critics have access to privileged information (global state and other robots' observations) to stabilize learning, while actors (the policies) only use local observations.
• Curriculum Learning: Training is divided into three progressive phases:
  1. Pinch: Robots learn to establish contact on a heavy, static object (100kg) to prevent premature movement.
  2. Lift: Robots learn to synchronize vertical lifting with randomized payload masses (0.1–2 kg).
  3. Move: Full transport with planar velocity and yaw commands.

C. Key Innovation: Constellation Reward

The most significant contribution is the Constellation Reward, designed to enforce "rigid-like" behavior without mechanical constraints.

• Concept: Instead of tracking a single point, the reward minimizes the Mean Squared Error (MSE) between two sets of "landmark points" (constellations):
  1. End-Effector Constellation: Points on the robot's arm pad vs. points on the target contact frame. This ensures surface normal alignment and contact stability.
  2. Base Tracking Constellation: Points on the robot's base vs. target points derived from the payload's motion. This ensures the robot's body moves kinematically consistent with the payload.
• Effect: This reward implicitly couples position and orientation, encouraging the team to behave as if they are rigidly attached to the object, even though they are only pinching it.

3. Key Contributions

1. Decentralized Contact-Only Manipulation: A novel framework enabling teams of quadrupeds to lift and transport ungraspable objects without communication or rigid coupling.
2. Constellation Reward Formulation: A unified reward function that enforces both positional and orientational alignment, effectively simulating rigid mechanical constraints through learned behavior.
3. Scalability and Generalization: Demonstrated that policies trained on a small team ($N=2$) generalize effectively to larger teams ($N=10$) and diverse payloads without retraining.
4. Sim-to-Real Transfer: Successful deployment on physical Unitree Go2 robots with D1 arms, validating the approach in the real world despite hardware limitations.

4. Experimental Results

Experiments were conducted in simulation (IsaacLab) and on physical hardware.

• Constellation Reward Efficacy: Models using the constellation reward ($const+$) significantly outperformed those without it ($const-$), achieving lower tracking errors and drop rates. Crucially, the constellation reward was found to be more critical than continuous access to contact frame pose information.
• Team Size Generalization:
  • Policies trained with $N=2$ robots scaled seamlessly to $N=10$.
  • Performance improved with team size: Linear velocity tracking error dropped by 80% (from 0.1 to 0.02 m/s) when scaling from 2 to 10 robots.
  • Drop rates decreased from 5% (2 robots) to <1% (10 robots).
• Force Distribution: The learned policies naturally balanced forces. In asymmetric configurations (e.g., 3 robots), robots on one side collectively applied forces equal to the single robot on the opposite side to maintain stability.
• Payload Mass & Complexity:
  • For heavy payloads (>15 kg), continuous contact frame pose information ($cf+$) became crucial to prevent failure.
  • The system generalized to out-of-distribution geometries (logs, barrels, couches) trained only on boxes, often outperforming specialized controllers trained directly on those specific shapes.
• Sim-to-Real: The system successfully executed coordinated pinch-lift-move sequences with 2–4 robots on lightweight objects (1–2 kg), though limited by arm torque and calibration issues.

5. Significance

This work represents a major step forward in multi-robot loco-manipulation. By eliminating the need for mechanical coupling and communication, the proposed method offers a more flexible and robust solution for real-world logistics, construction, and emergency response where objects are irregular and cannot be gripped. The Constellation Reward provides a powerful new tool for learning rigid-like coordination in contact-rich tasks, and the ability to train on small teams and scale to large ones significantly reduces the computational cost of developing multi-robot systems.