💻 computer science

CrazyMARL: Decentralized Direct Motor Control Policies for Cooperative Aerial Transport of Cable-Suspended Payloads

The paper introduces CrazyMARL, a decentralized reinforcement learning framework that enables multi-UAV teams to robustly transport cable-suspended payloads in unstructured environments by successfully handling complex nonlinear dynamics, cable slack-taut transitions, and external disturbances through zero-shot sim-to-real transfer.

Original authors: Viktor Lorentz, Khaled Wahba, Sayantan Auddy, Marc Toussaint, Wolfgang Hönig

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Viktor Lorentz, Khaled Wahba, Sayantan Auddy, Marc Toussaint, Wolfgang Hönig

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have two tiny, hummingbird-sized drones (called Crazyflies) that need to work together to carry a heavy, dangling weight suspended by a rope. This isn't just a simple lift; the weight swings like a pendulum, the rope can go slack and then snap tight, and the wind is trying to blow them apart.

This paper, CrazyMARL, describes how the researchers taught these tiny drones to do this job using a special kind of "team training" called Reinforcement Learning. Instead of programming them with rigid rules, they let the drones learn by trial and error in a super-fast video game simulation, eventually teaching them to fly so well that they can handle the real world without any human help.

Here is a breakdown of how they did it, using simple analogies:

1. The Challenge: The "Swinging Chandelier" Problem

Usually, when robots carry things, they assume the object is stuck to them like a backpack. But here, the payload is hanging on a cable.

The Problem: If the drones move too fast, the weight swings. If the cable goes slack, the weight drops; if it snaps tight, it jerks the drones. It's like trying to carry a chandelier on a bungee cord while someone is pushing you and the wind is howling.
The Old Way: Previous methods tried to solve this with complex math formulas (like rigid rods). They assumed the cable was stiff and unchanging. This worked okay in calm weather but failed miserably when things got messy or the cable went slack.

2. The Solution: The "Muscle Memory" Approach

The researchers didn't write a manual for the drones. Instead, they used Reinforcement Learning (RL).

The Analogy: Think of it like training a dog. You don't tell the dog, "Move your left leg 3 inches forward." You say, "Good boy!" when it does the right thing and "No" when it messes up. Over time, the dog learns the feeling of the right move.
The Training: The drones played a video game (simulation) millions of times. Every time they dropped the weight or crashed, they got a "bad score." Every time they kept the weight steady, they got a "good score."
The Speed: They used a powerful computer (GPU) to run 2 billion steps of training in just 70 minutes. This is like watching a lifetime of practice happen in an afternoon.

3. The Secret Sauce: "Direct Muscle Control"

Most robots have a "brain" that tells a "spine" what to do, and the "spine" tells the "muscles" (motors) what to do. This paper skips the spine.

The Analogy: Imagine a pianist. A normal robot is like someone reading sheet music, thinking about the note, and then telling their finger to move. CrazyMARL is like a jazz musician who feels the rhythm and their fingers just know where to go instantly.
The Result: The AI talks directly to the motors at 250 times per second. It sends raw electrical signals (PWM) directly to the motors. This allows the drones to react instantly to wind gusts or sudden pushes, operating right at the edge of their physical limits without crashing.

4. The "Magic Trick": Sim-to-Real Transfer

Usually, robots trained in a video game fail when they hit the real world because the game physics aren't perfect.

The Trick: The researchers used Domain Randomization. They didn't just train the drones in one perfect world. They trained them in thousands of slightly broken worlds:
- Sometimes the motors were weak.
- Sometimes the wind was crazy.
- Sometimes the cable was too long or too short.
- Sometimes the drones started on the ground or upside down.
The Outcome: Because they trained in such chaotic, varied conditions, the drones learned a "super-skill" that worked everywhere. When they were put on the real Crazyflie drones, they didn't need any retraining. They just worked. This is called Zero-Shot Transfer.

5. The Results: What They Actually Achieved

The paper claims specific, impressive results:

Disturbance Rejection: When the researchers physically pushed the drones hard or blew wind at them (3.5 m/s wind speed), the drones recovered 80% of the time. The old method only recovered 44% of the time.
Agility: The drones could take off from the ground, lift the weight, and fly in a figure-eight pattern, even with the weight swinging wildly.
Decentralized Teamwork: The two drones didn't need to talk to each other or have a central commander. Each drone had its own "brain" (policy) that looked at its own sensors and the position of its teammate, and they just knew how to coordinate.

Summary

In short, CrazyMARL is a system that teaches tiny drones to carry swinging loads by letting them practice in a chaotic, high-speed video game. By teaching them to control their motors directly and training them in every possible disaster scenario, the researchers created a team of drones that can fly together, handle strong winds, and recover from crashes better than any previous method, all without needing a human to steer them.

Technical Summary: CrazyMARL

Problem Statement

Collaborative transportation of cable-suspended payloads by teams of Unmanned Aerial Vehicles (UAVs) offers significant potential for increasing payload capacity and adapting to irregular loads. However, controlling such systems is challenging due to complex pendulum dynamics, tension coupling, and the transition between "slack" and "taut" cable modes. Existing control strategies face specific limitations:

Model-based approaches often rely on rigid-rod assumptions for cables, which fail to capture hybrid dynamics (slack-taut transitions) and limit agility. Centralized methods suffer from scalability bottlenecks and single points of failure, while decentralized model-based controllers may lack performance guarantees or struggle with iterative communication.
Reinforcement Learning (RL) approaches have shown promise in single-agent scenarios and some multi-agent tasks, but prior work in multi-UAV payload transport has largely relied on rigid-link assumptions or centralized training with decentralized execution (CTDE) architectures that require communication overhead.
Hardware Constraints: Most existing methods do not address the control of multiple robots operating on resource-constrained onboard microcontrollers (e.g., Crazyflie 2.1) near their actuation limits, where low thrust-to-weight ratios make operation close to motor saturation common.

Methodology

The authors propose CrazyMARL, a decentralized Multi-Agent Reinforcement Learning (MARL) framework designed for cooperative aerial transport with realistic cable dynamics.

1. Framework and Architecture

Algorithm: The system utilizes Independent Proximal Policy Optimization (IPPO) with parameter sharing. This follows a completely decentralized execution regime where each agent acts based on local observations without inter-robot communication, avoiding scalability bottlenecks.
Training Pipeline: Training is conducted end-to-end using JAX and MuJoCo MJX on GPUs, allowing for thousands of parallel environments. This high-throughput setup enables extensive domain randomization (DR) within a short timeframe (2 billion steps in 70 minutes).
Observation Space: Each agent $i$ $i$ receives a local observation $o^i_t$ $o_{t}^{i}$ comprising:
- Payload error ( $e^P_t$ ) and velocity ( $v^P_t$ ).
- Relative position ( $\delta^i_t$ ), orientation ( $vec(R^i_t)$ ), and velocities ( $v^i_t, \omega^i_t$ ) of the agent.
- The agent's previous action ( $a^i_{t-1}$ ).
- Relative positions of teammates ( $\{\delta^j_t\}_{j \neq i}$ ) for coordination.
Action Space: Agents output normalized motor commands directly mapped to Pulse Width Modulation (PWM) duty cycles at 250 Hz. The policy bypasses low-level cascaded controllers (e.g., PID or thrust mixing), mapping directly to motor actuation to operate effectively near physical limits.

2. Simulation and Dynamics

Hybrid Cable Model: Unlike prior works using rigid rods, the simulation employs Mujoco tendons to model cables that exert tension only when taut and zero force when slack. This captures the hybrid slack-taut dynamics essential for agile maneuvers.
Domain Randomization (DR): To ensure robust sim-to-real transfer, the training incorporates randomization of:
- Initial states (including ground starts and slack cables).
- Actuator parameters (thrust caps, time constants, motor offsets).
- Observation noise and stochastic external disturbances (wind, pushes).
- Target positions.

3. Reward Design

A modular, composite reward function $r = r_{track} \cdot r_{stable} + r_{safe}$ is used:

Tracking ( $r_{track}$ ): Minimizes payload error and aligns payload velocity with the target.
Stability ( $r_{stable}$ ): Penalizes excessive swing, tilt, and velocity, while encouraging taut cables.
Safety ( $r_{safe}$ ): Encourages smooth actions, balanced motor thrust, and collision avoidance.
The reward terms are bounded to [0, 1] to facilitate smooth gradients and prevent the agent from learning to terminate episodes prematurely.

Key Contributions

End-to-End Decentralized Control: The first demonstration of a fully decentralized MARL policy for multi-UAV cable-suspended transport that outputs direct motor PWM commands without low-level cascades, specifically tailored for resource-constrained platforms like the Crazyflie 2.1.
Hybrid Cable Dynamics: The framework explicitly models and learns to control transitions between slack and taut cable modes, a capability previously unaddressed in multi-UAV contexts which typically assumed rigid links.
Zero-Shot Sim-to-Real Transfer: Successful deployment of the learned policy on real hardware (Crazyflie 2.1) without fine-tuning, demonstrating robustness against wind, external pushes, and harsh initial conditions.
High-Throughput Training Pipeline: A GPU-parallelized JAX/MJX pipeline that trains complex multi-robot policies in minutes, significantly accelerating the iteration cycle compared to traditional methods.

Experimental Results

Simulation

Disturbance Rejection: In harsh recovery scenarios (randomized initial states, including ground starts and slack cables), the CrazyMARL policy achieved an 80% recovery rate (797/1000 trials), significantly outperforming a baseline model-based controller (44% success rate).
Agility: The learned policy maintained a mean speed of 0.58 m/s compared to 0.27 m/s for the baseline, with faster damping of payload swing and more direct trajectories.
Generalization: The policy generalized well to variations in cable length, payload mass, and observation noise, though performance degraded with very long cables (>1m) or extreme noise scales.
Scalability: The system successfully scaled from 1 to 3 quadrotors. While 6-quadrotor teams showed coordination limits (early termination), the framework demonstrated effective load sharing and recovery for smaller teams.

Hardware (Real-World)

Platform: Deployed on two Crazyflie 2.1 quadrotors carrying a 10g payload.
Performance: The system executed autonomous takeoffs and maintained stable flight under wind speeds averaging 3.5 m/s.
Precision: The policy maintained a position-holding Root Mean Square Error (RMSE) of 0.077 m in wind, a marginal increase over the 0.064 m RMSE in calm conditions.
Robustness: The robots successfully recovered from strong external pushes and maintained figure-eight trajectories with the payload, capabilities the baseline controller could not achieve.

Significance and Claims

The paper claims that CrazyMARL provides a scalable, resilient solution for cooperative aerial manipulation in unstructured environments. By combining decentralized execution with direct motor control and realistic hybrid cable dynamics, the work enables UAV teams to operate near their physical actuation limits while rejecting disturbances. The authors position this as a step toward autonomous teams capable of complex payload missions where traditional model-based controllers struggle due to modeling inaccuracies and communication overheads. The successful zero-shot transfer to hardware with low-thrust-to-weight ratio robots highlights the practical viability of high-frequency, end-to-end RL policies in real-world scenarios.