MOSAIC: Modular Scalable Autonomy for Intelligent Coordination of Heterogeneous Robotic Teams

The paper presents MOSAIC, a scalable autonomy framework that enables a single operator to supervise heterogeneous robotic teams for scientific exploration in hostile environments by dynamically allocating tasks based on Points of Interest, as demonstrated by a successful lunar prospecting field experiment where the team maintained high task completion and autonomy levels despite a robot failure.

The Gist

Imagine you are the director of a movie set in a dangerous, unknown alien landscape. You have a crew of five very different actors: two fast runners, one heavy-duty truck, and two agile climbers. Your goal is to explore the terrain, find interesting rocks, and take scientific measurements.

In the old days, you would have to be the "puppet master," holding a joystick for every single actor, telling them exactly where to step, when to stop, and what to measure. If one actor got stuck in the mud, the whole movie stops. This is how most robots work today: they need a human to hold their hand constantly.

MOSAIC is a new "smart director" system that changes the game. Instead of micromanaging every step, you give the crew a simple list of goals (called Points of Interest, or POIs) and let them figure out how to get there.

Here is how the MOSAIC system works, broken down into simple concepts:

1. The "To-Do List" (Points of Interest)

Instead of giving the robots complex commands like "move left 3 meters, then lift your leg," MOSAIC uses a universal language of Points of Interest (POIs).

• The Analogy: Think of POIs like sticky notes on a map. One note says "Explore this cave," another says "Take a picture of that rock," and a third says "Measure this patch of sand."
• How it works: The human operator places these sticky notes on a digital map. The robots then look at the list and say, "I can do that one," or "I'm better at that one." They automatically assign the tasks to themselves based on who is closest and who has the right tools.

2. The "Specialized Crew" (Heterogeneous Team)

The team isn't made of identical clones; they are specialists with different superpowers.

• The Scouts (The Runners): These are fast, agile robots (like the Boston Dynamics Spot and legged robots). They don't carry heavy science gear. Their job is to run ahead, map the area, find interesting rocks, and flag them on the map.
• The Scientists (The Trucks): These are slower, heavier robots (like the wheeled Husky and the legged Donkey). They carry heavy microscopes and lasers. They wait for the Scouts to say, "Hey, there's a cool rock over there!" and then they trudge over to take a close-up look.
• The Magic: If a Scout finds a rock, it doesn't need to stop and measure it. It just marks the spot. A Scientist robot then claims that job and goes to do the detailed work.

3. The "Three Levels of Control" (Autonomy)

The system is designed so you don't have to be a robot expert. You can switch between three modes of control, like a video game difficulty setting:

• Level 1: The Boss (Mission Level): You just say, "Go explore that hill." The robots figure out the path, avoid obstacles, and do the work. You are just watching the big picture.
• Level 2: The Coach (Task Level): If a robot looks confused, you can step in and say, "Don't go left, go right," or "Stop and take a picture." You guide the specific action, but the robot still drives itself.
• Level 3: The Remote Control (Driver Level): If a robot is truly stuck or in danger, you take full manual control, driving it like a remote-control car to get it out of trouble.

4. The "Resilient Team" (What Happened in the Test)

The researchers tested this system in a rocky quarry in Switzerland that looked like the Moon. They had five robots and one human operator.

• The Disaster: Halfway through the mission, one of the main Scout robots (named Dodo) got water damage and died. It stopped working completely.
• The Reaction: In an old system, the mission would have failed. But with MOSAIC, the system didn't panic. The other robots noticed the gap. The remaining Scout robot (Dilly) picked up the slack, and the human operator just added more "sticky notes" to the map manually.
• The Result: Even with one robot dead, the team completed 82% of their tasks. The human operator was only "busy" about 78% of the time (which is actually quite low for such a complex job), meaning the robots were doing most of the heavy lifting on their own.

5. The "Traffic Jam" Problem (Networking)

One of the biggest challenges was getting all these different robots to talk to each other without the internet getting clogged.

• The Analogy: Imagine five people trying to talk to a central office over a walkie-talkie. If everyone talks at once, it's noise. If they talk too reliably (waiting for an "acknowledgment" after every word), the conversation gets slow.
• The Fix: The team learned that for wireless connections, it's better to let some messages "drop" (like a radio signal fading) rather than waiting for a confirmation, which slows everything down. They also separated the "urgent" commands from the "video feed" data so the important stuff didn't get stuck in traffic.

The Big Takeaway

MOSAIC proves that you don't need a team of human operators to run a team of robots. You can have one human supervising five different robots in a dangerous environment.

If one robot breaks, the others keep going. If the human gets tired, the robots keep working. It turns a fragile, single-point-of-failure system into a resilient, self-healing team, much like a well-organized sports team where players cover for each other when someone gets injured.

This is a huge step toward future missions to the Moon or Mars, where communication delays make constant human control impossible, and robots must be smart enough to work together on their own.

Technical Summary

Here is a detailed technical summary of the paper "MOSAIC: Modular Scalable Autonomy for Intelligent Coordination of Heterogeneous Robotic Teams."

1. Problem Statement

Mobile robots are essential for exploring hostile environments (e.g., space, disaster zones), but current deployments are largely limited to teleoperation. This reliance on continuous human supervision creates significant bottlenecks:

• Scalability Issues: As team size increases, the number of required human operators grows, or robots experience idle time due to delayed inputs.
• Single Point of Failure: Traditional single-robot missions (like Mars rovers) are vulnerable; a critical failure ends the entire mission.
• Complexity vs. Robustness: Integrating diverse sensing and actuation into a single platform increases hardware/software complexity, whereas multi-robot teams offer redundancy and specialization but require complex coordination.
• Operator Workload: Managing heterogeneous teams with different autonomy levels and scientific tasks creates an unmanageable cognitive load for a single operator.

The core challenge is to enable scalable autonomy where a single operator can supervise a heterogeneous team of robots performing parallel scientific exploration and measurement tasks, even in the face of communication delays or individual robot failures.

2. Methodology: The MOSAIC Framework

The authors propose MOSAIC (Modular Scalable Autonomy for Intelligent Coordination of Heterogeneous Robotic Teams), a decentralized architecture built on ROS 2.

A. Core Concepts

1. Unified Mission Abstraction (POIs):
   • Mission objectives are abstracted as Points of Interest (POIs). A POI represents a goal (e.g., "explore area," "measure rock") independent of the specific robot executing it.
   • POIs are converted into robot-specific tasks based on the robot's capabilities (e.g., a "Rock Measurement" POI becomes a navigation + arm manipulation task for a quadruped, but a different sequence for a wheeled rover).
2. Three-Tier Autonomy Model:
   • Mission Level: The operator supervises the global mission, prioritizing POIs. Robots autonomously select and claim POIs.
   • Task Level: Robots execute assigned tasks (navigation, manipulation, data acquisition) autonomously. The operator can intervene to modify behavior or trigger specific actions.
   • Driver Level: Full teleoperation for fine-grained control or recovery, with assistance functions (e.g., collision avoidance).
3. Role-Based Team Composition:
   • Scouts: Fast, agile robots (e.g., quadrupeds) equipped with LiDAR and cameras for exploration and initial target detection.
   • Scientists: Robots with specialized payloads (spectrometers, microscopes, manipulators) for detailed in-situ analysis. They typically follow scouts to validated targets.
   • Redundancy: Each role is covered by multiple robots to ensure continuous operation if one fails.

B. System Architecture

• System Level (Centralized): A Mission Control Station and a Science Operations Station manage the global map, POI distribution, and operator interfaces. A central POI Handler manages the state of objectives.
• Robot Level (Decentralized): Each robot runs a local Behavior Tree for decision-making and a POI Planner.
  • Planning Algorithm: A greedy, multi-step allocation algorithm calculates the "utility" of unclaimed POIs based on distance, battery, and robot capabilities. It uses a two-step process: a fast Euclidean estimate to prune candidates, followed by a detailed path/battery calculation for the most promising targets.
• Communication: Uses ROS 2 Humble with FastDDS.
  • Implements a Domain Bridge to separate robot-specific communication from the shared team network.
  • Utilizes Quality of Service (QoS) tuning: "Best Effort" for high-frequency sensor data (to avoid latency from retransmissions) and "Reliable" for control commands.

C. Hardware Implementation

The system was tested with a heterogeneous team of five robots:

• Scouts:
  • Dodo & Dilly (ANYmal D): Quadrupeds with LiDAR, RGB-D cameras, and (for Dilly) thermal/zoom cameras. Dodo features "pedipulation" (using legs to dig/flip rocks).
  • Spot (Boston Dynamics): Quadruped with an arm for manipulation and rock flipping.
• Scientists:
  • Donkey (ANYmal D): Quadruped with a 6-DoF arm, Raman spectrometer, and microscope.
  • Husky (Clearpath A200): Wheeled rover with a UR5 arm, XRF spectrometer, and angle grinder for surface preparation.

3. Key Contributions

1. Scalable Autonomy Framework: Implementation of MOSAIC, enabling a single operator to manage a heterogeneous team via a unified POI abstraction and layered autonomy.
2. Unified Mission Representation: A system where mission goals are decoupled from execution, allowing dynamic task allocation based on robot capabilities.
3. Field Validation: Successful execution of a lunar prospecting analog mission in a real-world, unstructured environment (quarry) with challenging weather (snow/wet soil).
4. Practical Lessons Learned: A comprehensive analysis of interoperability challenges (ROS 1/2 bridging), networking (QoS tuning for Wi-Fi), and team composition strategies.

4. Results

The system was evaluated during a 40-minute field campaign involving five robots and one operator.

• Robustness: Despite the complete failure of the "Dodo" scout robot (due to water damage) and software issues with "Spot," the team completed 82.3% of assigned tasks.
• Autonomy Ratio: The system achieved an Autonomy Ratio of 86%, meaning the robots operated autonomously for the vast majority of the mission time.
• Operator Workload: The Quantitative Operator Workload was 78.2%. While high, this was largely due to the loss of an autonomous scout, forcing the operator to manually generate POIs.
• Efficiency:
  • Mapped 758 m² of terrain.
  • Achieved a mapping rate of 0.313 m²/s.
  • Task Success Ratio: 82.3% (including rock and ground measurements).
• Scientific Precision:
  • Successfully identified 56.2% of target resources (rocks and sand patches).
  • Obtained high-quality microscopic images and Raman spectra, though XRF data was compromised by wet soil conditions.
• Downtime: Total robot downtime was 52%, but analysis showed this was largely due to state-labeling errors in the behavior tree and initial technical issues, rather than a lack of autonomy capability.

5. Significance and Future Work

Significance:

• Proof of Concept: Demonstrates that a single operator can effectively manage a complex, heterogeneous multi-robot team in a real-world, unstructured environment.
• Resilience: Proves that team-based approaches with redundancy can sustain mission objectives even when critical hardware fails, a crucial requirement for deep-space exploration where repair is impossible.
• Interoperability Insights: Provides critical lessons on integrating ROS 1 and ROS 2, managing wireless QoS, and designing team compositions (recommending a 3:2 scout-to-scientist ratio).

Future Work:

• Decentralization: Moving toward fully decentralized task management to handle communication blackouts (mesh networking).
• Scalability: Optimizing the system to manage larger teams (~10 robots) with 2-3 operators.
• Standardization: Creating standardized interfaces for easier integration of new ROS 2-based robots.
• Operator Tools: Developing automated task queues to further reduce cognitive load.

In conclusion, MOSAIC represents a significant step toward practical, scalable multi-robot exploration, shifting the paradigm from teleoperation to supervised autonomy capable of handling the unpredictability of real-world scientific missions.