💻 computer science

Decentralized Multi-Robot Obstacle Detection and Tracking in a Maritime Scenario

This paper presents a decentralized multi-robot framework that integrates UAVs and an autonomous surface vessel to robustly detect and track floating containers in maritime environments by combining YOLOv8-based vision with stereo disparity, Covariance Intersection fusion, and information-driven task allocation, thereby achieving improved tracking accuracy and identity continuity despite visual artifacts and bandwidth constraints.

Original authors: Muhammad Farhan Ahmed, Vincent Frémont

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Muhammad Farhan Ahmed, Vincent Frémont

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy, choppy ocean where a fleet of floating shipping containers has been lost. To find them, we can't just send one boat; we need a team. This paper describes a smart, decentralized team made of one large autonomous boat (the "Captain") and several drones (the "Scouts").

Here is how this system works, explained through simple analogies:

1. The Problem: The "Foggy Mirror" Ocean

The ocean is a terrible place for cameras. The water acts like a shiny, wobbly mirror. Sunlight glares, waves distort the view, and the horizon often disappears.

The Challenge: If a drone just looks at a container, it might mistake a wave reflection for a boat, or lose track of a container when it dips behind a wave.
The Bandwidth Issue: These robots can't talk to each other constantly over long distances because the connection is weak and slow. They can't send huge video files; they can only send tiny text messages.

2. The Solution: A Team of "Smart Scouts"

Instead of one big brain controlling everything (which would get overwhelmed), every robot thinks for itself but shares its best guesses with the team.

Step A: Seeing with "3D Glasses"

The drones use a special trick to see depth. They combine a standard color camera with a stereo camera (like human eyes).

The Analogy: Imagine trying to find a specific red ball in a pile of red balls. It's hard. But if you put on 3D glasses, you can see exactly how far away each ball is.
The Tech: The drones use an AI (YOLOv8) that has been "fed" both color images and depth maps. This helps them ignore the shiny water reflections and lock onto the actual metal containers.

Step B: The "Trust but Verify" Tracker

Once a drone spots a container, it starts tracking it. But because the ocean is messy, the drone isn't 100% sure of the location.

The Analogy: Think of the drone as a detective keeping a file on a suspect. The file has a "confidence score." If the suspect hides behind a building (a wave), the detective's confidence drops, but they don't delete the file immediately. They keep watching.
The Tech: They use a math tool called an EKF (Extended Kalman Filter). It's like a smart calculator that predicts where the container should be based on physics, then updates that prediction when the drone sees it again.

Step C: The "Secret Handshake" (Fusion)

This is the most clever part. The drones fly around and see the same container from different angles. They need to combine their notes without double-counting or getting confused.

The Problem: If Drone A and Drone B both see Container C, they might be looking at the same wave reflection. If they just average their notes, they might become too confident and make a mistake.
The Solution (Covariance Intersection): Imagine two people trying to guess the temperature. One says "It's 20°C," the other says "It's 22°C." Instead of just averaging them to 21°C, they use a special rule that says, "We don't know if our thermometers are influenced by the same wind, so let's be conservative and assume the temperature is somewhere between 19°C and 23°C."
The Result: The team creates a "master list" of containers that is safe and consistent, even if the drones are talking over each other or missing parts of the picture.

Step D: The "Smart Dispatcher"

Now that the team has a list of containers, who goes where?

The Analogy: Imagine a ride-share app, but instead of picking up passengers, the drones are assigned to "hover" over the most confusing containers.
The Strategy: The system asks: "Which container is the most confusing (has the highest uncertainty)?" and "Which drone is closest and safest?"
The Hover: Once assigned, the drone doesn't just fly past; it flies to a specific spot in a circle around the container and hovers. This gives it a perfect, steady view to update the map and reduce the "uncertainty" of that container's location.

3. The Result: A Smooth Operation

The researchers tested this in a computer simulation that looked like a real ocean.

What happened: The drones flew out, spotted the containers, ignored the fake reflections, and worked together to track them perfectly.
The Win: They didn't lose track of any containers (no "identity switches"). They knew exactly where the containers were, even with bad camera views. And because they only sent tiny summaries of their data, they didn't clog up the communication channel.

Summary

Think of this system as a team of detectives solving a mystery in a foggy room.

They wear 3D glasses to see through the fog.
They keep private notebooks to track suspects.
They whisper tiny, safe summaries to each other so they don't get confused by shared mistakes.
They send the closest detective to stand right next to the most confusing suspect to get a better look.

This allows a small team of robots to monitor a vast, messy ocean efficiently without needing a super-fast internet connection.

1. Problem Statement

The paper addresses the challenges of deploying autonomous aerial–surface robot teams for maritime monitoring, specifically for detecting and tracking floating containers. The core difficulties in this domain include:

Visual Artifacts: Water reflections, waves, low texture, and glare degrade standard computer vision performance.
Communication Constraints: Long operating ranges and intermittent line-of-sight limit bandwidth, making centralized data fusion difficult.
Estimation Consistency: In decentralized systems, shared priors and repeated communications create unknown cross-correlations between robot estimates. Naive fusion can lead to overconfident (inconsistent) covariance estimates.
Coordination: Efficiently assigning multiple UAVs to multiple targets while balancing travel effort, safety separation, and information gain is complex.

2. Methodology

The authors propose a decentralized multi-robot framework implemented in ROS, consisting of multiple Unmanned Aerial Vehicles (UAVs) and one Autonomous Surface Vessel (Aquabot). The system operates through the following pipeline:

A. Perception and Detection

Sensor Fusion: UAVs utilize downward-facing RGB stereo cameras.
Detection Model: The authors modified YOLOv8 to accept a 4-channel input (RGB + Rectified Disparity) rather than standard RGB. This "YOLOv8-RGBDis" model is fine-tuned on maritime container classes.
3D Reconstruction: For each detection, a robust disparity value is estimated (median of valid pixels within the bounding box). Using the pinhole stereo model, 2D detections are converted into metric 3D coordinates relative to the camera frame.

B. Local Tracking

EKF Tracking: Each UAV maintains an independent Extended Kalman Filter (EKF) for every detected target.
State Model: Targets are modeled with a constant-velocity state ($position, velocity$) in the world frame.
Data Association: Uses Mahalanobis gating to associate new 3D measurements with existing tracks. Unmatched detections initialize new tracks; unmatched tracks are removed after a timeout or if uncertainty exceeds a threshold.

C. Decentralized Fusion

Covariance Intersection (CI): To fuse track summaries from different robots without knowing the cross-correlations, the system uses CI. This ensures the fused estimate remains conservative (consistent) even if estimates are correlated.
Process: Robots exchange only compact 3D position marginals ( $\hat{p}, P$ ). The Aquabot acts as a fusion node, applying CI sequentially to combine estimates from all UAVs.

D. Task Allocation and Viewpoint Selection

Min-Cost Flow (CMCF): Targets are assigned to UAVs using a capacitated min-cost flow formulation. The cost function balances:
- Information Gain: Expected reduction in target uncertainty (based on D-optimality).
- Travel Effort: Distance to the target.
- Constraints: Safety separation between UAVs and "stickiness" (reducing assignment chattering).
Hover Viewpoint Selection: Once assigned, a UAV selects a specific hover pose on a discrete ring around the target. The optimal pose maximizes information gain per unit of travel distance while ensuring collision avoidance.

E. Mode Management

The system operates in two modes:

SURVEILLANCE: UAVs follow patrol paths when not assigned.
TRACKING: UAVs switch to tracking mode upon assignment, moving to the selected hover viewpoint to refine the target's state.

Termination: Tracking stops when the target's uncertainty drops below a threshold or when no further information gain is possible.

3. Key Contributions

Decentralized Pipeline: A complete system integrating stereo-based 3D perception, EKF tracking, and conservative fusion via Covariance Intersection, specifically designed for maritime environments.
Robust Perception: A modified YOLOv8 model that fuses RGB and disparity data, improving detection reliability in low-contrast, reflective water conditions.
Information-Driven Allocation: A novel assignment strategy that prioritizes targets based on expected uncertainty reduction while enforcing safety and workload constraints.
Viewpoint Optimization: A strategy for selecting hover viewpoints that maximizes observability and tracking accuracy.

4. Experimental Results

The system was validated in a Gazebo simulation (ROS 2, Ubuntu 24.04) involving 3 UAVs and 5 floating containers over a 9-minute mission.

Detection Performance:
- The YOLOv8-RGBDis model outperformed standard YOLOv8, achieving higher mAP@0.5 (0.995 vs. 0.985) and mAP@0.5:0.95 (0.970 vs. 0.952), demonstrating better localization reliability.
- Localization errors remained stable (median 2.2–4.7 m) despite viewpoint changes.
Tracking Performance:
- Compared to baselines SORT and AB3DMOT, the proposed method achieved perfect identity continuity (IDF1 = 1.000) with zero identity switches (IDSW = 0).
- It significantly reduced track fragmentation (5 fragments vs. 142 for SORT).
- 3D localization accuracy was comparable to AB3DMOT (MedErr ~3.35m) but with superior stability.
Fusion Performance:
- The proposed CI fusion outperformed a Fault-Tolerant CI (FTCI) baseline in terms of accuracy (lower MedErr and RMSE) and outlier reduction, while maintaining conservative covariance estimates.
Allocation Efficiency:
- The allocator successfully partitioned targets among UAVs with minimal overlap.
- The system effectively reduced target uncertainty (log det P) during tracking phases, confirming the efficacy of the hover-viewpoint selection strategy.

5. Significance and Conclusion

This work demonstrates that decentralized multi-robot systems can effectively operate in challenging maritime environments where communication is limited and visual conditions are poor.

Robustness: By combining stereo disparity with deep learning and conservative fusion, the system mitigates the risks of overconfident estimates and false positives caused by water reflections.
Scalability: The use of Covariance Intersection and compact track summaries allows the system to scale without requiring high-bandwidth centralized processing.
Practicality: The integration of task allocation with uncertainty-aware viewpoint selection ensures that robot resources are used efficiently to maximize monitoring quality.

The authors conclude that the framework provides a scalable solution for maritime inspection and search-and-rescue, with future work planned for real-world trials under dynamic sea conditions and intermittent connectivity.