4th Workshop on Maritime Computer Vision (MaCVi): Challenge Overview

1. Problem Statement

The paper addresses the critical need for robust, real-time computer vision systems in maritime environments, which present unique challenges such as dynamic lighting, water reflections, cluttered scenes, and low-visibility conditions (night/fog). Autonomous Unmanned Surface Vehicles (USVs) require specialized algorithms to navigate, detect obstacles, and interpret surroundings.

The MaCVi 2026 workshop, held at CVPR 2026, focuses on five specific benchmark challenges designed to bridge the gap between high-accuracy research models and embedded real-time feasibility. The core problems tackled include:

• Vision-to-Chart Association: Matching visible buoys in RGB images to digital chart markers under geometric distortion.
• Thermal Object Detection: Detecting vessels and navigational objects in thermal infrared imagery (critical for night operations).
• Panoptic Segmentation: Parsing complex scenes into "stuff" (water, sky) and "thing" (boats, buoys) classes.
• Embedded Segmentation: Achieving high-quality semantic segmentation on resource-constrained hardware (Luxonis OAK4).
• Multimodal Segmentation: Fusing RGB, Thermal, and LiDAR data to maintain reliability when specific sensors degrade (e.g., night-time RGB failure).

2. Methodology and Challenge Tracks

The workshop utilized a standardized protocol involving public training/validation sets and private test sets. Submissions were evaluated on accuracy, robustness, and inference speed (FPS) on target hardware.

A. Vision-to-Chart Data Association

• Task: Associate detected buoys in a monocular RGB image with specific markers on a nautical chart.
• Top Method (1st Place - HD Korea Shipbuilding): A Skyline-aware ROI-calibrated pipeline. It estimates the vessel's roll via skyline detection, projects chart queries into image-space Regions of Interest (ROIs) using learned distance/bearing curves, and uses an RF-DETR detector with Hungarian assignment.
• Key Insight: Explicitly injecting geometric priors (skyline, IMU data) into the detection pipeline significantly outperforms standard fusion transformers.

B. Thermal Object Detection

• Task: Detect vessels and navigational objects in thermal imagery (704 training images).
• Top Method (1st Place - Schneider Electric Taiwan): A Multi-architecture Ensemble (6 models including Co-DINO, DDQDETR, DINO, RTMDet, RF-DETR) combined with Semi-Supervised Learning.
  • Technique: Used pseudo-labeling on the 381 unlabeled test images followed by MixPL (teacher-student) training.
  • Preprocessing: Applied CLAHE (Contrast Limited Adaptive Histogram Equalization) to enhance feature contrast.
• Key Insight: Ensembles of diverse architectures and semi-supervised learning on limited labeled data are crucial. Small object detection (<32px) remains the primary bottleneck.

C. LaRS Panoptic Segmentation

• Task: Panoptic segmentation of USV scenes (11 classes: water, sky, 8 dynamic obstacles).
• Top Method (1st Place - FER Zagreb): M2F-DINOv3. Utilizes Mask2Former with a DINOv3 (self-supervised) backbone.
• Key Insight: Large-scale self-supervised backbones improve robustness in challenging conditions (rain, fog, overexposure). However, detecting small "thing" instances (swimmers, small boats) remains difficult compared to "stuff" segmentation.

D. Embedded Semantic Segmentation

• Task: Real-time segmentation on the Luxonis OAK4 embedded device (target: >30 FPS, INT8 quantization).
• Top Method (1st Place - Independent Researcher): DSOS-Net.
  • Architecture: ConvNeXt backbone (pretrained with DINOv3) + lightweight RSOS-Net decoder.
  • Strategy: Two-stage training (high obstacle weight for recall, then balanced for precision).
• Key Insight: There is a strict trade-off between accuracy and throughput. Simple convolutional designs (PIDNet-S) quantize better than attention-heavy models, which suffer catastrophic performance drops under INT8 quantization.

E. Multimodal Semantic Segmentation (Multiaqua)

• Task: Segment static/dynamic obstacles using synchronized RGB, Thermal, and LiDAR, handling missing/degraded modalities.
• Top Method (1st Place - GIST AI LAB): GatedMemorySAM.
  • Architecture: Adapts SAM2 (Segment Anything Model) by treating modalities as video frames. Uses Soft MoE LoRA for parameter-efficient adaptation and a CrossModalFusionHead to score and gate modalities.
  • Strategy: Extensive Night Augmentation (brightness darkening, noise, cross-modal replacement) to bridge the day-night domain gap.
• Key Insight: Explicitly modeling visibility conditions and using memory-based fusion allows the model to rely on thermal/LiDAR when RGB fails at night.

3. Key Contributions

1. Comprehensive Benchmark Suite: Established five distinct tracks covering detection, segmentation, and multimodal fusion, specifically tailored to maritime constraints (embedded hardware, thermal imaging, chart association).
2. State-of-the-Art Results: Provided new baselines and top-performing methods that significantly outperform previous iterations, particularly in thermal detection (AP improved from ~0.31 to ~0.48) and panoptic segmentation.
3. Technical Reports: Detailed analysis of winning strategies, including:
   • The dominance of Ensemble Learning (WBF fusion) in detection tasks with limited data.
   • The efficacy of Self-Supervised Pretraining (DINOv3) for robustness.
   • The critical importance of Domain-Specific Adaptations (e.g., skyline filtering, annotation refinement, night simulation).
   • The necessity of Quantization-Aware Design for embedded deployment.
4. Generalist Meta Challenge: Introduced a new evaluation metric (Consistency Score) to reward methods that perform well across multiple tracks rather than overfitting to a single task (though no teams submitted eligible entries in this inaugural year).

4. Results Summary

Challenge	Top Method	Key Metric Improvement	Key Trend
Vision-to-Chart	Skyline-aware ROI-calibrated	Overall Score: 0.7646 (vs. 0.3333 baseline)	Geometric priors + DETR fusion.
Thermal Detection	Multi-arch Ensemble + SSL	AP: 0.4868 (vs. 0.3137 baseline)	Ensembles + Semi-supervised learning on small data.
Panoptic Seg.	M2F-DINOv3	PQ: 53.5 (vs. 41.4 baseline)	Self-supervised backbones improve "stuff" robustness.
Embedded Seg.	DSOS-Net	Q-Score: 61.9 @ 66.5 FPS	DINOv3 features + lightweight decoders.
Multimodal Seg.	GatedMemorySAM	M (Mean mIoU): 82.1	Memory-based fusion + Night simulation.

Note: All top methods significantly outperformed the provided organizer baselines.

5. Significance

The MaCVi 2026 workshop highlights a pivotal shift in maritime computer vision from pure accuracy optimization to holistic system design that balances performance with deployment constraints.

• Practical Deployment: The emphasis on embedded constraints (INT8, FPS) and thermal imaging ensures that research is directly applicable to real-world USVs operating in diverse conditions.
• Data Efficiency: The success of semi-supervised learning and pseudo-labeling in the Thermal challenge offers a blueprint for domains where annotated data is scarce.
• Robustness: The findings on geometric conditioning (Vision-to-Chart) and domain adaptation (Multimodal) demonstrate that physical priors and simulation are essential for handling the extreme variability of maritime environments.
• Future Direction: The paper suggests a move toward "generalist" models that can adapt across tasks and modalities, moving away from task-specific silos.

This report serves as a definitive reference for researchers and practitioners aiming to deploy autonomous maritime systems, providing both quantitative benchmarks and qualitative lessons learned from the state-of-the-art.