A drone-based framework for coral habitat mapping via weakly supervised segmentation

Technical Summary: A Drone-Based Framework for Coral Habitat Mapping via Weakly Supervised Segmentation

Problem Statement
Deploying machine learning for large-scale ecological monitoring is hindered by the bottleneck of obtaining pixel-level annotations, which are labor-intensive and costly to produce for extensive reef areas. While Unmanned Aerial Vehicles (UAVs) offer scalable aerial coverage, their imagery often lacks the resolution to distinguish fine-scale coral morphotypes compared to underwater observations. Conversely, Autonomous Surface Vehicles (ASVs) provide high-resolution underwater data but cover limited spatial extents. Existing Weakly Supervised Semantic Segmentation (WSSS) methods typically operate within a single modality or spatial scale, failing to address the challenge of transferring knowledge across distinct modalities (underwater to aerial) and aligning sparse, point-wise predictions with high-resolution orthophotos.

Methodology
The authors propose a multi-scale WSSS framework that bridges fine-scale underwater observations with large-scale aerial mapping. The workflow consists of six key steps:

1. Underwater Data Acquisition & Classification: High-density geolocated images are collected via ASV transects (spaced 0.5–1 meter apart). A deep learning "teacher" model (DinoVDeau) classifies coral morphotypes in these images, generating point-wise probability predictions.
2. Interpolation & Rasterization: The point-wise predictions are linearly interpolated to create continuous probability rasters. To handle class imbalance and varying confidence levels, a class-specific quantile normalization is applied to the probability maps before assigning the final class label to each pixel (argmax of normalized probabilities).
3. Aerial Data Acquisition: High-resolution UAV orthophotos are generated using Structure from Motion (SfM) and refined with Ground Control Points (GCPs) for precise georeferencing.
4. Dataset Construction: The interpolated ASV rasters are aligned with UAV orthophotos and upsampled to create weak segmentation masks. The dataset is tiled (512×512 pixels), with a focus on retaining classes distinguishable in aerial imagery (e.g., Acropora Branching, Acropora Tabular, Non-acropora Massive, Other Corals, and Sand).
5. Student Model Training: A transformer-based semantic segmentation model (SegFormer) is trained on the UAV imagery using the weak ASV-derived masks. The authors employ a region-based Dice loss function to mitigate the effects of label noise and class imbalance.
6. Mask Refinement & Self-Distillation: The initial model predictions are refined using SAMRefiner (adapting the Segment Anything Model for mask refinement). These refined masks serve as pseudo-labels to retrain the SegFormer model (Self-Knowledge Distillation), further improving spatial accuracy without new manual annotations.

Key Contributions

• Cross-Modal WSSS: The framework successfully transfers fine-scale information from ASV imagery to UAV segmentation models, effectively bridging the gap between underwater classification and aerial segmentation.
• Annotation Efficiency: The method eliminates the need for pixel-level manual labeling of aerial images, reducing the annotation burden to the simpler task of classifying underwater images.
• Refinement Pipeline: The integration of SAMRefiner and self-distillation significantly enhances the spatial precision of the segmentation masks derived from coarse weak labels.
• Extensibility: The approach demonstrates flexibility in incorporating new classes (e.g., sea cucumbers) via limited manual annotation or additional ASV sessions, and can adapt to different underwater classification architectures.

Results
The framework was evaluated on two distinct reef zones in Réunion Island (Trou d'eau and Saint-Leu lagoons) using manually annotated ground truth for validation.

• Performance: The final model (SegFormer B2 with full refinement pipeline) achieved 86.07% pixel accuracy and 52.23% mean Intersection over Union (mIoU) on the test zones.
• Component Analysis: Ablation studies confirmed that the full pipeline (Coarse + Refined + Distilled) outperformed variants using only coarse labels or single-step refinement, offering the best trade-off between accuracy and spatial precision.
• Class Performance: The model performed robustly on dominant classes like Sand (95.5% recall) and Acropora Branching. Performance was lower for visually complex or rare classes (e.g., Other Corals mIoU ~31.6%), attributed to intra-class variability and spectral ambiguity in aerial views.
• Case Studies:
  • Cyclone Impact: The model successfully quantified coral loss following Cyclone Garance, identifying an 89.70% loss of Non-acropora Massive colonies in a specific zone by comparing pre- and post-cyclone orthophotos.
  • New Class Integration: The framework successfully integrated a "Sea Cucumber" class using only 338 manually annotated instances, demonstrating the ability to learn from sparse new data while maintaining reasonable performance on existing classes.

Significance and Claims
The paper claims that this framework provides an efficient, scalable solution for deploying segmentation models in ecological settings where pixel-level annotations are unavailable. By leveraging weak supervision from dense underwater surveys, the method enables large-area segmentation of coral morphotypes with high-resolution detail. The authors position this as a practical tool for conservation monitoring, capable of detecting ecological changes (such as those caused by cyclones) and supporting biodiversity assessment without the prohibitive costs of manual annotation. The study emphasizes that while the current implementation focuses on specific classes, the methodology is generic and can be extended to other benthic classes or scaled up to satellite imagery, offering a pathway for hierarchical remote sensing in marine ecology.