Technical Summary: Multimodal Detection for Underwater Cultural Heritage

Problem Statement

Reliable object detection is critical for the investigation, interpretation, and conservation of underwater cultural heritage (UCH), including shipwrecks, submerged settlements, and artifacts. However, the development of automated detection systems in this domain is hindered by three primary challenges:

1. Data Scarcity: There is a lack of domain-specific datasets with fine-grained annotations. Existing underwater benchmarks (e.g., URPC2020, RUOD) focus primarily on marine ecology (organisms, debris) rather than cultural heritage structures (pottery, hulls, anchors), which differ significantly in visual appearance and spatial distribution.
2. Imaging Degradation: Underwater environments introduce severe optical degradation, including light attenuation, scattering, and non-uniform color shifts (often blue-green casts). These factors obscure object boundaries, degrade texture, and distort colors, rendering standard RGB-based detectors vulnerable.
3. Complex Scene Composition: Archaeological sites often feature long-tailed class distributions, severe occlusion, and targets of vastly different scales (from large hull structures to small coins), complicating multi-scale detection.

Methodology: UCH-Net and the WRF Dataset

To address these challenges, the authors propose UCH-Net, a physics-guided, color-bias-aware multimodal detection network, and introduce the WRF (Wrecks, Ruins, and Fragmentary heritage) dataset.

1. The WRF Dataset

The authors constructed a dedicated multimodal dataset comprising 4,732 RGB images and corresponding depth maps collected from diverse deep-sea archaeological sites in the Gulf of Mexico, the Black Sea, and the Mediterranean Sea.

• Annotations: The dataset contains 17,059 annotated instances across 21 categories, including wooden components, pottery jars, sculptures, coins, and glass bottles.
• Depth Generation: Since original images lacked ground-truth depth, depth maps were generated using Depth Anything V2, ensuring spatial alignment with RGB images to serve as geometric priors.
• Characteristics: The dataset exhibits a long-tailed distribution and includes challenging conditions such as partial occlusion, truncation, and varying illumination.

2. UCH-Net Architecture

UCH-Net is a dual-stream detector designed to integrate appearance (RGB) and geometric (Depth) cues through a physics-guided fusion mechanism. It consists of four core components:

• Dual-Stream Backbone:

  • RGB Branch: Extracts color and texture features using C3k2 modules (improved CSPNet), SPPF for receptive field expansion, and C2PSA attention to suppress background noise (e.g., suspended particles).
  • Depth Branch: Processes depth maps in parallel to extract geometric contours and spatial structural information, operating independently to avoid premature semantic misalignment.

• Dual-Feature Fusion Pyramid (DFFP):

  • Adapted from PANet, this module facilitates cross-scale feature interaction.
  • It introduces cross-layer skip connections allowing original multimodal features to participate directly in fusion, mitigating information loss from deep convolutional smoothing.
  • It employs multi-path adaptive weighted fusion to dynamically adjust contributions from different feature pathways based on underwater optical conditions.

• Shared Decoupled Detection Head (SDDH):

  • Designed for resource efficiency, this head uses a shared depthwise separable convolution across three scales (P3, P4, P5) for classification and regression.
  • This design reduces parameter count while encouraging consistent target representations across scales, preventing overfitting to specific underwater noise patterns.

• Color-Bias-Aware Fusion Module (CBFM):

  • This is the core innovation for handling optical degradation. CBFM estimates local color bias, regional brightness, and global degradation context using early RGB reference features.
  • It generates bounded scale and bias parameters to adaptively modulate current-layer RGB features.
  • The modulated RGB features are fused with depth-derived geometric features via spatial, channel-wise, and pixel-level gating. This allows the network to rely on geometric structure in regions where color and texture are severely degraded.

Key Contributions

1. WRF Dataset: A new multimodal benchmark for underwater cultural heritage detection, providing 4,732 images with 17,059 instances across 21 categories, addressing the scarcity of heritage-specific data.
2. UCH-Net Architecture: A novel dual-stream framework that integrates RGB and depth modalities through a physics-guided color-bias-aware fusion mechanism, specifically tailored to compensate for underwater optical degradation.
3. Performance Gains: The proposed method demonstrates superior performance in both detection accuracy and localization precision compared to state-of-the-art general and underwater-specific detectors.

Experimental Results

The authors evaluated UCH-Net on the WRF dataset and the public RUOD dataset, comparing it against 16 representative methods (including DETR variants, YOLO series, and two-stage detectors).

Performance on WRF Dataset

• Overall Accuracy: UCH-Net achieved an mAP of 77.5% and mAP75 of 88.0%.
• Comparison: It outperformed the representative Deformable-DETR baseline by 12.8% in mAP and 14.9% in mAP75.
• State-of-the-Art: It exceeded the strongest competing results by 1.2% in mAP75 and 1.5% in recall.
• Efficiency: With 48.8M parameters and 143.8 GFLOPs, UCH-Net has fewer parameters than several top competitors (e.g., BSR5-YOLO, U-DECN) but incurs higher computational costs than lightweight YOLO variants.

Performance on RUOD Dataset

• UCH-Net achieved an mAP of 66.0% and mAP75 of 72.7%, securing the highest scores in Precision, F1-score, mAP50, mAP75, and mAP among evaluated methods.
• It demonstrated a 1.5 percentage point advantage in mAP75 over the second-best method (BSR5-YOLO), highlighting its strength in strict localization.

Ablation Studies

• Modality: Adding the depth branch (DSB) improved mAP by 1.0% (RUOD) and 0.8% (WRF).
• Fusion: The CBFM module provided significant gains, particularly on the WRF dataset (+2.5% mAP), confirming the utility of adaptive appearance modulation in complex archaeological scenes.
• Components: The complete CBFM (combining Physical Prior Estimation, Channel, Spatial, and Positional Attention) yielded the best results, validating the complementary nature of these attention mechanisms.

Qualitative Analysis

Grad-CAM++ visualizations indicate that UCH-Net generates more concentrated high-response regions around artifacts compared to general-purpose detectors, which often exhibit dispersed attention or false positives in background areas. UCH-Net showed robustness in detecting overlapping coins, partially buried artifacts, and small distant objects under suspended particles.

Significance and Claims

The paper claims that the combination of a dedicated multimodal dataset and a physics-guided detection network offers a viable technical basis for automated screening and object localization in underwater archaeology.

• Localization Precision: The authors emphasize that the improvement in mAP75 is particularly significant for cultural heritage applications. Accurate delineation of object extent is crucial for candidate-object cropping, site inventory construction, and repeated-survey comparisons, extending the value of the method beyond simple presence detection.
• Robustness to Degradation: By explicitly modeling color bias and integrating geometric priors, UCH-Net addresses the specific physical limitations of underwater imaging (scattering and attenuation) that standard detectors fail to handle.
• Practical Application: The framework is positioned as a tool to support the efficiency and consistency of underwater archaeological image analysis, potentially aiding in the digital archive management and monitoring of heritage sites by ROVs, AUVs, or divers.

Limitations Acknowledged:
The authors note that depth maps are estimated rather than measured, which may introduce errors in highly scattering or low-texture environments. Additionally, the current dataset split is temporally ordered rather than site-independent, and the computational cost (143.8 GFLOPs) may pose challenges for real-time deployment on resource-constrained underwater platforms. Future work is suggested to incorporate measured depth data and model lightweighting.