Operational Evaluation of Submanifold Sparse Convolutional Networks for Airborne

Technical Summary: Operational Evaluation of Submanifold Sparse Convolutional Networks for Airborne LiDAR Bathymetry Processing

Problem Statement
Airborne LiDAR Bathymetry (ALB) is a critical technology for mapping coastal zones and the land-sea interface, supporting safe navigation and environmental monitoring. While ALB systems significantly reduce data acquisition costs compared to traditional acoustic methods, the resulting point clouds require extensive manual processing to distinguish the seabed from the water surface and noise. Automating this classification process remains a key operational objective for hydrographic offices. However, the development of automated methods is hindered by a lack of large-scale reference datasets and evaluation frameworks that align with nautical charting standards. Existing studies often rely on spatially limited datasets or metrics that do not directly reflect the nautical impact of misclassifications. Furthermore, while deep learning architectures like Submanifold Sparse Convolutional Networks (SSCN) have shown promise in airborne laser scanning, they have not been systematically evaluated for ALB point cloud processing.

Methodology
The study introduces a standardized evaluation framework to assess SSCN performance on large-scale ALB datasets, utilizing a NOAA survey dataset from the Florida Keys (1,599 tiles, totaling 747 million Georeferenced LiDAR Measurements or GLMs).

• Model Architecture: The proposed model is a U-Net-based SSCN designed to handle sparse volumetric data efficiently. It employs an encoder-decoder structure with residual blocks and residual-connected attention mechanisms to preserve sparse structures while capturing multi-scale features. The input is restricted to voxel density to minimize preprocessing complexity and avoid sensor-specific features.
• Preprocessing: GLMs are discretized into a voxel grid (1m horizontal, 25cm vertical resolution) in accordance with IHO S-44 Order 1a specifications. A bounded logarithmic normalization is applied to handle significant voxel density variability.
• Training Strategy: The model is trained using an asymmetric Unified Focal Loss (LaUF) to address class imbalance. Deep supervision is integrated at each decoder stage to improve optimization stability. Data augmentation includes random 90-degree rotations and flipping to mitigate directional biases.
• Evaluation Protocol: The study employs Repeated Random Subsampling Validation (RRSV) to assess model robustness and sensitivity to training set size. The evaluation combines point-level segmentation metrics (Global Accuracy, True/False Positive/Negative Rates, Producer's/User's Accuracy) with Digital Bathymetry Model (DBM) quality indicators. These DBM metrics include Grid-Based Producer's/User's Accuracy, Mean Absolute Error (MAE), and Error Exceedance Ratio (EER) based on IHO Order 1a Total Vertical Uncertainty (TVU) tolerances.

Key Contributions

1. Operational Evaluation of SSCN: The paper provides the first rigorous assessment of SSCN for ALB processing under real-world survey conditions, demonstrating its ability to deliver consistent results with sparse and noisy data.
2. Unified Benchmarking Framework: A standardized evaluation scheme is introduced that integrates segmentation accuracy with DBM quality. This ensures compliance with nautical charting standards and enables fair comparisons between automated methods.
3. Robustness Validation via RRSV: The study validates the stability and reliability of SSCN through RRSV, confirming its suitability for operational deployment across diverse survey scenarios and varying training data compositions.

Results
The study evaluated the model's performance across different training dataset sizes (5 to 40 tiles) and on the full survey (1,599 tiles).

• Training Efficiency: The model achieves high performance with limited training data. When trained on just 2.5% of the dataset (40 tiles), it delivers a median Producer's Accuracy (PAB) of 99.7% and a median Global Accuracy (GA) of 97.2%.
• DBM Quality: The resulting Digital Bathymetry Models show high fidelity. The Error Exceedance Ratio (EER) is extremely low, with only 4 in 10,000 DBM cells exceeding the IHO Order 1a tolerance. The Grid-Based User's Accuracy (GB-UAB) reaches 97.2%.
• Scalability and Speed: The SSCN approach demonstrates significant operational efficiency. Processing the entire Key West survey (1,599 tiles) took approximately 3 hours and 16 minutes, a substantial improvement over the estimated 65 days required by the statistical CHRT-ML method for the same dataset.
• Limitations Observed: The model showed sensitivity to density variations caused by overlapping flight paths, leading to artifacts (holes) in the DBM. Additionally, the model struggled with out-of-distribution features, such as power lines, which were underrepresented in the training data.

Significance
The paper claims that this work establishes a robust basis for the adoption of automated ALB processing in hydrographic workflows. By connecting segmentation results directly to DBM quality and IHO standards, the study bridges the gap between research metrics and operational requirements. The results confirm that SSCN is a reliable, scalable, and computationally efficient solution for high-precision bathymetry data processing, supporting the transition from manual to automated workflows in nautical charting. The study emphasizes that while the method is robust, its operational deployment requires careful curation of diverse training data to handle specific geomorphic and anthropogenic features.