A Probabilistic Framework for Improving Dense Object Detection in Underwater Image Data via Annealing-Based Data Augmentation

1. Problem Statement

Object detection models, particularly those based on Convolutional Neural Networks (CNNs) like YOLO, perform well in controlled environments (e.g., fish farms, tanks) but degrade significantly in real-world underwater settings. The primary challenges identified are:

• Environmental Variability: Fluctuating lighting, water clarity, and viewpoints.
• Dense Crowding: Natural habitats often feature high-density fish populations (schools) with frequent occlusions, a scenario underrepresented in standard training datasets.
• Data Scarcity: Existing datasets (like DeepFish) often contain images with very few fish per frame (mode of 1), failing to teach models how to handle complex, crowded scenes.
• Domain Shift: Models trained on specific species (e.g., Australian fish in DeepFish) struggle to generalize to different species and densities found in other regions (e.g., Florida Keys).

2. Methodology

The authors propose a novel pipeline involving dataset construction, a custom data augmentation strategy, and model training.

A. Dataset Preparation

• Source: The DeepFish dataset was used as the primary training source.
• Annotation Conversion: Since DeepFish provided segmentation masks but not bounding boxes, the authors used OpenCV's connected-components function to generate bounding box annotations from the masks, converting them into the YOLOv10 format.
• Test Set: A new, challenging test set was created by manually annotating 50 images from a live underwater stream in the Florida Keys, featuring dense, diverse fish populations (including sharks and barracuda) not present in the training data.

B. Proposed Algorithm: Pseudo-Simulated Annealing Data Augmentation (PSADA)

Inspired by Deng et al.'s "Copy-Paste" strategy for human detection, the authors developed a modified PSADA algorithm to synthesize realistic crowded fish scenarios. Key technical modifications include:

1. Group Center Selection:
   • Instead of sampling centers from existing objects (which is impossible in sparse images), group centers are randomly sampled from the image dimensions.
   • The number of groups ($|C|$) is sampled from a Poisson distribution ($\lambda=3$) rather than a uniform range. This models the probabilistic nature of fish schools appearing in a camera's field of view.
2. Object Generation & Placement:
   • For each center, a group of fish is generated. The number of objects per group is scaled to ensure balanced distribution.
   • Simulated Annealing Integration: Unlike standard copy-paste, the placement of fish relative to the group center is optimized using a simulated annealing approach.
     • Temperature Decay: A temperature parameter $T$ decays geometrically ($T \leftarrow T \cdot 0.95$) over iterations.
     • Acceptance Probability: Initially, fish can be placed far from the center (high probability of "far" placement). As $T$ decreases, the algorithm favors placements closer to the center, simulating the tightening of a school.
   • Size Distribution: Object sizes follow a Gaussian distribution with a fixed standard deviation ($\sigma = 30$ pixels) to simulate natural size variations within a school.
3. Augmentation Process:
   • Fish are "cut" from the original segmentation masks and pasted into base images with color augmentation (e.g., yellow-green shifts) to increase robustness against lighting changes.

C. Model Architecture

• Baseline: A standard YOLOv10 model trained on the original DeepFish bounding boxes.
• PSADA Model: A YOLOv10 model trained on the DeepFish dataset augmented with the PSADA algorithm, including 500 additional synthetic images.

3. Key Contributions

1. Novel Augmentation Framework: Introduction of PSADA, a probabilistic data augmentation method that adapts simulated annealing principles to generate realistic, dense fish schools from sparse training data.
2. Dataset Engineering: Successful conversion of the DeepFish segmentation dataset into a detection-ready format and the creation of a rigorous, manually annotated test set from live Florida Keys footage.
3. Addressing the "Sparse-to-Dense" Gap: The method specifically targets the limitation of training data lacking crowded scenes, enabling the model to generalize to high-density environments without requiring massive new real-world data collection.

4. Experimental Results

The models were evaluated on the Florida Keys live-stream dataset.

• Training Efficiency:
  • The PSADA model converged faster, reaching minimum box loss (1.5) in 28 epochs compared to the baseline's 50 epochs.
  • PSADA showed immediate loss reduction, whereas the baseline required 10 epochs before losses began to decrease.
• Performance Metrics:
  • mAP50: PSADA achieved 0.8, significantly outperforming the baseline.
  • mAP50-95: PSADA achieved 0.7.
  • Detection Count: On average, the PSADA model detected more than double the number of fish compared to the baseline.
  • Ground Truth Comparison: The baseline detected less than 25% of the ground truth fish count, while PSADA detected nearly 50%.
• Qualitative Analysis:
  • IoU Distribution: PSADA showed a distribution skewed toward higher Intersection over Union (IoU) scores, indicating more accurate bounding box localization.
  • Crowded Scenes: The baseline failed to detect fish in dense schools, often merging them or missing them entirely. PSADA successfully identified individual fish within crowded clusters, though it still struggled in areas of extreme maximal density.

5. Significance and Limitations

Significance:

• The study demonstrates that probabilistic data augmentation can effectively bridge the gap between sparse training data and dense real-world inference scenarios.
• It offers a viable solution for ecological monitoring, allowing researchers to use existing datasets to train models for complex, natural habitats without expensive new data collection.

Limitations & Future Work:

• Domain Shift: The model still missed ~50% of fish in the Florida test set, partly due to species differences (Australian vs. Florida species like sharks/barracuda) and extreme density variations not fully captured by the augmentation.
• Compute Constraints: Training was limited by Google Colab GPU availability; more intensive training could yield further improvements.
• Future Directions: The authors suggest adding a classification head to identify specific species within crowded schools and further fine-tuning the augmentation parameters to handle "maximal crowding" scenarios.

Conclusion

The paper successfully validates that a Pseudo-Simulated Annealing Data Augmentation (PSADA) strategy significantly enhances the robustness of YOLOv10 for underwater object detection. By synthetically generating crowded scenarios and optimizing fish placement via annealing, the model achieves superior generalization in dense, natural environments compared to standard baselines.