UAV-Based Drifting Victim Detection in Flash Floods Using Region-Based Convolutional Neural Networks

Technical Summary: UAV-Based Drifting Victim Detection in Flash Floods Using Region-Based Convolutional Neural Networks

Problem Statement
Flash floods present a critical challenge for Search and Rescue (SAR) operations due to their rapid onset, high-velocity flows, and the difficulty ground teams face in accessing affected areas, particularly when infrastructure is damaged. Conventional ground surveys are often too slow or dangerous, while satellite imagery suffers from limitations in resolution, revisit time, and cloud cover. Although Unmanned Aerial Vehicles (UAVs) offer high-resolution, flexible aerial surveillance, the sheer volume of visual data generated makes manual inspection impractical for rapid emergency response. Furthermore, detecting drifting victims in dynamic river environments is complex; targets are often small, partially obscured, and visually similar to floating debris, requiring more than simple rectangular bounding boxes to accurately delineate spatial boundaries and shapes.

Methodology
The study proposes a deep learning-based system utilizing UAV-acquired aerial video data to detect and segment drifting objects (potential victims) using the Mask R-CNN algorithm. The methodology follows a four-stage pipeline:

1. Data Collection: A customized dataset was created using two sources:

   • Primary Data: 26 video files captured via a DJI Mavic Air 2S drone at the Telocor Marine Tourism area (Porong River, Indonesia) at various altitudes. These were extracted into 589 image frames.
   • Secondary Data: 463 images sourced from social media.
   • The total dataset was split into training, validation, and testing sets. For the primary dataset (589 images), 515 images were used for training, 48 for validation, and 25 for testing. For the secondary dataset (463 images), the split was 278 for training, 93 for validation, and 92 for testing. Annotations were performed using labels such as "Severe," "Moderate," and "Mild" to denote flood status.

2. Data Processing: The raw images underwent a rigorous preprocessing pipeline including:

   • Quality Checking: Removal of corrupted, low-resolution, duplicate, or irrelevant images.
   • Annotation: Labeling objects with bounding boxes.
   • Resizing: Standardizing input dimensions (e.g., 224×224 or 256×256 pixels) for the CNN.
   • Normalization: Scaling pixel values from 0–255 to a 0–1 range using Min-Max Normalization.
   • Augmentation: Increasing dataset volume through horizontal flipping, rotation, scaling, cropping, and adjustments to brightness, contrast, blur, and Gaussian noise.

3. Model Development: The core of the system is a fine-tuned Mask R-CNN model. The architecture includes:

   • Backbone: A CNN for feature extraction.
   • Feature Pyramid Network (FPN): To generate multi-scale feature maps ($P1$–$P6$) capable of detecting objects at various scales.
   • Region Proposal Network (RPN): To identify candidate regions of interest (RoI).
   • RoIAlign: To extract fixed-dimensional feature representations from the proposed regions.
   • Prediction Head: To perform classification, bounding box regression, and pixel-level instance segmentation.
     The study compares a standard Mask R-CNN (trained on the original dataset) against a fine-tuned version trained on the customized, augmented dataset.

4. Evaluation: Performance was assessed using Average Precision (AP), Average Recall (AR), detection confidence, and inference time. Metrics were evaluated for both bounding box detection and instance segmentation across different Intersection over Union (IoU) thresholds.

Key Contributions
The paper outlines three primary contributions:

1. Customized Dataset: The development of a localized UAV-based image dataset representing drifting-victim conditions in flood-affected river environments, addressing the lack of domain-specific training data.
2. Fine-Tuned Detection Framework: The adaptation and fine-tuning of the Mask R-CNN model specifically for aerial flood imagery. This leverages instance segmentation to provide pixel-level delineation of targets, offering superior spatial information compared to standard bounding-box-only approaches.
3. Rapid Visual Assessment Tool: A demonstration of the system's potential to support SAR operations by providing timely spatial information on victim locations and movement directions in environments where ground access is restricted.

Results
Experimental results indicate that fine-tuning the Mask R-CNN model on the customized dataset significantly outperformed the conventional model:

• Detection Confidence: The fine-tuned model achieved an average detection confidence exceeding 90%, compared to the standard model's stabilization around 85–89%.
• Average Precision (AP):
  • Bounding Box: AP at IoU 0.5 increased from 54.6% (standard) to 94.94% (fine-tuned).
  • Segmentation: AP at IoU 0.5 increased from 52.8% (standard) to 96.94% (fine-tuned).
• Average Recall (AR): Significant improvements were observed in recall, particularly for segmentation at IoU 0.5, rising from 52.8% to 92.9% with the fine-tuned model.
• Efficiency: The fine-tuned model reduced average inference time from 0.5 seconds to 0.3 seconds per image.
• Overall Metrics: The study reports a detection accuracy increase from 87% to 92%, an F1-score of 0.88, and a mean Average Precision (mAP) of 0.82.

Significance and Claims
The authors claim that the proposed approach provides reliable object detection and spatial information for estimating drift trajectories in dynamic river environments. By utilizing instance segmentation, the system can distinguish between victims and visually similar debris more effectively than methods relying solely on bounding boxes. The study positions this technology as a viable tool for rapid aerial reconnaissance, capable of supporting decision-making during emergency responses in flash flood conditions characterized by high water velocity and complex terrain. The authors maintain a modest scope, noting that while the system is effective, some detection errors persist in low-light scenes and during rapid camera movement, and future work should focus on validating performance in diverse real-world SAR scenarios.