A geometric and deep learning reproducible pipeline for monitoring floating anthropogenic debris in urban rivers using in situ cameras

Technical Summary: A Geometric and Deep Learning Reproducible Pipeline for Monitoring Floating Anthropogenic Debris in Urban Rivers

Problem Statement
The proliferation of floating anthropogenic debris in rivers poses significant threats to biodiversity, water quality, and human activities. While direct monitoring methods (e.g., GPS trackers, net sampling) exist, they lack the capacity for continuous, catchment-scale monitoring. Recent approaches utilizing deep learning (DL) and in-situ cameras have shown promise but suffer from three primary limitations: (1) a lack of rigorous investigation into dataset biases, specifically regarding negative images and temporal data leakage; (2) a reliance on vertical-view cameras (e.g., bridge-mounted) or UAVs, leaving bank-mounted, oblique-view configurations underexplored despite their operational advantages for long-term deployment; and (3) a limitation to detection outputs without physically interpretable measurements, such as object dimensions derived from monocular imagery.

Methodology
The study proposes a comprehensive framework integrating dataset construction, deep learning detection, and geometric size estimation.

• Data Acquisition and Curation:

  • Study Site: The Steingiessen River, a tributary of the Ill River near Strasbourg, France.
  • Hardware: Two pan-tilt-zoom (PTZ) cameras mounted on riverbanks (upstream and downstream).
  • Dataset: 6,013 images containing 10,873 anthropogenic debris, 754 natural debris, and 1,010 non-debris materials (e.g., kayaks).
  • Negative Images: 14,030 images of empty river scenes were acquired under varying solar conditions (sunny and cloudy) to test the hypothesis that background-only data reduces false positives.
  • Leak-Free Construction: To prevent temporal data leakage (where similar frames from the same video sequence appear in both training and test sets), the authors employed a clustering strategy. Visual features were extracted using a pre-trained YOLOv8 encoder, combined with bounding box metadata and timestamps. These vectors were reduced to 2D using t-SNE and clustered via DBSCAN. Clusters were then split into training, validation, and test sets (80/10/10), ensuring no temporal overlap between sets.

• Deep Learning Detection:

  • Models: Six variants of YOLO architectures (v5, v8, v11; nano and medium sizes) were evaluated.
  • Training Scenarios: Models were trained on: (1) randomly split datasets (baseline); (2) leak-free datasets; (3) leak-free datasets augmented with varying proportions (0–40%) of negative images; and (4) datasets with class weighting to address imbalance.
  • Hyperparameters: Training utilized transfer learning, data augmentation (tilting, mosaic, color jitter), early stopping, and L2 regularization.

• Geometric Size Estimation:

  • Pipeline: A monocular geometric pipeline estimates object dimensions (width and height) from 2D bounding boxes.
  • Mechanism: Using the pinhole camera model, the system reconstructs rays from the camera's optical center through the bounding box corners. These rays intersect with a reference ground plane (assuming a flat water surface) to calculate metric dimensions.
  • Correction: Two regression-based corrections were applied to mitigate systematic biases: (1) Bounding Box Shape Correction (correcting the AI-predicted box dimensions against manually annotated boxes) and (2) Predicted Dimensions Correction (correcting the geometric model's output against manually measured ground truth). Both linear and polynomial regressions were tested.

Key Results

• Impact of Data Leakage: Random splitting of video-derived data artificially inflated performance metrics. When using leak-free datasets, mAP@50 dropped significantly (e.g., YOLOv8-n from 0.939 to 0.488; YOLOv11-m from 0.962 to 0.536), revealing that previous benchmarks may have been overestimated by over 40 percentage points.
• Negative Images: An optimal proportion of 20% negative images was identified. This proportion maximized the trade-off between recall and precision, reducing false positives without causing the "dilution effect" seen at higher proportions (30–40%), which degraded recall.
• Class Weighting: While class weighting improved the detection of underrepresented natural debris (increasing true positive rates), it increased false positives and slightly reduced global metrics (mAP), indicating a precision-recall trade-off rather than a universal improvement.
• Model Performance: Under leak-free conditions, YOLOv11-m consistently outperformed YOLOv8-n. The best-performing configuration (YOLOv11-m with 20% negative images) achieved an mAP@50 of 0.648 and a precision of 0.704.
• Size Estimation Accuracy: The geometric pipeline, combined with regression corrections, achieved a Root Mean Square Error (RMSE) of 1.45 cm for height and 2.03 cm for width. These errors approach the theoretical resolution limit of the system (approx. 1.3 cm per pixel), suggesting the method extracts nearly all available spatial information.
• Transferability: A preliminary qualitative test on the Bruche River (a different site) demonstrated the model's ability to detect debris in a new environment, though quantitative validation across sites remains future work.

Significance and Contributions
The paper claims three main contributions:

1. Reproducible Framework for Bias Investigation: It establishes a methodology for constructing leak-free datasets from video streams using clustering, demonstrating that dataset constitution (leakage control, negative image proportion, and class weighting) has a greater impact on reported performance than architectural differences between models.
2. Oblique-View Detection Evaluation: It evaluates object detection under bank-mounted, oblique-view conditions, which are more representative of real-world, low-cost monitoring deployments than vertical bridge-mounted setups.
3. Interpretable Geometric Pipeline: It demonstrates the feasibility of estimating actual object dimensions from monocular RGB imagery using projective geometry and regression corrections, achieving centimeter-level accuracy sufficient for downstream mass estimation.

The authors conclude that this approach paves the way for robust, low-cost, automated monitoring systems for urban aquatic environments, provided that future work addresses the need for diverse datasets (including rainfall events) and validation across multiple sites.