YOLO-RCD: A Lightweight Pavement Damage Detector Validated by Controlled Multi-Seed Reproduction and Measured Edge Deployment

Technical Summary: YOLO-RCD: A Lightweight Pavement Damage Detector Validated by Controlled Multi-Seed Reproduction and Measured Edge Deployment

Problem Statement
While YOLO-family detectors are widely adopted for automated pavement damage detection, the literature suffers from three critical evaluation gaps that hinder reliable comparison and deployment:

1. Lack of Multi-Seed Statistical Validation: Most studies report single-run results without random seed control, significance testing, or confidence intervals. Given the small size of pavement validation sets (e.g., 216 images in RDD2022 China Drone) and the modest claimed improvements (1–4 percentage points), single-run results often conflate architectural gains with random initialization variance.
2. Absence of Cross-Viewpoint Generalization: Models are typically trained and evaluated on a single sensor viewpoint (e.g., aerial drone). Real-world deployment often requires transferring models to fundamentally different viewpoints (e.g., ground-level motorcycle cameras) without retraining, a scenario rarely quantified under strict zero-shot protocols.
3. Uncharacterized Edge Quantization Sensitivity: Post-training INT8 quantization is standard for edge deployment, but its accuracy loss is often reported as a fixed property of the architecture based on a single training run. The sensitivity of quantization loss to random seeds and hardware generations remains largely unmeasured in this domain.

Methodology
The authors propose YOLO-RCD, a lightweight detector based on YOLOv11n, and evaluate it through a rigorous, controlled protocol on the RDD2022 China Drone dataset (source) and China MotorBike dataset (target).

• Model Architectures: Two variants are introduced:
  • YOLO-RCD-Star: Replaces standard C3k2 backbone blocks with C3k2 Star (utilizing element-wise multiplication of parallel projections) and modifies the backbone by swapping SPPF for SPPELAN, removing C2PSA, and inserting a BasicRFB block.
  • YOLO-RCD-MPDIoU: Shares the backbone modifications of the Star variant but retains standard C3k2 blocks. It replaces the standard CIoU loss with a strict MPDIoU formulation, where the corner-distance penalty is normalized by a fixed image-space scale ($640^2 + 640^2$) rather than a pair-dependent convex hull diagonal, aiming to stabilize training for small, elongated cracks.
• Controlled Training Protocol:
  • Seeds: Core models are trained across seven random seeds ({42, 123, 456, 789, 1024, 2024, 3141}). Competitor models are reproduced using a matched three-seed subset.
  • Environment: All models use identical hyperparameters (300 epochs, SGD, specific learning rate schedule), a pinned software stack (PyTorch 2.0, Ultralytics 8.3.172), and are trained from scratch (no COCO pre-training).
  • Statistical Analysis: Comparisons utilize Welch's t-tests, Cohen's d effect sizes, 95% confidence intervals, and Benjamini–Hochberg false-discovery-rate (FDR) correction.
• Evaluation Protocols:
  • In-Domain: RDD2022 China Drone validation set.
  • Zero-Shot Cross-Viewpoint: Direct evaluation on RDD2022 China MotorBike without fine-tuning.
  • Test-Time Augmentation (TTA): Applied at inference to assess training-free domain generalization recovery.
  • Edge Deployment: Models are exported to TensorRT 10.3 (FP32, FP16, INT8) and evaluated on Jetson AGX Orin and NVIDIA V100 to measure latency, power, and INT8 sensitivity across seeds and hardware generations.

Key Contributions

1. Multi-Seed Reproduction Protocol: The study establishes a controlled benchmark where four concurrent competitor designs (including YOLO-ROC, Star-YOLO11, YOLO11-MBC, and YOLOv11n-ATL) are retrained under identical conditions.
2. Zero-Shot Cross-Viewpoint Benchmark: It quantifies the "aerial-to-ground" domain gap under a strict no-adaptation protocol, establishing a baseline for future domain generalization research.
3. Seed-Dependent Quantization Characterization: It demonstrates that INT8 quantization loss is highly sensitive to random seeds and that this sensitivity transfers across GPU generations, while the magnitude of spread is device-dependent.

Results

• In-Domain Performance: Under the three-seed competitor comparison, the proposed variants achieve 74.4–74.6% mAP@0.5, outperforming the YOLOv11n baseline (70.18%) by +4.3 to +4.4 percentage points (pp). In contrast, the best reproduced competitor (Star-YOLO11) exceeds the baseline by only +0.52 pp, and none of the reproduced designs recover their originally published margins.
  • Note: Under the full seven-seed protocol, the in-domain means are 70.95% (Star) and 71.64% (MPDIoU) vs. 69.90% (Baseline). The differences are not statistically significant after FDR correction, suggesting that single-run claims of large gains may be artifacts of seed selection.
• Cross-Viewpoint Generalization: All models suffer a massive accuracy drop (48–51 pp) when moving from aerial to ground views, retaining only 28–32% of their in-domain accuracy.
  • YOLO-RCD-MPDIoU outperforms YOLO-RCD-Star by 2.30 pp in the zero-shot setting (Cohen's $d=1.22$), though this remains "suggestive" rather than confirmatory after FDR correction.
  • Test-Time Augmentation (TTA) recovers 3.8–4.5 pp across all models without retraining. This is the only comparison in the study that remains statistically significant after FDR correction, establishing TTA as a robust, training-free baseline for cross-viewpoint detection.
• Edge Deployment & Quantization:
  • On Jetson AGX Orin, the detectors sustain 434–484 FPS with an energy cost of 13–15 mJ/image and an inference power increment of ~1.1 W over idle.
  • INT8 Sensitivity: A single-seed pilot (seed 42) suggested an 11.47 pp advantage for the Star variant over the baseline. However, the seven-seed evaluation reduced this to a non-significant 2.58 pp.
  • Hardware Transfer: Per-seed INT8 loss patterns correlate significantly across V100 and Orin ($r=0.59$), but the spread (variance) of the loss is device-dependent. For instance, the baseline had the widest spread on V100 but the narrowest on Orin.

Significance and Claims
The paper argues that the current pavement detection literature relies on single-run evaluations that cannot distinguish architectural improvements from random seed variance. By introducing a multi-seed, controlled reproduction protocol, the authors demonstrate that:

1. Reported gains are often non-transferable: Competitor models that claimed large improvements in their original papers failed to reproduce those margins under a controlled protocol.
2. Cross-viewpoint gaps are severe and architectural changes alone are insufficient: Standard architectural tweaks do not significantly bridge the aerial-to-ground gap; training-free TTA is currently the most effective mitigation.
3. Deployment decisions require multi-seed validation: Selecting an architecture based on a single seed (e.g., for INT8 performance) can lead to erroneous conclusions. The authors recommend YOLO-RCD-MPDIoU for deployment due to its higher zero-shot mean, smallest parameter count (2.19 M), and largest TTA gain, despite its wider INT8 spread on specific hardware.

The study concludes by proposing a minimum reporting standard for the field: multi-seed evaluation (3–5 seeds), effect size reporting, full hyperparameter disclosure, and software stack pinning to ensure reproducibility.