Technical Summary: SynDroneVision-Weather (SDV-W)

1. Problem Statement

Reliable drone detection in real-world deployment requires training data that spans the full Operational Design Domain (ODD), including adverse weather and seasonal variations. However, acquiring and annotating such data is resource-intensive and difficult to control systematically, particularly for adverse conditions like rain, snow, and fog. Existing real-world datasets often lack coverage of these scenarios, while current synthetic solutions are frequently limited to clear-sky conditions or rely on physically inconsistent image-space augmentations (e.g., superimposing rain streaks) that fail to capture complex interactions between weather, illumination, and scene geometry. Consequently, detectors often suffer performance degradation when encountering distribution shifts caused by seasonality or adverse atmospheric conditions.

2. Methodology

The authors introduce SynDroneVision-Weather (SDV-W), a systematic extension of the existing SynDroneVision (SDV) dataset. SDV-W is generated using a game-engine-based pipeline (Unreal Engine 5.0 and the Colosseum simulator) to ensure pixel-precise annotations and controlled environmental variation.

Data Generation

• Base Configuration: SDV-W re-renders 15 curated sequences from the original SDV dataset across three distinct urban environments (University Site, Venetian City, Urban Downtown).
• Controlled Variables: The generation preserves the original scene geometry, camera placement, and drone trajectories. This ensures a strict one-to-one correspondence between "clean" and "adverse" condition frames, isolating weather and seasonal effects from scene-content variations.
• Seasonal Configurations: Three seasonal states are implemented: Spring/Summer (full green foliage), Autumn (yellow-brown foliage), and Winter (defoliated vegetation).
• Weather Conditions: Weather is rendered directly within the simulation using the Colosseum weather API. Conditions include:
  • Precipitation: Rain and snow (with snow restricted to winter).
  • Atmosphere: Fog and falling leaves (restricted to autumn).
  • Combinations: A compound rain-fog condition parameterized asymmetrically to approximate realistic lighting and contrast reduction.
  • Severity: Conditions are rendered at "light" and "heavy" severity levels, in addition to sunny baselines.
• Dataset Composition: The final dataset comprises 55,187 annotated high-resolution images (2560×1477 pixels) distributed across the three seasons and various weather conditions. It includes a mix of object sizes, with a systematic shift toward smaller projected object extents compared to the original SDV.

3. Key Contributions

1. SDV-W Dataset: A large-scale, annotated dataset extending SDV with controlled seasonal and adverse-weather variations, enabling matched clean-adverse comparisons.
2. Controlled Robustness Analysis: A methodology to quantify condition-specific detector degradation by evaluating models on paired synthetic conditions without explicit weather exposure during training.
3. Cross-Domain Generalization Quantification: An evaluation of SDV-W's contribution to detector performance across diverse architectures (YOLO variants and YOLO-FEDER FusionNet) and real-world datasets covering seasonal and weather shifts.
4. Disentanglement of Factors: An analysis separating the effects of environmental diversity from training-set scale and characterizing detector behavior across different object sizes.

4. Experimental Results

The study evaluated standard YOLO models (v5, v8, v9, v11) and the specialized YOLO-FEDER FusionNet on public real-world datasets (DUT Anti-UAV, LRDDv1, LRDDv2) and custom seasonal recordings.

Synthetic Weather Impact

• Degradation: When tested on SDV-W without weather-specific training, all adverse conditions reduced performance relative to clear weather.
• Severity: Snow and fog caused the most significant degradation, increasing the False Negative Rate (FNR) by ~15.8% and ~12.7% respectively, and reducing mAP@0.25 by ~8.2% and ~6.2%.
• Failure Modes: Adverse weather primarily suppressed object saliency below the detection threshold (increasing missed detections) rather than inducing false alarms, with the exception of falling leaves which increased false positives.

Training Impact (SDV-W as a Resource)

• Performance Gains: Integrating SDV-W into the training set (alongside SDV and real data) improved detector reliability under adverse appearance shifts.
  • On LRDDv1, mAP@0.5 increased by +10.4 to +14.1 percentage points across YOLO variants.
  • On DUT Anti-UAV (where baselines were near saturation), SDV-W reduced FNR by an average of 4.4 percentage points without compromising reference-domain performance.
  • On LRDDv2, SDV-W consistently reduced FNR and False Discovery Rate (FDR), with the most significant gains observed for medium-sized objects.
• Specialized Models: YOLO-FEDER FusionNet showed substantial improvements on custom seasonal recordings (R1-POS3, R2-POS7), particularly in reducing FDR in summer and FNR in winter.
• Diversity vs. Scale: Replacing general-purpose SDV samples with SDV-W (keeping total size constant) did not yield consistent benefits on large, diverse public datasets. This suggests SDV-W is most effective as a complement to general synthetic data rather than a replacement.

Small Object Recovery

• Tiled Inference: To address the compression of object scales caused by resizing to 640×640, the authors tested tiled inference (splitting images into overlapping patches).
• Trade-offs: Tiled inference significantly improved detection for extra-small objects (e.g., doubling mAP@0.5 on LRDDv2) but degraded localization for large objects and increased inference time (from ~12ms to >50ms).

5. Significance and Claims

The paper claims that SDV-W provides a scalable, controlled means to improve the operational robustness of drone detectors against seasonal and weather-induced appearance shifts. Its primary significance lies in:

• Reducing Operational Failures: It effectively reduces missed detections and false alarms in adverse conditions, which are critical for real-world deployment.
• Complementary Utility: The dataset is most valuable when used to complement general-purpose synthetic data, broadening environmental coverage without sacrificing cross-domain generalization.
• Methodological Control: By preserving scene geometry and trajectories, SDV-W enables the isolation of weather-specific effects, a level of control rarely achievable in real-world recordings.

The authors maintain modesty regarding their claims, noting that while SDV-W mitigates appearance-driven degradation, scale-induced failures still require complementary inference strategies (like tiling) which introduce their own trade-offs. Furthermore, the lack of diverse real-world adverse-weather data for final evaluation is acknowledged as a limitation to be addressed in future work.