Impact of Dataset Composition on Embedded Real-Time UAV Wildfire Detection Using Compact YOLO Models

Technical Summary: Impact of Dataset Composition on Embedded Real-Time UAV Wildfire Detection Using Compact YOLO Models

Problem Statement
The development of vision-based wildfire detection systems for Unmanned Aerial Vehicles (UAVs) is currently constrained by the scarcity of diverse, real-world training images, particularly those captured from aerial viewpoints with suitable annotations. While deep learning models, specifically the YOLO family, offer a favorable trade-off between inference speed and detection quality for embedded edge devices, their practical deployment is often limited by data availability. Consequently, researchers frequently turn to synthetic data generation and image augmentation to expand training sets. However, the practical value of these synthetic expansions for embedded UAV wildfire detection under resource-constrained conditions remains an empirical question rather than an established fact.

Methodology
This study employs a dataset-centered approach to evaluate how training data composition affects detection performance, using compact YOLO models as a controlled validation family rather than proposing new detector architectures.

• System Platform: The experiments utilized a custom 3D-printed "Buzzard" UAV equipped with an NVIDIA Jetson Nano 4 GB for onboard processing and an RGB camera. The system supports VTOL (Vertical Take-Off and Landing) and fixed-wing forward flight, designed for endurance and low-power operation.
• Dataset Construction: The study assembled a pool of images from four sources: 447 images captured by the authors' UAV, 857 images from public datasets, 82 images from the Uruguayan Air Force remote sensing service, and 1,296 AI-generated synthetic images.
• Experimental Configurations: Four distinct dataset configurations were evaluated:
  1. Real non-augmented: 1,386 real images.
  2. Real augmented: The real set expanded to 8,316 images via geometric and photometric transformations (flips, rotations, brightness/contrast variation, blur).
  3. Hybrid non-augmented: 2,682 images (1,386 real + 1,296 synthetic).
  4. Hybrid augmented: 16,092 images (augmented real + augmented synthetic).
• Models and Training: Four compact YOLOv5 variants (YOLOv5n, YOLOv5n6, YOLOv5s, YOLOv5s6) were trained on each configuration for 1,000 epochs. The models were selected for their suitability for embedded deployment.
• Evaluation Metrics: Performance was measured using Precision, Recall, mAP@0.5, mAP@0.5:0.95, and Box Loss. The study prioritized Recall and mAP, as missing a wildfire event is considered more costly than generating a limited number of false alarms in an early warning system.

Key Results
The experimental results yielded a counter-intuitive finding regarding data expansion strategies:

• Superiority of Real Non-Augmented Data: The best overall operating point was achieved using the real non-augmented dataset. Specifically, YOLOv5n6 trained on this configuration achieved the highest Recall, highest mAP@0.5, and highest mAP@0.5:0.95.
• Limited Value of Augmentation and Hybridization: Neither data augmentation nor the inclusion of synthetic data (hybridization) produced a better final deployment choice than the real non-augmented baseline.
  • While the real augmented dataset remained competitive, it did not surpass the non-augmented real set.
  • Hybrid configurations (both augmented and non-augmented) resulted in fragmented performance patterns and generally lower Recall and mAP compared to the real-only baseline.
• Model Behavior: Different YOLO variants exhibited complementary strengths; YOLOv5n6 excelled in recall and mAP on real data, while YOLOv5s6 often achieved the lowest Box Loss and highest precision, particularly in augmented settings. However, the dominant trend across all models was that dataset composition (real vs. synthetic/augmented) had a more significant impact on performance than the specific model variant chosen.
• Real-World Validation: Real-world flight tests confirmed that the model trained on the real non-augmented dataset could effectively detect wildfire events during real-time operation, even under motion and viewpoint variations.

Significance and Claims
The paper claims that for embedded UAV wildfire detection, dataset realism and domain alignment are more valuable than increasing training set size through synthetic expansion.

The study concludes that while synthetic data and augmentation are valuable tools when real annotated imagery is scarce, their utility is contingent upon how well they preserve the visual characteristics of the target deployment domain. In the evaluated scenario, the visual differences between synthetic images and real aerial wildfire scenes (in texture and scene naturalness) negatively impacted the detector's ability to generalize. Therefore, the most effective strategy for practical deployment was to train on high-quality, real-world imagery without artificial expansion, prioritizing the fidelity of the training data to the operational environment over the sheer volume of data.

The authors position this work not as a proposal of a new detector, but as a controlled comparative analysis demonstrating that, under resource-constrained conditions, preserving the original appearance and domain characteristics of real imagery yields superior practical detection performance.