💻 computer science

Drone-Based Early Forest Fire Detection Using Vision-Language Models on NVIDIA Jetson Orin Nano

This paper presents an edge-AI framework deployed on an NVIDIA Jetson Orin Nano-equipped UAV that combines a fine-tuned YOLOv8s detector with a CLIP-based vision-language model to achieve real-time, high-accuracy forest fire detection with natural language scene descriptions, significantly reducing false positives while maintaining low latency.

Original authors: Mohamed Amine Jabli, Abdellaoui Mehrez, Douik Ali

Published 2026-10-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mohamed Amine Jabli, Abdellaoui Mehrez, Douik Ali

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Wildfires are a relentless force of nature, growing faster and more destructive as the climate warms. For decades, the race to stop them has relied on spotting the first wisp of smoke or the first flicker of flame before the fire spreads out of control. Traditional methods, like satellites or fixed cameras, often arrive too late or miss the fire entirely because of clouds, terrain, or the sheer vastness of the landscape. To bridge this gap, scientists have turned to drones, sending them into the sky to act as mobile eyes. However, a drone is only as good as the brain it carries. If the drone's computer is too slow, it cannot process images fast enough to spot a fire in real time. If it is too simple, it mistakes a sunset reflection or a patch of dry grass for a blaze, causing false alarms that waste precious resources. The challenge lies in building a system that is both smart enough to understand the difference between a fire and a trick of the light, and fast enough to run on a battery-powered drone without overheating.

This is the problem a team of researchers from the University of Sousse in Tunisia set out to solve. They developed a new way for drones to detect forest fires early, combining two different types of artificial intelligence into a single, efficient system. The first part of their system acts like a rapid scanner, using a specialized image detector to quickly find anything that looks like fire or smoke. The second part acts as a critical thinker, using a vision-language model—a type of AI that can "read" an image and describe it in words—to double-check the scanner's findings. By running this two-step process on a compact, powerful computer called the NVIDIA Jetson Orin Nano, the researchers created a drone system that can patrol forests, spot a fire, and confirm it is real, all while flying autonomously and sending an alert within seconds.

The researchers built a custom drone equipped with a high-definition camera and this onboard computer. They trained their system using thousands of images of forests, fires, and smoke, teaching it to recognize the specific shapes and colors of a blaze. When the drone flies, the first AI model scans every frame of video it sees. If it spots something that might be fire, it passes that specific image to the second model. This second model does not just say "yes" or "no"; it looks at the image and generates a short sentence describing what it sees, such as "dense smoke rising from the trees" or "sunset glare on a lake." This extra step allows the system to rule out confusing situations, like bright sunlight hitting a reflective surface or autumn leaves that look orange like flames. The system is designed to be selective, only calling on the second, more complex model when the first one is unsure, which keeps the drone's computer from getting overwhelmed.

In their tests, the system proved to be remarkably effective. The drone successfully identified fires and smoke with an accuracy of nearly 95 percent, a significant improvement over systems that rely on a single detector. More importantly, the addition of the second, thinking model reduced false alarms by about 21 percent. This means the drone is far less likely to send a panic alert for something that isn't a fire, which is crucial for maintaining the trust of the human operators who manage the response. The system achieved this while running at a speed of 26 frames per second, fast enough to keep up with the drone's movement and provide a live, real-time view of the forest floor. The entire process, from seeing the image to sending the alert, took less than a tenth of a second, ensuring that the window to stop a fire before it grows out of control remains open.

The researchers also tested how the system performed under difficult conditions, such as when the sun was setting or when wind was blowing the smoke in unpredictable directions. While the system remained highly accurate in most scenarios, it did struggle slightly when the smoke was very thin and spread out by the wind, or when the lighting was dim. These are the moments where the difference between a real fire and a shadow is hardest to see. To address this, the team noted that adding a thermal camera, which sees heat rather than light, could help the drone spot fires even in the darkest or most confusing weather. For now, however, the system represents a major step forward in autonomous surveillance. It demonstrates that a drone can carry a sophisticated, two-layered intelligence on a small battery-powered platform, turning a simple flying camera into a reliable early-warning guardian for our forests.

Technical Summary: Drone-Based Early Forest Fire Detection Using Vision-Language Models on NVIDIA Jetson Orin Nano

Problem Statement
Wildfire incidence has risen significantly, with the critical window for effective suppression narrowing to the first 10–15 minutes after ignition. Conventional detection methods—satellite monitoring, fixed cameras, and ground sensors—suffer from inherent latency, coverage gaps, or resolution constraints that delay human awareness. While Unmanned Aerial Vehicles (UAVs) offer high-resolution, on-demand coverage, most existing onboard systems rely solely on standard object detectors (e.g., YOLO variants). These detectors often produce false positives when distinguishing fire from visually similar non-fire elements like sun glare, seasonal foliage changes, or reflective water. Furthermore, standard detectors lack the contextual reasoning to provide natural-language scene descriptions, limiting situational awareness for operators. The challenge is to develop an edge-AI framework that combines high-precision detection with contextual verification and natural-language reporting, all while operating within the strict power, thermal, and latency constraints of a battery-powered UAV.

Methodology
The authors propose a two-stage conditional inference pipeline deployed on a custom-built S500 quadrotor equipped with an NVIDIA Jetson Orin Nano (8 GB) and a Sony IMX219 RGB camera.

Stage 1: Detection (YOLOv8s): A fine-tuned YOLOv8s detector processes every frame to identify candidate fire and smoke regions. The model was trained on a composite dataset of 21,040 images spanning Mediterranean, temperate, and boreal biomes, utilizing augmentation techniques like Mosaic and CutMix to handle small targets and class imbalance.
Stage 2: Verification (CLIP-based VLM): Only frames where the detector confidence exceeds a threshold ( $\tau_{det} = 0.45$ $τ_{d e t} = 0.45$ ) are forwarded to a Vision-Language Model (VLM) based on OpenCLIP (ViT-B/32). This stage performs semantic verification using a library of 24 handcrafted text prompts categorized into:
- Affirmative: Descriptions of active fires.
- Confuser-Negative: Descriptions of non-fire lookalikes (e.g., "sunset glare," "misty forest").
- Ambiguous: Borderline cases for human review.
  The VLM computes cosine similarity between the image embedding and prompt embeddings. If the "confuser-negative" score is high, the detection is suppressed as a false positive; if the "affirmative" score is high, an alert is triggered with a natural-language description.
Edge Optimization: To meet real-time constraints, the YOLOv8s detector is quantized to INT8 using TensorRT (post-training quantization), while the CLIP encoder runs in FP16. This mixed-precision approach allows the system to maintain a high frame rate while keeping the VLM resident in memory alongside the detector.

Key Contributions

Integrated Edge Pipeline: An end-to-end UAV system that couples a fine-tuned YOLOv8s detector with a CLIP-based VLM for joint detection and natural-language scene description, deployed entirely on-device.
Conditional Inference Architecture: A design that decouples detection from verification, invoking the computationally expensive VLM only on 8–12% of frames (those flagged by the detector), thereby sustaining real-time throughput.
Hardware Characterization: Comprehensive deployment and quantification of the pipeline on the NVIDIA Jetson Orin Nano, including rigorous comparisons of FP32, FP16, and INT8 precision modes.
Ablation and Operational Analysis: Systematic studies on detection thresholds, prompt library sizes, model variants, and flight parameters (altitude, speed), alongside an analysis of power budgets and communication robustness under packet loss.
Performance Validation: Experimental evidence demonstrating that VLM integration reduces false positives by 21% compared to detector-only baselines while maintaining real-time performance (26.2 FPS).

Experimental Results
Evaluated on a held-out test set of 932 images (1,847 instances) across diverse environmental conditions:

Accuracy Metrics: The proposed system achieved 94.7% accuracy, 93.1% precision, 95.4% recall, and an F1-score of 0.942. This represents a +1.5 percentage point improvement in accuracy and precision over the YOLOv8s-only baseline.
False Positive Reduction: The CLIP verifier reduced false positives from 147 to 116 instances (a 21.3% reduction), primarily by suppressing detections caused by sun glare and seasonal color changes.
Latency and Throughput: With TensorRT INT8 acceleration, the average end-to-end latency is 38.1 ms per frame (26.2 FPS). The YOLO detector alone takes 18.4 ms, while the CLIP verifier takes 74.6 ms when invoked. The conditional strategy ensures the system remains above the 20 FPS threshold required for autonomous patrol.
Environmental Performance: The system maintained an F1-score >0.90 across most conditions but showed degradation in wind-driven smoke dispersal (F1=0.874) and low-illumination dusk (F1=0.887), where detector recall is the limiting factor.
Operational Constraints: The system operates within a 10 W power envelope for the compute module (total avionics ~29.5 W), contributing minimally to the UAV's total power budget. End-to-end alert latency (including radio transmission) is approximately 109 ms.

Significance and Claims
The paper claims that VLM-augmented edge inference is a practical approach to autonomous wildfire surveillance. The primary significance lies in the ability to run a vision-language model on a resource-constrained UAV edge device without sacrificing real-time performance. By integrating natural-language reasoning, the system not only reduces operator alert fatigue through false-positive suppression but also provides immediate, human-readable context (e.g., "dense smoke column consistent with active ground fire") that aids in rapid decision-making.

The authors position this work as a step toward more robust autonomous surveillance, noting that while the system is effective in Mediterranean and temperate biomes, performance in extreme conditions (wind-dispersed smoke, dusk) remains below the threshold for fully unsupervised triggering. They suggest that future work should focus on RGB+thermal sensor fusion and dynamic prompt adaptation to address these specific failure modes. The study concludes that the proposed architecture fits within realistic power and payload envelopes, offering a viable path for early-warning systems that intervene before fires reach the crown transition phase.

Technical Summary: Drone-Based Early Forest Fire Detection Using Vision-Language Models on NVIDIA Jetson Orin Nano

More like this