💻 computer science

A drone that learns to efficiently find non-uniformly distributed objects in agricultural fields: from simulation to the real world

This paper presents a Reinforcement Learning-based drone path planner that leverages low-quality prior knowledge to efficiently detect non-uniformly distributed agricultural objects, achieving significantly shorter flight paths than full-coverage methods in both simulation and real-world trials while demonstrating a manageable simulation-to-reality gap.

Original authors: Rick van Essen, Eldert van Henten, Gert Kootstra

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Rick van Essen, Eldert van Henten, Gert Kootstra

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where tiny, buzzing robots fly over our farms, acting like super-powered scouts. These aren't just any robots; they are drones equipped with cameras and brains, designed to help farmers grow better crops. But there's a catch: these drones run on batteries, and just like your phone, they can only fly for so long before they need to recharge. In the past, farmers told these drones to fly in a perfect, boring grid pattern, like a lawnmower cutting grass, to make sure they saw every single inch of the field. This works, but it's slow and wastes a lot of battery on empty spots where nothing interesting is happening.

Enter a new idea: what if the drone could be smarter? What if it could learn to spot where the "weeds" (unwanted plants) are hiding and zoom straight to them, skipping the empty spaces? This is the world of Reinforcement Learning (RL). Think of RL like teaching a dog a trick. You don't give the dog a manual; you let it try things, and every time it does something good, you give it a treat (a reward). Over millions of tries, the dog learns the best way to get the most treats. In this case, the "dog" is a computer program, the "treat" is finding a weed, and the "trick" is flying the shortest path possible. The big question scientists are asking is: Can we teach a drone to do this in a video game (simulation) and then have it work perfectly in the messy, real world?

The Paper: Teaching a Drone to Hunt Weeds Like a Pro Gamer

This paper tells the story of researchers who tried to teach a drone to hunt for weeds in a field without wasting a single drop of battery. They didn't just write a set of rules for the drone to follow; instead, they let a computer "brain" learn how to fly by playing a game over and over again.

The Training Ground: A Video Game World
First, the researchers built a super-realistic video game. In this game, the drone flies over a digital field filled with fake weeds. The weeds aren't scattered randomly like sand on a beach; they are clustered in patches, just like real weeds often are. The drone has a "brain" (an AI agent) that looks at a map. This map has two layers: a zoomed-out view showing where the drone thinks weeds might be (based on a quick, blurry scan from high up), and a zoomed-in view showing what the drone sees right now.

The goal was simple: Find as many weeds as possible before the battery dies. The drone gets points for finding weeds, but it loses points for every second it flies and for crashing. The researchers used a method called Deep Reinforcement Learning to train the drone. They let the drone fly millions of times in the simulation, trying different paths. Sometimes it flew in circles and got no points; other times it zig-zagged perfectly and found a whole patch of weeds. Slowly, the drone learned the best strategy: "Don't fly everywhere. Fly where the clues say the weeds are."

The Big Test: From Game to Reality
Once the drone was a pro in the video game, the researchers had to see if it could handle the real world. This is where things get tricky. Real fields aren't perfect grids, the wind blows, cameras get blurry, and the "clues" (prior knowledge) aren't always right.

To test this, they didn't just jump straight to flying a real drone. They created a "ladder of reality" with six steps:

Level 1: Pure video game (everything is fake).
Level 2: The game uses real weed locations from photos, but the rest is fake.
Level 3: The game uses real "clues" about where weeds might be.
Level 4: The game uses a real camera to "see" the weeds in the photos.
Level 5: The drone uses real clues collected during a flight.
Level 6: The drone actually flies in the real world, taking live photos and making decisions on the spot.

What They Found
The results were a mix of "Wow!" and "Oops."

In the Game (Simulation): The trained drone was amazing. It flew 57% shorter paths than the old-fashioned "mow-the-lawn" drone. It found almost as many weeds (only 13% fewer), but it did it so much faster that it saved a huge amount of battery.
On Real Photos: When they tested the drone on real photos of fields (but not flying yet), it still flew 38% shorter paths than the old method. However, it found 21% fewer weeds. This means it was faster, but it missed some targets.
In the Real World (Flying): When they actually flew the drone in the field, the results were a bit shaky. In one test, the drone found 73% of the weeds. In another, it only found 23%.

Why Did It Struggle?
The researchers discovered that the main culprit wasn't the drone's flying skills or its ability to see weeds. The problem was the "clues" (the prior knowledge map).
Imagine you are playing a game of hide-and-seek, and someone gives you a map saying, "The hider is in the red house." If the map is wrong and the hider is actually in the blue house, you'll waste time looking in the red house. In the real-world flights, the "map" they made from a high-altitude scan was so poor (it missed almost all the weeds) that the drone got confused. It didn't know where to look, so it just flew around aimlessly or gave up too soon.

The paper also noted that sometimes the drone got stuck in a loop, flying back and forth between two spots like a hamster on a wheel, because it didn't know when to stop and land.

The Bottom Line
This paper shows that teaching a drone to be smart and efficient is possible, but it's not perfect yet. The AI learned to fly much shorter paths than the old methods, which is great for saving battery. However, for this to work in real life, the "clues" we give the drone need to be much better. If the map is bad, the smartest drone in the world will still get lost.

The researchers conclude that while this approach isn't ready to replace all farm drones tomorrow, it's a huge step forward. For tasks like finding weeds or spotting sick plants—where you don't need to find every single one, just most of them quickly—this smart, learning drone could save farmers a lot of time and money. But first, they need to fix the "clue" system so the drone doesn't get confused by bad maps.

Technical Summary: A Simulation-Trained RL Drone for Efficient Object Search in Agriculture

Problem Statement

Unmanned aerial vehicles (drones) offer significant potential for precision agriculture, enabling tasks such as early disease detection, weed localization, and soil assessment. However, their operational efficiency is severely constrained by limited battery capacity, which restricts the area coverable in a single flight. Traditional approaches rely on full coverage planners that map entire fields with uniform spatial distribution. This is often suboptimal for agricultural tasks where target objects (e.g., specific weed species or disease patches) are non-uniformly distributed in spatial clusters rather than randomly scattered. In such scenarios, exhaustive coverage is unnecessary; the goal is to detect the majority of objects with a minimal flight path. While adaptive path planners exist, rule-based methods often require explicit redesign for new environments and struggle with uncertainty. This paper addresses the challenge of developing a learning-based adaptive path planner that can efficiently navigate non-uniformly distributed objects, balancing detection recall with flight path length, and successfully transferring from simulation to real-world deployment.

Methodology

Reinforcement Learning Framework

The core of the approach is a Deep Q-Network (DQN) agent trained to maximize long-term rewards by making sequential decisions in a Markov Decision Process (MDP).

State-Space: The agent observes a drone-centered global map (low-resolution, covering the whole field) and a local map (high-resolution, corresponding to the camera's field of view). These maps include layers for field borders, previously found objects, a coverage map (to discourage revisiting), and a prior knowledge map (derived from high-altitude surveys). The state also includes the drone's current battery level.
Action-Space: The agent selects from eight directional actions (North, South, East, West, and four diagonals) and a "land" action to terminate the search. Each action moves the drone a fixed number of grid cells.
Reward Function: The agent receives positive rewards for correctly detecting objects and negative penalties for flight steps, entering no-fly zones, crashing (battery depletion), and revisiting already-explored grid cells.

Simulation and Domain Randomization

Training occurs in an abstract simulation environment designed to bridge the sim-to-real gap through domain randomization:

Environment: Fields are generated as random convex or concave polygons. Object distributions are modeled as clusters following multivariate Gaussian distributions, with varying numbers of clusters and objects.
Noise Modeling: The simulation incorporates realistic detection errors (false positives and false negatives) for both the observation map (simulated camera) and the prior knowledge map.
Training: The agent is trained using experience replay and a discount factor of 0.99 over $2 \times 10^7$ steps. The network architecture processes the global and local maps via parallel convolutional layers, concatenates them with battery status, and outputs action values through fully connected layers.

Real-World Application Pipeline

To deploy the simulation-trained policy:

Object Detection: A YOLOv11-m network detects objects in real-time drone images.
Coordinate Conversion: Real-world UTM coordinates are transformed into the simulation's grid system, accounting for field rotation and scale.
Prior Knowledge Generation: A high-altitude, full-coverage flight (without overlap) generates a binary prior knowledge map. This map is intentionally noisy, prioritizing recall over precision to ensure no potential targets are missed.
Execution: The RL agent's actions are converted to GPS coordinates and executed via a custom MAVLink application on a DJI M350 drone.

Evaluation Strategy: Six Levels of Realism

The paper introduces a systematic evaluation framework using six levels of realism to quantify the contribution of each component to the sim-to-real gap:

Level 1: Pure simulation (all components simulated).
Level 2: Real object positions (from orthomosaics) in a simulated environment.
Level 3: Real object positions + Real prior knowledge (generated from orthomosaics).
Level 4: Real object positions + Real prior knowledge + Real observations (YOLOv11 applied to orthomosaic crops).
Level 5: Real object positions + Real prior knowledge (collected during flight) + Real observations (from orthomosaics).
Level 6: Full real-world deployment (live flight with real controller and live camera feed).

Key Contributions

Enhanced Simulator: Extension of the original simulator to include larger, randomly shaped fields and an expanded set of diagonal flight actions.
Domain Randomization: Training on a diverse set of object distributions, cluster counts, and field shapes to improve generalization.
Real-World Prior Knowledge: Integration of prior knowledge derived from real-world high-altitude flights.
Systematic Gap Analysis: Evaluation across six realism levels using 28 orthomosaic-based repetitions and two real-world flight trials to isolate specific sources of performance degradation.
Real-World Deployment: Demonstration of the RL agent directly controlling a DJI M350 drone for navigation decisions.
Open Source: Public release of code, data, and evaluation pipelines.

Results

Simulation-to-Reality Performance

Efficiency vs. Recall Trade-off: In pure simulation (Level 1), the RL agent achieved a 57% shorter flight path compared to a full coverage planner, with a 13% lower recall (0.85 vs. 0.98).
Real-World Orthomosaics (Level 4): On real-world orthomosaic data, the RL agent achieved a 38% shorter flight path with a 21% lower recall (0.77 vs. 0.98) compared to full coverage.
Generalization: The agent generalized well to real-world object distributions (Level 1 to 2 drop was only ~2%). The transition to real detection networks (Level 3 to 4) caused a 6% recall drop, indicating the simulation of detection errors was reasonably accurate.

Real-World Flight Trials

Two real-world flights (Level 6) were conducted:

Trial 1: The agent found 73% of the weeds in 91 steps.
Trial 2: The agent found only 23% of the weeds in 93 steps.
Root Cause Analysis: The poor performance in Trial 2 (and the drop in Trial 1 compared to orthomosaic evaluations) was primarily attributed to low-quality prior knowledge. In Trial 2, the prior knowledge recall dropped to 0%, effectively providing no guidance to the agent. In Trial 1, prior knowledge recall was only 5%. The study notes that while prior knowledge need not be perfect, a recall of 0% leads to significant performance degradation.

Flight Path Inefficiencies

The agent exhibited looping behavior (repeatedly flying back and forth between two points) and sometimes continued flying after most targets were found. This suggests the reward penalty for revisiting areas ( $r_{nocov}$ ) was insufficient to fully discourage such behavior in complex real-world scenarios.

Significance and Claims

The paper claims that learning-based adaptive path planning can significantly improve data collection efficiency in precision agriculture when exhaustive coverage is not required. By trading a moderate reduction in recall for a substantial reduction in flight time (38% shorter paths), the approach offers a viable alternative to full coverage planners for tasks like weed and disease detection, where targets are clustered.

The authors emphasize that the sim-to-real gap is manageable but sensitive to specific components:

Object Distribution: The agent generalizes well to real spatial patterns.
Detection Networks: Simulated errors effectively prepare the agent for real-world detection noise.
Prior Knowledge: This is the most critical factor. The study highlights that inaccurate or missing prior knowledge is the dominant source of failure in real-world deployment, more so than the flight controller or live camera feed quality.

The paper concludes that while the approach shows promise for practical deployment, further improvements in prior knowledge quality (e.g., reusing data from previous flights) and flight path efficiency (e.g., hybrid routing or hierarchical strategies) are necessary before widespread adoption. The work serves as a proof-of-concept that simulation-trained RL policies can be transferred to real drones for adaptive search tasks.