💻 computer science

Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning

This paper proposes a vision-guided reinforcement learning framework that enables quadcopters to autonomously navigate and evade obstacles in unknown, GNSS-denied outdoor environments through a two-stage training process involving supervised learning and curriculum fine-tuning, achieving successful zero-shot sim-to-real transfer.

Original authors: Shiladitya Dutta, Aayush Gupta, Varun Saran, Avideh Zakhor

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shiladitya Dutta, Aayush Gupta, Varun Saran, Avideh Zakhor

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a remote-controlled drone. Usually, you need to hold a controller, watch a screen, and constantly steer it to avoid crashing into trees, buildings, or other obstacles. If you lose the signal or the battery dies, the drone stops. This paper presents a way to teach a drone to fly itself through a messy, unknown forest or city without a human pilot, using only its own "eyes" and "brain."

Here is how they did it, explained simply:

The Problem: The "Pilot" Bottleneck

Most drones today are like cars with a driver who never sleeps. They are great at moving fast and turning sharply, but they can't fly alone in tricky places (like under a dense forest canopy or in a war zone) because they need a human to constantly tell them where to go. The goal of this research was to build a drone that can look at the world, figure out where the obstacles are, and fly to a destination all by itself.

The Solution: A Two-Part Brain

The researchers built a "brain" for the drone that works in two main parts, similar to how a human learns to drive:

The Eyes (The Autoencoder): The drone uses a stereo camera (two lenses like human eyes) to see depth. However, raw video is too much data for a small computer to process quickly. So, they first trained a special AI to act like a "summarizer." It looks at the complex 3D view of the world and compresses it into a simple, low-level "feeling" of the space. Think of it like turning a high-definition movie into a quick sketch that captures the essence of the room.
The Pilot (The LSTM Network): Once the drone has this "sketch" of the world, it passes it to a second part of the brain called an LSTM. This is a type of AI that is good at remembering things over time. It looks at the sketch, remembers where it was a second ago, and decides: "Okay, I need to turn left and speed up." It outputs simple commands like "move forward at this speed" or "turn slightly," which the drone's built-in software can follow immediately.

The Training: From Video Games to Reality

You can't just teach a real drone to fly by crashing it a thousand times; it would break. So, the researchers taught the drone in a video game simulation first.

Stage 1: The Tutor (Privileged Learning): Imagine a student pilot learning to fly. At first, they have a super-tutor who knows the perfect path through the forest. The AI drone watches this "perfect path" and gets a reward for staying close to it. It's like training wheels; the drone learns the basic rules of not hitting things and moving toward a goal.
Stage 2: The Solo Flight (Curriculum Learning): Once the drone knows the basics, the tutor disappears. The drone is thrown into a harder, more complex game level with random obstacles. Now, it has to figure out the path on its own. The researchers slowly made the game harder (adding more walls, tighter turns) to teach the drone how to adapt to new situations.

The Magic Trick: Bridging the Gap (Sim-to-Real)

The biggest challenge in robotics is that what works in a video game often fails in the real world. In a game, physics are perfect. In reality, there is wind, the camera is blurry, and the drone might be heavier or lighter than expected.

To fix this, the researchers used a technique called Domain Randomization.

The Analogy: Imagine training a soccer player. If you only practice on a perfect, dry grass field, they might fail when it rains or the field is muddy. Instead, this team trained their drone in a "chaos simulator." They randomly changed the wind speed, added static noise to the camera images, made the drone heavier or lighter, and even simulated sensor glitches.
The Result: By training in this chaotic, unpredictable environment, the drone learned to be "tough." When it finally flew outside, it didn't care if the wind blew or the camera was a little blurry. It had already seen "worse" in the simulation.

The Real-World Test

They tested this system on a real, heavy-duty drone (a DJI M300, which is about 10 times heavier than the tiny drone they simulated) in an outdoor field.

The Setup: They set up obstacles like pillars and walls in a courtyard and an open field.
The Conditions: It was windy, the lighting was tricky, and the drone had to navigate without GPS (using only its camera and internal sensors).
The Outcome: The drone successfully flew through these unknown environments, dodged obstacles, and reached its goal 100% of the time in their tests. It flew at about 2.6 meters per second (roughly 6 mph), which is a brisk walking pace.

Why This Matters

The paper claims that this method is special because the drone didn't need to be re-tuned for the specific heavy drone they used. They trained it on a tiny, light drone in a game, and it worked perfectly on a giant, heavy real-world drone. This suggests that if you teach a drone to fly using "velocity commands" (telling it how fast to go) rather than complex physics, it can adapt to almost any drone, big or small, without needing a new manual for each one.

In short, they taught a drone to be a brave, self-reliant explorer that can handle messy, unknown worlds by learning in a chaotic video game and then taking that toughness to the real world.

Technical Summary: Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning

Problem Statement

While quadcopters offer high mobility, their application in GNSS- and telemetry-denied environments (e.g., sub-canopy forests, underground, war zones) is currently limited by the need for continuous pilot control or reliance on preset paths. Traditional autonomous navigation often decomposes tasks into separate perception, planning, and control units, which can suffer from compounding errors in state estimation and sensor noise. Furthermore, existing learning-based methods often produce policies tied to specific drone dynamics or require model predictive controllers (MPC) that cannot be directly deployed on off-the-shelf commercial drones via standard velocity command APIs. The core challenge addressed is achieving robust, zero-shot autonomous navigation in unknown, obstacle-rich outdoor environments using only onboard sensors (stereo vision and VIO) and a generic velocity command interface, without requiring platform-specific tuning.

Methodology

The authors propose a modular, end-to-end sensorimotor policy trained via a two-stage Reinforcement Learning (RL) and Privileged Learning paradigm. The system architecture and training pipeline are as follows:

1. System Architecture

Perception Head: A pre-trained denoising autoencoder maps high-dimensional stereo depth images ( $192 \times 108$ ) to a low-dimensional latent space ( $R^{128}$ ). This encoder is trained on the DIML dataset with injected Gaussian noise to ensure robustness to stereo depth noise and varied environments.
Planning and Control: A Long Short-Term Memory (LSTM) network processes the concatenated latent depth features and state vector (relative goal position, roll/pitch/yaw rates, heading error). The LSTM outputs velocity commands ( $v_x, v_z, \omega$ ) directly compatible with commercial drone APIs (e.g., DJI).
State Space: Includes relative goal position, angular rates, heading difference, and flattened depth images.
Action Space: Horizontal velocity, vertical velocity, and yaw angular velocity, scaled to the drone's maximum capabilities.

2. Training Pipeline

The policy is trained in the Flightmare simulator using Proximal Policy Optimization (PPO) through three distinct phases:

Stage 1: Optimal Trajectory Generation: A global motion planner generates dynamically feasible, jerk-minimized trajectories in randomized environments.
Stage 2: Privileged Learning: The policy is trained in a simple environment (spaced pillars) using the optimal trajectories as a supervisory signal. The reward function includes a term for proximity to the optimal trajectory to guide initial convergence and teach time-efficient flight behaviors.
Stage 3: Curriculum Fine-tuning: The policy is fine-tuned in a complex curriculum environment with three regions of increasing difficulty (pillars, panels, walls). Crucially, the optimal trajectory reward is removed in this stage to prevent the policy from overfitting to a single path in multi-modal environments, encouraging robust generalization.

3. Sim-to-Real Bridging

To ensure transfer to real-world hardware, the authors employ:

Domain Randomization: Randomization of drone mass, inertia, and control latency; addition of Gaussian noise to state measurements; and Gaussian blur on depth inputs to simulate real-world sensor degradation.
Reward Shaping: Specific penalties are applied to yaw rate magnitude, acceleration, and jerk (third-order derivative) to discourage high-frequency oscillations and ensure smooth, mechanically sound flight profiles that generalize across different drone dynamics.

Key Results

Simulation Evaluation

Performance: The policy achieved high success rates (99.2% at 1.0 m/s, dropping to 78% at 7.0 m/s) in simulated obstacle courses.
Ablation Studies: Removing reward terms for jerk, acceleration, or velocity magnitude resulted in erratic, high-frequency oscillations, confirming the necessity of smoothness penalties for stable control.
Training Strategy: The hybrid approach (Privileged Learning + Curriculum) outperformed policies trained solely on privileged learning or solely on curriculum learning, achieving the highest stable success rates and lowest time-to-goal.
Robustness: Compared to the baseline MAVRL, the proposed method maintained high success rates as drone dynamics were scaled up (simulating a 10x weight increase), whereas MAVRL performance degraded rapidly. The proposed policy generated significantly lower trajectory jerk across varying scales.

Real-World Experiments

Setup: Experiments were conducted on a DJI Matrice 300 RTK (6.3 kg, ~10x heavier than the simulated Kingfisher drone) equipped with a Zed2i stereo camera and Jetson AGX Xavier.
Environments: Tests were performed in three outdoor configurations (Wall, Triangle, Inverted Triangle) with varying start/goal positions, including modified environments with concealed obstacles and reduced clearance.
VIO Trials: Using only Visual-Inertial Odometry (VIO) for positioning, the system achieved a 100% success rate across 18 trials. The average distance to the goal was 2.27m, attributed to VIO drift (up to 3m) rather than navigation failure, as the drone successfully avoided obstacles and reached the vicinity of the goal.
GNSS Trials: When using GNSS for positioning, the system achieved a 100% success rate with an average distance to goal of 0.81m, demonstrating accurate collision-free navigation.
Speed: The drone maintained an average speed of 2.58 m/s, close to the commanded maximum of 3.0 m/s.

Significance and Claims

The paper claims that the proposed system successfully bridges the Sim-to-Real gap for vision-based autonomous flight in unknown environments. The primary significance lies in the demonstration of zero-shot transfer to a drone platform (DJI M300) with dynamics substantially different from the training simulation (10x weight difference) and in environments never encountered during training.

The authors emphasize that by using reinforcement learning with specific reward shaping and domain randomization, rather than imitation learning tied to specific dynamics, the policy learns to generate low-jerk, robust velocity commands. This allows the system to interface with off-the-shelf commercial drones via standard APIs without platform-specific tuning. The work validates that robust policies can abstract away complex flight dynamics, enabling reliable navigation in GNSS-denied, cluttered outdoor settings using only onboard stereo vision and VIO.