AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control

The paper introduces AerialVLA, a minimalist end-to-end Vision-Language-Action model that enables autonomous UAV navigation and precision landing by directly mapping raw visual inputs and fuzzy linguistic instructions to continuous control signals, thereby eliminating the need for dense oracle guidance or auxiliary object detectors while achieving state-of-the-art performance and superior generalization on the TravelUAV benchmark.

The Gist

Imagine you are trying to teach a drone to find a specific object, like a red motorcycle, in a giant, messy city park. You can't give it a GPS coordinate because the drone doesn't know where it is. You can only talk to it in plain English: "Fly to the right and find the red motorcycle parked on the crosswalk."

For a long time, scientists tried to teach drones this way using a "modular" approach. Think of this like a strictly supervised tour guide.

1. The Guide: A human (or a computer program) constantly whispers exact directions into the drone's ear: "Turn right now," "Go forward 5 meters," "Stop."
2. The Spotter: A separate, specialized camera system (like a security guard) constantly scans the ground to tell the drone, "Okay, I see the motorcycle. Stop!"

The problem? If the guide gets tired or the security guard gets confused, the whole system fails. The drone becomes a passive passenger, not a pilot. It doesn't actually learn how to navigate; it just follows orders.

Enter AerialVLA: The "Self-Reliant Pilot"

The authors of this paper, AerialVLA, decided to throw out the tour guide and the security guard. Instead, they built a drone that acts like a smart, intuitive human pilot who just needs a rough idea of where to go.

Here is how they did it, using simple analogies:

1. The "Two-Eye" Strategy (Minimalist Perception)

Most drones try to see everything with five or six cameras (front, back, left, right, down). It's like trying to read a book while someone is shouting five different stories at you at once. It's overwhelming and slow.

AerialVLA is like a pilot who only looks through two windows:

• The Front Window: To see where to fly and avoid trees.
• The Down Window: To see the ground for landing.
  By ignoring the "noise" of the other angles, the drone processes information faster and focuses on what actually matters.

2. The "Fuzzy Compass" (Fuzzy Directional Prompting)

Instead of a robot voice saying "Turn 45 degrees right," the drone gets a vague hint based on its own internal sensors: "The target is somewhere to your right."

This is like telling a friend, "The coffee shop is somewhere down that street," instead of giving them a turn-by-turn map.

• Why is this better? It forces the drone to actually look and think. It has to scan the environment, recognize the motorcycle, and figure out the path itself. It stops being a robot that follows orders and starts being an agent that solves problems.

3. The "Magic Landing" (Unified Control)

Usually, when a drone finds its target, a separate computer program has to jump in and say, "Okay, stop!" If that program glitches, the drone crashes or flies past the target.

AerialVLA is different. It learns to land itself as part of the same thought process.

• Imagine a driver who doesn't just drive to a house but also knows exactly when to hit the brakes and park the car without needing a second person to yell "STOP!"
• The drone learns to say, "I see the target, I'm close, and I'm going to land now" all in one smooth motion.

The Result: A Super-Adaptable Drone

The researchers tested this new drone in a virtual world that was full of obstacles and new, unseen environments.

• The Old Way: When the drone saw a new type of object (like a blue truck instead of a red motorcycle) or a new map, it got confused and crashed. It relied too much on the "guide" and the "security guard."
• The AerialVLA Way: It handled new situations three times better than the best existing drones. Because it learned to see and act directly, it could generalize. If it learned to find a motorcycle, it could easily find a dog or a car, even if it had never seen them before.

In a Nutshell

Previous drone navigation was like a puppet being pulled by strings (guidance) and watched by a supervisor (detectors).
AerialVLA is like a child learning to ride a bike: You give them a general direction ("Go find the ice cream truck"), and they learn to balance, steer, and stop on their own by looking at the road. It's simpler, faster, and much more capable of handling the real, messy world.

Technical Summary

1. Problem Statement

Vision-Language Navigation (VLN) for Unmanned Aerial Vehicles (UAVs) faces significant challenges in transitioning from ground-based 2D navigation to complex 3D open-world environments. Existing approaches suffer from two primary limitations, termed the "double crutches":

1. Dependency on Oracle Guidance: State-of-the-art methods rely on dense, ground-truth directional hints (e.g., "Turn Right") injected into prompts. This turns the agent into a passive instruction follower, bypassing the need for active spatial reasoning and path planning.
2. Reliance on External Detectors: Modular pipelines often require external object detectors (e.g., Grounding DINO) to trigger landing or stopping. This creates a disjointed perception-control loop where the policy controls movement but relies on a "black box" to decide when to stop, reducing robustness in open-world scenarios where detectors may fail.

The goal is to enable UAVs to navigate autonomously to natural-language-described targets using only onboard sensors and raw visual inputs, without external guidance or detectors.

2. Methodology: AerialVLA

AerialVLA is a minimalist, end-to-end Vision-Language-Action (VLA) framework that maps raw visual observations and fuzzy linguistic instructions directly to continuous physical control signals. It is built upon the OpenVLA-7B backbone (Llama-2 + SigLIP/DINOv2) with three core innovations:

A. Minimalist Dual-View Perception

• Strategy: Instead of processing redundant multi-camera arrays (e.g., 5 views), AerialVLA fuses only Front and Down views into a vertical mosaic.
• Rationale: The front view handles obstacle avoidance and target identification, while the down view is critical for precise ground alignment and landing.
• Implementation: The images are concatenated ($I_{comp} = [I_{front}; I_{down}]$), resized to $224 \times 224$, and processed by a hybrid visual encoder (SigLIP for semantics, DINOv2 for spatial features). This design aligns with consumer UAV hardware, reduces computational latency, and prevents intra-patch corruption in the Vision Transformer (ViT).

B. Fuzzy Directional Prompting

• Strategy: The system eliminates dense oracle guidance. Instead, it uses fuzzy directional hints derived solely from onboard IMU/GPS estimations of the target's relative bearing.
• Mechanism: The relative angle $\theta$ is discretized into coarse semantic buckets (e.g., "straight ahead," "forward-right," "to your left rear") as shown in Table 1 of the paper.
• Benefit: This forces the agent to perform active visual grounding and robust spatial reasoning rather than passive following. It introduces spatial ambiguity that mimics real-world localization uncertainty, enhancing generalization.
• Geometry-Consistent Filtering: To resolve causal confusion during training (e.g., a lateral hint paired with straight flight), the authors filter out ambiguous frames where a lateral target exists but the path is clear, ensuring the model learns valid evasion maneuvers only when obstacles are present.

C. Unified 3-DoF Control via Numerical Tokenization

• Action Space: The model outputs a continuous 3-Degree-of-Freedom (3-DoF) command: $\langle \Delta x, \Delta z, \Delta \psi \rangle$ (forward displacement, vertical displacement, and yaw change).
• Numerical Tokenization: Instead of learning new action tokens from scratch (which causes a "cold-start" problem), AerialVLA maps continuous actions to existing numerical tokens in the LLM vocabulary (discretized into 99 bins). This leverages the pre-trained model's inherent understanding of magnitude and order.
• Intrinsic Landing: The model learns a unified stopping policy. Landing is triggered either by generating a specific LAND token or by predicting near-zero spatial offsets. This eliminates the need for external object detectors to decide when to stop.

3. Key Contributions

1. Minimalist Dual-View Perception: A streamlined visual interface that balances information gain with efficiency, discarding redundant views while preserving essential cues for navigation and landing.
2. Fuzzy Directional Prompting: A novel prompting mechanism that removes reliance on step-by-step oracle guidance, forcing the agent to learn active spatial reasoning and robustness against sensor noise.
3. High-DoF End-to-End Control: A unified control space that integrates continuous kinematic commands with an intrinsic landing signal, achieving precision stopping without external detectors.
4. Numerical Tokenization Strategy: Leveraging pre-trained LLM numerical reasoning for action prediction, enabling faster convergence and smoother control trajectories compared to custom action token schemes.

4. Experimental Results

The model was evaluated on the TravelUAV benchmark (UAV-Need-Help task) across Seen, Unseen Object, and Unseen Map splits.

• Seen Environments: AerialVLA achieved a 47.96% Success Rate (SR) and 38.54% Success weighted by Path Length (SPL), outperforming the strongest baseline (LongFly) by +11.57% SR and +7.47% SPL.
• Unseen Objects: It demonstrated superior generalization to novel target categories with a 56.60% SR, significantly outperforming baselines that rely on specific object detectors.
• Unseen Maps (Zero-Shot Generalization): In entirely novel environments, AerialVLA achieved 37.58% SR, which is nearly three times the performance of the SOTA baseline (LongFly at 11.27%). This validates the model's ability to transfer visual-motor representations to new geometries without relying on historical spatial memory.
• Efficiency: The system runs with 0.38s total latency on an RTX 4090, significantly faster than modular baselines that require external grounding modules (0.63s).

5. Significance and Impact

• Paradigm Shift: AerialVLA challenges the prevailing modular approach in UAV navigation. It demonstrates that a minimalist, autonomy-centric paradigm can outperform complex systems that rely on dense oracles and external detectors.
• True Autonomy: By removing "double crutches," the system achieves genuine autonomy, capable of navigating and landing in dynamic, unstructured 3D environments using only onboard sensors.
• Robustness: The end-to-end nature of the model allows it to handle visual ambiguity and distractors through active error correction, rather than failing when external detectors miss a target.
• Scalability: The use of numerical tokenization and dual-view perception offers a computationally efficient path toward deploying VLA models on real-world consumer UAVs with limited onboard compute.

In conclusion, AerialVLA establishes a new standard for UAV-VLN by proving that stripping away redundant modules and oracle dependencies leads to more robust, generalizable, and truly autonomous aerial agents.