💻 computer science

Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System

Qwen-RobotNav is a scalable, parameterized navigation model trained on 15.6M samples that enables agentic systems to dynamically reconfigure observation strategies and task modes at inference time without architectural changes, achieving state-of-the-art performance and strong zero-shot generalization across diverse benchmarks and real-world robots.

Original authors: Jiazhao Zhang, Gengze Zhou, Hale Yin, Yiyang Huang, Zixing Lei, Qihang Peng, Haoqi Yuan, Jie Zhang, Xudong Guo, Xiaoyue Chen, An Yang, Fei Huang, Junyang Lin, Dayiheng Liu, Jingren Zhou, Zhuoyuan Yu

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiazhao Zhang, Gengze Zhou, Hale Yin, Yiyang Huang, Zixing Lei, Qihang Peng, Haoqi Yuan, Jie Zhang, Xudong Guo, Xiaoyue Chen, An Yang, Fei Huang, Junyang Lin, Dayiheng Liu, Jingren Zhou, Zhuoyuan Yu, Jingyang Fan, Zhixuan Liang, Pei Lin, Ye Wang, Anzhe Chen, Kun Yan, Xiao Xu, Jiahao Li, Lulu Hu, Minying Zhang, Shurui Li, Wenhu Xiao, Shuai Bai, Xuancheng Ren, Chenxu Lv, Chenfei Wu, Xiong-Hui Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot dog or a self-driving car. You want it to do many different things: follow a spoken instruction like "go to the kitchen," chase a moving person, find a lost set of keys, or drive safely through traffic.

Usually, building a robot that can do all these things is like trying to hire one person who is a master chef, a race car driver, and a detective all at once. It's incredibly hard because each job requires a different way of looking at the world. A detective needs to remember clues from hours ago, while a race car driver only cares about what's happening right in front of them this second.

Qwen-RobotNav is a new "brain" for robots that solves this problem. Instead of building a separate brain for every job, the researchers built one flexible brain that can change its "glasses" depending on what it's doing.

Here is how it works, using simple analogies:

1. The "Swiss Army Knife" Brain

Think of Qwen-RobotNav as a Swiss Army knife. Most navigation robots are like a single screwdriver; they are great at one thing but useless for others. Qwen-RobotNav is the whole tool.

The Switch: It has a "mode switch." If you tell it to "follow that person," it switches to Tracking Mode. In this mode, it puts on "fast glasses" that only look at the last few seconds of video, ignoring old history so it can react instantly.
The Memory: If you tell it to "find the red sofa," it switches to Search Mode. Here, it puts on "wide-angle glasses" that remember everything it has seen for the last 10 minutes, so it doesn't walk in circles.
The Magic: The best part is that you don't need to retrain the robot to change these glasses. You just tell it what to do, and it instantly adjusts how it pays attention to the world.

2. The "Smart Budget" for Eyes

Robots have a limit on how much visual information they can process at once (like a computer running out of RAM).

The Problem: If a robot looks at 100 frames of video, it might get overwhelmed. If it only looks at 1 frame, it might miss a crucial detail.
The Solution: Qwen-RobotNav uses a Smart Budget. Imagine you have $100 to spend on buying pictures of a room.
- If you are driving, you spend most of your money on the picture of the road right now (the front view) and very little on the rear view.
- If you are searching for a lost item, you spread your money out to buy pictures of the whole room over the last few minutes.
- The robot learns to do this math automatically. It decides exactly how many "pixels" (visual details) to keep for every camera and every moment in time, based on the task.

3. Learning from a "Mega-Mix" of Experiences

To teach this robot, the researchers didn't just show it one type of video. They fed it a massive "smoothie" of 15.6 million different experiences:

Instruction Following: "Turn left at the blue chair."
Point Navigation: "Go to coordinate (5, 2)."
Object Search: "Find the TV remote."
Tracking: "Follow the man in the black hat."
Driving: "Drive safely through an intersection."

They also mixed in general "world knowledge" (like reading signs or recognizing objects) so the robot doesn't forget how to understand language while learning to move. This prevents the robot from becoming a "mindless reflex machine" that just moves without thinking.

4. The "Manager and the Worker" System

For very long and complicated tasks (like "Find the green umbrella at the coffee shop and tell me if it's there"), the robot works in a team:

The Manager (Upper Planner): This is a high-level AI that breaks the big goal into small steps. It decides, "First, go to the hallway. Then, look for the coffee shop."
The Worker (Qwen-RobotNav): This is the navigation model. The Manager tells the Worker, "Go to the coffee shop, but use your 'Search Mode' with a big memory budget."
The Loop: The Worker goes, finds the shop, and reports back, "I'm here, but no umbrella." The Manager updates its memory and says, "Okay, check the table next." This happens over and over until the job is done.

5. Real-World Results

The paper shows that this robot isn't just a simulation trick.

It wins competitions: It beat other top robots in tests for following instructions, finding objects, and tracking moving targets.
It drives: It learned to drive cars safely in complex traffic simulations.
It works on real robots: The team tested it on a real robot dog in a real building it had never seen before. They gave it spoken instructions like "Walk to the medical room, then turn around and walk backward." The robot successfully navigated the strange building, used landmarks to find its way, and even walked backward exactly as asked.

In summary: Qwen-RobotNav is a universal navigation brain that learns to "tune" its own attention. It knows when to focus on the present moment and when to remember the past, allowing a single robot to be a driver, a tracker, and a searcher all at once.

Technical Summary: Qwen-RobotNav

Problem Statement

Embodied navigation encompasses a diverse family of tasks—including instruction following, point-goal navigation, object searching, target tracking, and autonomous driving. While recent unified navigation models have demonstrated that a single architecture can handle multiple task families, they fail to address a critical requirement for agentic navigation systems: the need for an observation strategy that can be externally reconfigured at inference time.

Different navigation tasks demand fundamentally different temporal context and memory strategies. For instance, instruction following requires retaining observations spanning dozens of prior steps to re-reference distant landmarks, whereas target tracking relies almost exclusively on the most recent frames, treating stale history as noise. Object searching further complicates this by shifting requirements within a single episode (from broad global memory during exploration to tight recency-focused attention during approach). Existing unified models (e.g., NavFoM, ABot-N0) embed a single, fixed assumption about which observations matter (e.g., uniform sub-sampling or sliding windows), making them unable to adapt mid-episode or serve as flexible building blocks for higher-level planners.

Methodology

The authors introduce Qwen-RobotNav, a scalable navigation model built on the Qwen3-VL backbone. Rather than designing task-specific architectures, the authors reframe the challenge as observation context modeling. The core innovation is a parameterized interface that allows an external agent to dynamically configure how the model consumes the visual stream without architectural modification or task-specific retraining.

1. Architecture and Interface

Qwen-RobotNav inherits the Qwen3-VL vision encoder and language backbone, augmented with a lightweight 4-layer MLP action head that predicts an 8-waypoint trajectory $(x, y, \theta)$ . The model exposes two complementary configuration dimensions:

Task Modes: The model supports multiple modes (VLN, PointNav, ObjNav, Tracking) selectable via natural language prompts. These are not separate policies but different interfaces to the same model weights.
Controllable Observation Parameters: The model accepts a configuration $\Phi = (B, \gamma, \{w_c\}, m, b_{min}, b_{max})$ $Φ = (B, γ, {w_{c}}, m, b_{min}, b_{ma x})$ that governs visual token allocation:
- Token Budget ( $B$ ): Total visual tokens across all cameras and timesteps.
- Temporal Decay ( $\gamma$ ): Controls recency bias, weighting recent frames more heavily than older ones.
- Camera Weights ( $w_c$ ): Per-camera importance multipliers (e.g., front camera weighted higher than rear).
- Frame Sampling ( $m$ ): Random (global coverage) or Latest (recency window).

2. Task-Adaptive Observation Encoding

The model employs a constrained token allocation algorithm. Given a budget $B$ , it distributes tokens across $T'$ frames and $N$ cameras based on a joint weight matrix $W[t, c] = \omega_t \cdot w_c$ . The algorithm ensures a minimum floor ( $b_{min}$ ) and maximum ceiling ( $b_{max}$ ) per image, redistributing surplus tokens iteratively. This allows the model to dynamically adjust resolution and context length based on the task's needs (e.g., high resolution for recent frames in tracking, broader coverage for exploration).

3. Training Strategy

Data Scale: The model is trained on 15.6M samples, comprising 85% navigation trajectory planning data (spanning instruction following, point-goal, object-goal, tracking, and autonomous driving) and 15% navigation-related vision-language reasoning data.
Training-Time Randomization: A critical design principle is that no observation configuration is fixed during training. Hyperparameters ( $B, \gamma, w_c$ , etc.) are independently randomized per sample. This ensures the model generalizes to any inference-time configuration without task-specific tuning.
Co-training: Vision-language reasoning data is co-trained to prevent the "collapse" into reactive action-sequence mappers, preserving the model's open-world perception and spatial reasoning capabilities.
Embodiment Awareness: Different physical platforms (robots vs. cars) are distinguished via natural-language prompt preambles (e.g., "Imagine you are a robot..."), requiring zero architectural changes.

4. Agentic Navigation System

Qwen-RobotNav is designed as a reactive executor within a hierarchical agentic system. An upper-level planner (e.g., Qwen3.6-Plus) decomposes long-horizon goals into sub-tasks, dynamically switching Qwen-RobotNav's task mode and context strategy mid-episode. The system maintains a two-level memory: a compact single-episode summary and a persistent cross-episode "evidence notebook" for long-horizon reasoning.

Key Contributions

Parameterized Navigation Interface: A unified model that allows external reconfiguration of observation strategies (token budget, temporal decay, camera weights) at inference time, enabling a single model to serve diverse tasks and dynamic sub-goals.
Training-Time Randomization: A training regime that exposes the model to a broad combinatorial space of context configurations, ensuring robust zero-shot generalization to any inference-time setting without fine-tuning.
Agentic Building Block: A demonstration of how a navigation foundation model can serve as a modular, reconfigurable primitive for complex agentic systems, enabling long-horizon tasks like Embodied Question Answering (EQA) through dynamic task switching.
Scalable Multi-Task Learning: Joint training across five distinct task families (including autonomous driving) develops a shared spatial-planning substrate that transfers across task families and embodiments.

Experimental Results

Qwen-RobotNav achieves state-of-the-art (SOTA) or competitive results across diverse benchmarks:

Vision-and-Language Navigation (VLN-CE):
- RxR Val-Unseen: 76.5% Success Rate (SR) and 72.5 nDTW (8B model), surpassing NavFoM by 12.1% SR.
- R2R Val-Unseen: 72.1% SR (8B model), surpassing NavFoM by 10.4% SR.
- VLNVerse: 63.75% SR on fine-grained and 46.59% SR on coarse-grained splits, outperforming NavFoM and Uni-NaVid significantly.
Object-Goal Navigation:
- HM3D v2: 75.6% SR (4B model), establishing a new SOTA on this benchmark despite using only RGB observations (no depth).
- HM3D-OVON: 57.7% SR on the Seen split and 60.1% on Synonyms, outperforming ABot-N0 despite using a single forward-facing camera versus ABot-N0's panoramic input.
Active Visual Tracking (EVT-Bench):
- Achieves the highest tracking rate (90.0% TR) among all evaluated methods, surpassing ABot-N0 (87.6%) and TrackVLA++ (81.0%).
Autonomous Driving (NAVSIM):
- Achieves 91.4 PDMS (4B model) and 90.9 PDMS (8B model), outperforming NavFoM, AutoVLA, and other specialized driving models.
Embodied Question Answering (EQA):
- An agentic system built with Qwen-RobotNav sets new SOTA on HM-EQA (76.7 SR), MT-HM3D (54.4 SR), and EXPRESS-Bench (79.27 LLM Score), improving over the best prior method by 10.8% and 15.4% respectively, while requiring 77% fewer navigation steps.
Real-World Deployment:
- Demonstrated zero-shot transfer to a Unitree Go2 quadruped robot in a real-world exhibition hall, successfully executing complex language instructions (e.g., "walk backward," "stop beside the sofa") and long-horizon agentic tasks (e.g., finding a specific object and reporting observations).

Significance and Claims

The paper claims that Qwen-RobotNav represents a shift from designing fixed navigation policies to creating reconfigurable navigation primitives. By treating observation context as a first-class, externally controllable degree of freedom, the model bridges the gap between specialized task performance and the flexibility required for agentic systems.

The authors emphasize that the parameterized interface allows complex long-horizon behaviors to emerge from the composition of simple, well-defined working modes (e.g., switching from "global-history mode" for exploration to "recency-focused mode" for local maneuvering). Furthermore, the use of natural language for viewpoint and temporal identification preserves the model's open-world language grounding without architectural overhead.

The results validate that a unified model, trained with randomized context strategies, can achieve superior performance across instruction following, object search, tracking, and driving, while serving as a robust, zero-shot transferable component for general-purpose embodied agents. The work suggests that task-adaptive observation encoding is a promising direction for building navigation foundation models that are both broadly capable and practically deployable.