💻 computer science

NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation

NavWAM is a diffusion-transformer policy that unifies future visual prediction, goal-progress value estimation, and action generation into a shared latent sequence, enabling a robot to directly execute closed-loop navigation without relying on external planners or action search.

Original authors: Daichi Azuma, Taiki Miyanishi, Koya Sakamoto, Shuhei Kurita, Yaonan Zhu, Petr Khrapchenkov, Motoaki Kawanabe, Yusuke Iwasawa, Yutaka Matsuo

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daichi Azuma, Taiki Miyanishi, Koya Sakamoto, Shuhei Kurita, Yaonan Zhu, Petr Khrapchenkov, Motoaki Kawanabe, Yusuke Iwasawa, Yutaka Matsuo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to find a specific room in a house, but the robot can only see what's directly in front of its "eyes" (a camera). It doesn't have a map, and it can't see around corners. This is the challenge of goal-conditioned visual navigation.

Here is the story of how the authors, Daichi Azuma and his team, solved this problem with a new system called NavWAM.

The Problem: The "Crystal Ball" vs. The "Driver"

In the past, researchers tried to teach robots to navigate using two separate tools:

The Crystal Ball (World Model): This tool is great at guessing what the robot will see if it moves forward. "If I turn left, I'll see a kitchen." "If I go straight, I'll hit a wall."
The Driver (Planner): This tool looks at the Crystal Ball's guesses and decides, "Okay, the kitchen looks good, so I'll turn left."

The Flaw: The paper argues that keeping these two separate is inefficient. It's like having a passenger who screams out directions ("Turn left!") while the driver has to stop, think, and then turn. It creates a bottleneck. The "Crystal Ball" predicts the future, but it doesn't know how to drive the car to get there.

The Solution: NavWAM (The "Driver-Predictor")

The authors created NavWAM (Navigation World Action Model). Think of NavWAM as a super-driver who can also see the future.

Instead of having a separate "Crystal Ball" and a "Driver," NavWAM combines them into one brain. It doesn't just guess what it will see; it guesses what it will see, how close it is to the goal, and exactly what steering commands to give—all at the same time.

The Creative Analogy: The "Shared Canvas"

To understand how NavWAM works, imagine a painter working on a single canvas with nine different panels:

The Bottom Panels (What we know): These show the robot's current view, where it is right now, and the picture of the goal (e.g., a photo of a specific chair).
The Top Panels (What we predict): The robot paints these panels to show:
1. What the robot will see in the future.
2. How close it will be to the goal.
3. The most important part: The actual steering commands (the "action chunk") needed to get there.

The robot learns by "denoising" this canvas. It starts with a messy, blurry version of the future and cleans it up until it has a clear picture of the path, the destination, and the steps to take. Because the "steering commands" are painted on the same canvas as the "future view," the robot learns that to see the goal, it must take these specific actions.

How It Beats the Competition

The paper tested NavWAM against two other types of robots:

The Old Way (NWM): The robot predicts the future, then uses a complex, slow math search (called CEM) to figure out which action to take. It's like a chess player calculating 100 moves before making one.
The Direct Way (OmniVLA): The robot just looks at the goal and guesses the next move without thinking about the future. It's like a driver who just reacts to the road without looking ahead.

The Results:

NavWAM vs. The Old Way: NavWAM was much faster and more accurate because it didn't need to stop and calculate a complex search. It just "knew" the move. It reached the goal in 79% of real-world tests, while the Old Way only succeeded 16% of the time.
NavWAM vs. The Direct Way: NavWAM performed just as well as the Direct Way (which uses a much larger, more powerful computer brain), but NavWAM had the added superpower of actually predicting what the future would look like.

The Real-World Test

The team didn't just test this in a computer simulation. They put NavWAM on a real robot named Diablo and sent it into four different indoor environments (an office, a storage room, a meeting room, and a hallway).

The Goal: Find a specific image captured in that room.
The Outcome: NavWAM successfully navigated to the goal in 19 out of 24 attempts.
The "Foresight" Check: Even though NavWAM didn't need to use its future predictions to move (it just used the steering commands), the paper shows that its predictions were surprisingly accurate. When the robot predicted "I will see a door in 4 seconds," it actually saw a door in 4 seconds.

The Big Takeaway

The paper concludes that for a robot to navigate well, it shouldn't just be a "predictor" or just a "driver." It needs to be both at the same time. By learning to predict the future and the actions required to create that future in one single step, the robot becomes much better at finding its way, even in places it has never seen before.

In short: NavWAM teaches the robot to "drive while looking ahead," rather than "look ahead, stop, calculate, and then drive."

Technical Summary: NavWAM - A Navigation World Action Model for Goal-Conditioned Visual Navigation

1. Problem Statement

Goal-conditioned visual navigation requires a robot to navigate from partial egocentric observations to a specified goal (e.g., a target image). While recent direct policies can map observations to actions efficiently, they often lack explicit visual foresight—the ability to anticipate how their motion will alter future views and whether those changes bring them closer to the goal.

Conversely, Navigation World Models (NWMs) explicitly predict future egocentric observations conditioned on candidate actions. However, NWMs function as prediction modules rather than control policies. They rely on an external planner (typically using Cross-Entropy Method, CEM) to search over candidate action sequences, score the predicted futures, and select the best action for execution. This separation creates a bottleneck:

Computational Overhead: Test-time trajectory optimization is required.
Decoupling: The learned world model does not directly produce executable actions; the closed-loop behavior depends on the external planner's choices rather than the model's internal representation alone.

The central question addressed by this work is: How can explicit visual foresight be made directly usable for executable closed-loop control without relying on external planning?

2. Methodology: NavWAM

The authors propose NavWAM (Navigation World Action Model), a diffusion-transformer policy that unifies future prediction, value estimation, and action generation into a single joint prediction problem.

Core Architecture

NavWAM is built upon Cosmos Predict2, a pretrained diffusion-transformer video world model. Instead of using the model solely to predict future views for external planning, NavWAM treats navigation variables as a shared latent sequence (a "latent canvas").

Latent Canvas Representation: The model operates on a fixed nine-frame latent sequence.
- Conditioning Frames (Observed): A blank frame (for VAE temporal compression), the current robot state, the goal image, and the current egocentric observation.
- Prediction Frames (Generated): An executable action chunk, a future state, two future egocentric observations, and a scalar goal-progress value.
Unified Objective: The model learns to denoise this entire sequence simultaneously. This transforms navigation into a joint denoising problem over:
1. Future Perception: Predicting future egocentric views ( $o_{t+H-1:t+H}$ ).
2. Goal Progress: Estimating a scalar value ( $v_{t+H}$ ) indicating proximity to the goal.
3. Executable Motion: Generating action chunks ( $a_{t:t+H-1}$ ).

Training and Inference

Training: The model is trained using a diffusion objective ( $L_{diff}$ $L_{d i f f}$ ) on the latent canvas. The action frame is upweighted to ensure the low-dimensional action signal is not dominated by the high-dimensional image reconstruction loss. Three training modes are used:
1. Policy Mode: Conditions on current state/observation/goal; predicts action, future, and value.
2. World-Model Mode: Conditions on action as well; predicts resulting future/value.
3. Value Mode: Conditions on future frames; predicts value.
Inference: In the default Policy Mode, NavWAM takes the current observation and goal, denoises the sequence, and directly outputs an action chunk. This chunk is executed in a receding-horizon loop.
- No External Planner: Unlike NWMs, NavWAM does not require CEM-style search or candidate sampling during standard inference.
- Optional Best-of-N: The model supports an optional mode where $N$ candidates are sampled and scored using the auxiliary world-model/value heads, but the primary results utilize the default policy mode.

3. Key Contributions

NavWAM Policy: A novel architecture that converts NWM-style visual foresight into an action-producing policy for goal-conditioned visual navigation.
Joint Prediction Formulation: A method to represent future observations, goal-progress values, and executable action chunks in a shared latent sequence, allowing future prediction to directly support closed-loop action generation.
Performance without Search: Demonstration that NavWAM improves upon planning-based world-model baselines without CEM-style test-time trajectory optimization, while remaining competitive with larger direct navigation policies.

4. Experimental Results

The authors evaluated NavWAM on image-goal navigation using GO STANFORD (for comparison with NWM) and SIT (for comparison with OmniVLA). Real-world validation was performed on a Diablo mobile robot across four indoor environments.

Offline Benchmarks (GO STANFORD)

vs. Planning-Based Baselines: NavWAM significantly outperformed Cosmos Predict2 and NWM in Absolute Trajectory Error (ATE) and Relative Pose Error (RPE).
- NavWAM (w/ Fine-tuning): ATE 0.192, RPE 0.070.
- NWM: ATE 0.453, RPE 0.107.
- Cosmos Predict2: ATE 0.455, RPE 0.109.
Preservation of Foresight: Despite being an action-producing policy, NavWAM maintained high Subject Consistency (visual similarity between predicted and ground-truth future views), outperforming NWM (0.668 vs. 0.524).
Ablation: The study showed that image-only future prediction is insufficient for navigation. Adding action and value supervision was critical, with value supervision providing the best performance.

Comparison with Direct Policies (SIT)

vs. OmniVLA: NavWAM (using a 2B-parameter video backbone) achieved comparable or slightly better ATE and Success Rate (SR) than OmniVLA (built on a 7B-parameter OpenVLA backbone).
- NavWAM: ATE 0.077 (h=4), SR 46.3%.
- OmniVLA: ATE 0.086 (h=4), SR 45.4%.

Real-World Deployment

Setup: 24 closed-loop episodes across Office, Storage, Meeting Room, and Hallway environments.
Success Rate: NavWAM achieved the highest success rate (79.2%, 19/24 episodes), compared to OmniVLA (58.3%) and NWM (16.7%).
Qualitative Behavior: NavWAM demonstrated consistent goal-reaching, whereas NWM often drifted and OmniVLA sometimes stopped short or took less direct paths. Real-world rollouts confirmed that predicted future views qualitatively matched subsequent actual observations.

5. Significance and Claims

The paper claims that NavWAM successfully bridges the gap between visual foresight and executable control. Its primary significance lies in demonstrating that:

Visual Foresight need not be a separate planning module. By integrating future prediction, value estimation, and action generation into a single latent sequence, the model learns to use foresight directly for control.
Efficiency: High-performance navigation can be achieved without the computational cost of CEM-style trajectory optimization at test time.
Scalability: A model with a 2B-parameter backbone can compete with 7B-parameter direct policies while providing the additional interpretability of future-view predictions.

The authors conclude that future prediction is most effective for robot navigation when learned jointly with the action and value targets that determine closed-loop behavior, rather than as a separate planning step.

Limitations

The authors modestly note that:

Evaluation is currently limited to image-goal navigation in indoor environments; language or object-specified goals are future work.
The real-world study is limited in scale (24 episodes).
The optional value-guided best-of-N sampling mode was not used in main results, and its impact on robustness remains to be studied.