NavThinker: Action-Conditioned World Models for Coupled Prediction and Planning in Social Navigation

1. Problem Statement

Social Navigation requires robots to navigate safely through dynamic crowds while respecting human comfort and social norms. The core challenge is the coupled prediction-planning problem:

• Mutual Influence: Robot actions influence pedestrian behavior, and pedestrian movements influence the robot's feasible paths.
• Partial Observability: Robots often have limited fields of view and cannot see the full scene or future human states.
• Limitations of Existing Methods:
  • Reactive approaches (e.g., social force models) are myopic and fail in dense crowds.
  • Decoupled approaches treat pedestrian forecasts as fixed inputs independent of robot actions, missing the feedback loop.
  • Optimization-based methods are computationally expensive and scale poorly with many agents.
  • Standard World Models often compress spatial information into global vectors, losing fine-grained geometry, or rely on pure imagination which accumulates errors.

The goal is to develop a framework that enables a robot to "think ahead" by reasoning about how the scene and pedestrians will evolve under different candidate actions before executing them.

2. Methodology: NavThinker

NavThinker is a future-aware framework that couples an action-conditioned world model with on-policy reinforcement learning (RL). It operates in two main stages:

A. Scene-and-Interaction World Model

The world model learns the dynamics of the environment in a latent feature space to predict future states conditioned on robot actions.

• Feature Space: Instead of raw pixels or global vectors, the model operates in the Depth Anything V2 (DA-V2) patch feature space. A frozen ViT encoder extracts patch embeddings from depth images, preserving spatial structure.
• Architecture:
  • Observation Encoder: Frozen DA-V2 ViT encodes current depth observations into patch embeddings ($z_t$).
  • Transition Model: A Causal Attention Transformer predicts future latent features ($\hat{z}_{t+1}$) autoregressively. It takes the sequence of past patches and the current action token as input.
  • Multi-Head Decoders:
    1. Depth Decoder: Reconstructs future depth maps to ensure the model understands scene geometry and traversability.
    2. Trajectory Decoder: Predicts future human trajectories ($\hat{\xi}$) to capture interaction dynamics.
    3. Reward Decoder: Predicts scalar rewards.
• Training Objective: The model is trained with a latent consistency loss (aligning predicted features with ground truth encoder outputs) and task-specific losses (depth reconstruction and trajectory prediction). This ensures the "imagination" is faithful to both geometry and human motion.

B. Imagination-Augmented Policy Learning

The policy is trained using DD-PPO (Decentralized Distributed Proximal Policy Optimization) but is enhanced with "think-ahead" mechanisms derived from the world model.

• Imagination Module: Before selecting an action, the policy queries the world model to generate action-conditioned future features for all candidate actions (Forward, Left, Right, Stop).
• Two "Think-Ahead" Mechanisms:
  1. Action-Conditioned Future-Feature Fusion: The predicted future latent features are fused with the current observation embedding. This allows the policy to compare the "imagined" outcomes of different actions before acting.
  2. Social Reward Shaping: The predicted human trajectories are used to compute a predictive social cost. If a candidate action leads to a predicted collision or social violation in the future, the reward is penalized. This couples prediction directly into the decision-making loop.
• Training Loop: The policy is trained on real interactions (on-policy) but utilizes the world model to anticipate consequences, creating a closed-loop "imagine-then-act" system.

3. Key Contributions

1. NavThinker Framework: A novel architecture that couples an action-conditioned world model with policy optimization, enabling true coupled prediction and planning.
2. Task-Aligned World Modeling: Operating in the DA-V2 patch feature space with dedicated depth and trajectory decoders. This preserves spatial fidelity (crucial for navigation) and explicitly models human dynamics, unlike standard latent world models.
3. Dual Think-Ahead Mechanisms:
   • Injecting future features into the observation embedding for lookahead context.
   • Using predicted trajectories for social reward shaping to penalize unsafe future states.
4. Empirical Validation: Demonstrated state-of-the-art performance on single- and multi-robot benchmarks, zero-shot generalization to new environments, and successful real-world deployment.

4. Experimental Results

The method was evaluated on Social-HM3D (single and multi-robot) and Social-MP3D (zero-shot transfer).

• Single-Robot Performance (Social-HM3D):
  • NavThinker achieved a Success Rate (SR) of 59.46% and SPL of 55.00, outperforming the previous state-of-the-art (Falcon) by 3.2% in SR and 2.95% in SPL.
  • It reduced human collisions (H-Coll) from 41.22% to 39.09%.
• Multi-Robot Performance:
  • In a complex setting with multiple independent robots, NavThinker consistently outperformed all baselines (A*, ORCA, Falcon) in both per-robot and team-level metrics (Team SR and Team SPL).
• Zero-Shot Generalization:
  • Trained on Social-HM3D, the model achieved competitive results on Social-MP3D without fine-tuning, proving robustness to different scene layouts.
• Real-World Deployment:
  • Successfully deployed on a Unitree Go2 quadruped robot in real-world environments, navigating around groups of people while maintaining safe distances.
• Ablation Studies:
  • Removing the "lookahead imagination" mechanism dropped SR from 59.46% to 55.87%.
  • Removing "trajectory-based reward shaping" further dropped SR to 43.36%.
  • Adding both decoders (depth and trajectory) to the world model significantly improved latent consistency and prediction fidelity.

5. Significance

NavThinker addresses a critical gap in social robotics: the inability of current systems to effectively reason about the consequences of their own actions on human behavior.

• Paradigm Shift: It moves beyond reactive navigation or decoupled prediction-planning toward a unified "imagination" loop where the robot simulates the future to make safer, more socially compliant decisions.
• Practical Applicability: By leveraging a frozen foundation model (DA-V2) and on-policy RL, it achieves high performance without the computational overhead of complex optimization or the instability of pure latent imagination.
• Scalability: The framework scales to multi-robot scenarios and generalizes to unseen environments, making it a strong candidate for real-world deployment in crowded human spaces.