Unveiling the Surprising Efficacy of Navigation Understanding in End-to-End Autonomous Driving

1. Problem Statement

The paper identifies a critical flaw in current end-to-end (E2E) autonomous driving systems: the failure to effectively utilize global navigation information.

• Over-reliance on Local Perception: Experimental results show that many state-of-the-art E2E models (e.g., Transfuser) rely heavily on local scene understanding. When global navigation inputs are removed, corrupted, or replaced with random commands, planning performance often remains unchanged or even improves.
• Limitations of Current Representations: Existing systems typically use discrete driving commands (e.g., "Turn Left," "Go Forward") derived from expert trajectories. The authors argue this representation is insufficient because:
  • Ambiguity: In complex scenarios like roundabouts, a "Go Forward" command might be mislabeled as "Turn Left" due to lateral displacement, confusing the model.
  • Oversimplification: In Beyond Visual Range (BVR) scenarios (e.g., lane changes needed for a distant turn), a simple "Go Forward" command fails to convey the necessary intent, leading to causal confusion when the model encounters lane-changing behaviors in the data.
• The Core Question: Do current E2E systems truly understand navigation, or are they simply overfitting to specific input channels? The authors conclude they do not.

2. Methodology

To address these limitations, the authors propose a new paradigm called Sequential Navigation Guidance (SNG) and a corresponding framework SNG-VLA.

A. Sequential Navigation Guidance (SNG)

Instead of discrete commands, SNG integrates two complementary components to represent global navigation:

1. Navigation Path: A static, predefined trajectory segment extracted from the global route (e.g., road centerlines within a 40m range). This serves as a long-term reference line for trajectory planning.
2. Turn-by-Turn (TBT) Information: Dynamic, high-level textual guidance including:
   • Current driving actions with distance/time estimations.
   • Future actions.
   • Supplementary actions (e.g., "enter highway," "use right-turn lane") to disambiguate complex maneuvers.
   • Note: This mimics real-world navigation APIs (like Google Maps).

B. SNG-QA Dataset

To align local planning with global planning, the authors constructed the SNG-QA dataset (100k QA pairs).

• Structure: It decomposes reasoning into three hierarchical stages:
  1. Global Planning: Summarizing the SNG input.
  2. Local Planning: Integrating global navigation with local scene perception (e.g., traffic, pedestrians) to generate causal explanations.
  3. Trajectory Planning: Generating the actual waypoints.
• Annotation: Built using Qwen2.5 VL 72B with a three-stage validation process (accuracy, consistency, and refinement) to ensure high-quality reasoning data.

C. SNG-VLA Model Architecture

The proposed model is a Vision-Language-Action (VLA) system based on a unified Transformer backbone:

• Backbone: Uses LLaVA architecture with Qwen2.5 (LLM) and SigLIP (Vision Encoder).
• Multimodal Fusion:
  • Text: Encodes TBT information.
  • Path: Encodes the Navigation Path via MLP.
  • Vision: Encodes front/rear camera images.
  • Ego State: Encodes vehicle state (velocity, acceleration) using an Attention-based State Dropout Encoder (SDE) to prevent overfitting.
• Generation: The model autoregressively generates textual reasoning followed by trajectory waypoints. It uses a cross-attention mechanism to interact navigation queries with the hidden states.

3. Key Contributions

1. Novel Navigation Representation: Introduced Sequential Navigation Guidance (SNG), which combines long-term trajectory constraints (Navigation Path) with real-time decision logic (TBT), overcoming the ambiguity of discrete driving commands.
2. SNG-QA Dataset: Created a large-scale dataset that forces models to reason about the alignment between global navigation and local perception, enhancing planning rationality.
3. State-of-the-Art Model (SNG-VLA): Developed an efficient E2E model that achieves SOTA performance without auxiliary perception loss functions or complex multi-task supervision, relying purely on precise navigation modeling.

4. Experimental Results

The method was evaluated on NAVSIM (real-world data) and Bench2Drive (CARLA closed-loop).

• Navigation Robustness (NAVSIM):
  • Command Ablation: When Transfuser's driving commands were randomized or removed, performance barely changed (PDMS score ~84.4 vs 84.0), proving it ignores navigation.
  • SNG Impact: Replacing commands with SNG significantly boosted Transfuser's performance (PDMS +1.5).
  • SNG-VLA: Achieved the highest PDMS score (88.24) on NAVSIM, outperforming Hydra-MDP and DiffusionDrive without using perception auxiliary losses.
• Closed-Loop Performance (Bench2Drive):
  • Driving Score: SNG-VLA achieved 35.90, surpassing UniAD-Base (16.36) by 119.4% in Success Rate and significantly improving the Driving Score.
  • Multi-Ability: Outperformed VAD by 110.7% in mean multi-ability scores (merging, overtaking, emergency braking, etc.).
  • Efficiency: Achieved favorable latency (159.6ms) using a single front-view camera and a smaller backbone.
• Ablation Studies:
  • Confirmed that Navigation Path + TBT yields the best results.
  • Found that sparse navigation paths (4 points, 10m spacing) combined with TBT are optimal; overly dense paths hinder obstacle avoidance.
  • Demonstrated that TBT information alone outperforms discrete commands, but the combination is superior.

5. Significance

• Paradigm Shift: The paper challenges the prevailing assumption that E2E systems implicitly learn navigation. It proves that without explicit, structured navigation guidance, models fail to generalize to complex, long-horizon tasks.
• Practical Deployment: The SNG approach relies on off-the-shelf navigation APIs (Path + TBT), making it highly deployable in real-world scenarios without requiring complex map reconstruction or privileged information.
• Reasoning Capability: By integrating VQA-style reasoning, the model not only plans trajectories but also explains why it is making a decision based on global goals, improving interpretability and safety.
• Efficiency: The approach achieves top-tier performance without the computational overhead of auxiliary perception tasks, offering a "plug-and-play" solution for enhancing existing E2E architectures.