💻 computer science

Addressing the Waypoint-Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models

This paper proposes a differentiable vehicle-motion framework that bridges the gap between waypoint-based and action-based end-to-end autonomous driving by enabling action-based policies to be trained and evaluated within standard waypoint-based benchmarks, achieving state-of-the-art performance on NAVSIM.

Original authors: Jorge Daniel Rodríguez-Vidal, Gabriel Villalonga, Diego Porres, Antonio M. López Peña

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jorge Daniel Rodríguez-Vidal, Gabriel Villalonga, Diego Porres, Antonio M. López Peña

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. There are two main ways to tell the robot what to do:

The "Steering Wheel" Method (Action-Based): You tell the robot, "Turn the wheel 5 degrees left, press the gas pedal a little, and don't brake." This is like giving a pilot specific, low-level commands.
The "Map Route" Method (Waypoint-Based): You tell the robot, "Drive to this specific point on the map, then to that one, then to the next." This is like giving a GPS a list of destinations to hit.

The Problem: The Language Barrier
For a long time, the "Map Route" method became the gold standard. Most tests, competitions, and training programs for self-driving cars are designed to check if the robot hits those specific map points (waypoints).

However, many researchers prefer the "Steering Wheel" method because it's more flexible and doesn't need perfect GPS maps (great for rural areas). The problem? You can't easily test a "Steering Wheel" robot in a "Map Route" competition. It's like trying to enter a chess tournament when you only know how to play checkers; the rules don't match your moves. This created a "gap" where good steering-wheel robots couldn't prove they were good because the tests were built for map-route robots.

The Solution: The "Universal Translator"
The authors of this paper built a clever "translator" or a bridge to fix this.

They created a special, math-based tool (called a differentiable vehicle model) that acts like a simulator inside the training process. Here is how it works:

The Input: The robot gives its "Steering Wheel" commands (turn, gas, brake).
The Simulation: The tool instantly calculates: "If the car does exactly what you said, where will it be in 1 second? Where will it be in 2 seconds?" It turns those low-level commands into a list of map points (waypoints).
The Check: The system then checks if those calculated map points match the "correct" path.
The Learning: If the robot went off course, the system sends a "correction signal" back through the simulator to the robot, teaching it how to adjust its steering and gas to hit the target points.

Why is this cool?

It's a Fair Play Field: Now, a robot that thinks in "Steering Wheel" commands can compete in "Map Route" tournaments without changing the rules of the game.
It's Flexible: The authors tested two types of "translators":
- The Bicycle Model (KBM): Imagine the car is a bicycle. It's a simple, fast way to guess where the car goes.
- The Smooth Curve Model (CCPP): Imagine the car is drawing a perfect, smooth ribbon on the ground. This is more complex but handles sharp turns and curves much better, like a professional race car driver.
Better Results: When they used this bridge, the "Steering Wheel" robots performed just as well as, or even better than, the "Map Route" robots in tough driving tests.

The Takeaway
The paper shows that you don't have to choose between teaching a robot to drive by steering or by following a map. You can teach it to steer, and use this "math bridge" to pretend it's following a map for the purpose of testing and grading. This allows the best driving styles to be compared fairly, leading to safer and smarter self-driving cars.

What they didn't claim:

They did not claim this solves all self-driving problems (like 3D hills or suspension bounces).
They did not claim this is a medical tool or a specific future application; it is strictly a method to improve how we train and test driving software right now.

Technical Summary: Addressing the Waypoint–Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models

1. Problem Statement

End-to-End Autonomous Driving (E2E-AD) systems are currently bifurcated into two paradigms based on their output: waypoint-based models, which predict future trajectories, and action-based models, which directly output low-level controls (throttle, steering, brake). While action-based policies offer advantages such as reduced reliance on high-precision GPS and better suitability for rural areas with poor connectivity, the research community has increasingly favored waypoint-based approaches.

This shift has created a "Waypoint–Action gap." Most modern benchmarks (e.g., NAVSIM, Bench2Drive) and training pipelines are designed exclusively for waypoint outputs. Consequently, action-based policies cannot be directly trained or evaluated within these frameworks without introducing non-standard interfaces (e.g., discretizing control spaces). This imbalance hinders the fair comparison and progress of action-based methods. The paper aims to bridge this gap by enabling action-based architectures to be trained and evaluated within existing waypoint-based benchmarks without modifying the evaluation protocols.

2. Methodology

The authors propose a differentiable vehicle-model framework that acts as an analytic bridge between action and waypoint representations. The core mechanism is a lifting operator ( $F_\phi$ ) that rolls out predicted action sequences into corresponding ego-frame waypoint trajectories. This allows the system to supervise action-based policies using waypoint-space loss functions.

Key Components:

The Lifting Operator ( $F_\phi$ ): This operator takes a sequence of low-level actions ( $a_t$ $a_{t}$ ) and the initial vehicle state ( $s_t$ $s_{t}$ ) to produce a sequence of waypoints ( $w_t$ $w_{t}$ ). It is decomposed into three modular, differentiable components:
1. Control Activation ( $\psi_\phi$ ): Maps raw network outputs to bounded physical controls (e.g., using sigmoid/tanh for throttle and steering).
2. Dynamics Rollout ( $f_\phi$ ): Propagates the kinematic state forward in time using vehicle dynamics.
3. Pose Projection ( $h$ ): Extracts the planar position $(x, y)$ from the full state vector.
Model Instantiations: The framework is instantiated with two specific vehicle models:
- Kinematic Bicycle Model (KBM): A single-track model capturing nonholonomic constraints, integrated via Euler or Runge-Kutta (RK4) methods.
- Continuous Curvature Path Planner (CCPP): A model generating smooth trajectories using clothoid arcs to ensure curvature continuity, avoiding the steering discontinuities of straight-circular paths.
- MLP Baseline: A Multi-Layer Perceptron is also used as a data-driven lifting operator to compare against physics-based models.
Training Pipeline: The policy network ( $N_\theta$ ) predicts actions from observations and commands. These actions pass through the differentiable lifting operator to generate predicted waypoints. A weighted L1 loss is computed between the predicted waypoints and ground-truth waypoints. Gradients flow back through the lifting operator to update the policy parameters, while the vehicle model parameters remain fixed.

3. Key Contributions

The paper outlines three primary contributions:

Waypoint-based Training Objective for Action Policies: The authors demonstrate that coupling action policies to their framework and supervising them with a waypoint-space loss results in more stable evaluation, higher closed-loop performance, and a stronger correlation with driving outcomes compared to standard action-space objectives.
Differentiable and Deterministic Framework: This work introduces, for the first time, a deterministic and differentiable vehicle-dynamics framework specifically designed to lift low-level actions to waypoints, enabling their use in standard benchmarks.
Unified Comparison Across Paradigms: By instantiating the framework with KBM and CCPP, the authors enable action-based and waypoint-based policies to be trained and evaluated under the same waypoint protocol without altering existing benchmark interfaces.

4. Experimental Results

The framework was evaluated across four challenging benchmarks: NAVSIM navtest, NAVSIM navhard, Bench2Drive, and a CARLA-based protocol.

NAVSIM navhard (Reactive): The approach achieved state-of-the-art performance among vision-only models. Specifically, MILE coupled with the CCPP lifting operator achieved an Extended Predictive Driver Model Score (EPDMS) of 29.5, outperforming the strongest vision-only baseline (Latent TransFuser).
NAVSIM navtest (Non-reactive): The framework allowed action-based policies to match or closely approach waypoint-based baselines. CIL++ with KBM achieved a Planning Discrepancy Metric Score (PDMS) of 83.3, within 1.5% of the state-of-the-art waypoint-only model (LAW).
Bench2Drive (Closed-loop): The framework significantly improved closed-loop performance. For the CIL++ baseline, the Driving Score (DS) improved by up to 61.1% (from 43.6 to 66.9 with CCPP), and Route Completion (RC) improved by over 30%.
Training Stability: The use of RK4 integration (particularly with CCPP) significantly reduced the variance in training dynamics compared to Euler integration, leading to more robust model selection.
Correlation Study: The paper found that waypoint-level errors generated by the framework (KBM/CCPP) correlate more strongly with closed-loop Success Rates than raw action-space errors, suggesting that trajectory-level supervision is a better proxy for real-world performance.

5. Significance and Claims

The paper claims that its framework effectively bridges the methodological divide between action-based and waypoint-based autonomous driving. By allowing action-based policies to be trained and evaluated within the dominant waypoint-centric benchmark ecosystem, the work removes a significant barrier to the development and comparison of control-based approaches.

The authors emphasize that their method does not require modifying existing benchmark protocols, making it a general mechanism for adapting action-based policies. They note that while the framework assumes planar motion ( $R^2$ ) and neglects 3D effects like banking or suspension, it successfully leverages physical vehicle models to improve the stability and performance of end-to-end learning. The work suggests that incorporating kinematic structure via differentiable lifting is beneficial, particularly in reactive environments where errors compound over time.