Technical Summary: DriveSafer

Problem Statement

While End-to-End (E2E) autonomous driving models have demonstrated growing capabilities on complex benchmarks, they remain prone to a significant number of catastrophic failures in safety-critical scenarios. The authors observe that many of these failures stem from violations of physical constraints and safety requirements, leading to unsafe behaviors such as collisions, driving off permitted regions, or deviating from intended directions.

Current evaluation metrics, such as L2 distance error from ground truth trajectories, often mask these critical failures. A model can achieve state-of-the-art performance on trajectory error metrics while still exhibiting dangerous "blind" behaviors. The paper argues that average planning quality improvements are insufficient; the primary challenge is the "long tail" of catastrophic failures (defined as a Predictive Driver Model Score, or PDMS, of 0). These failures include:

1. Collisions with moving objects (vehicles or pedestrians).
2. Driving off the permitted drivable region (e.g., onto sidewalks).
3. Significant deviations from the intended driving direction.

Methodology

The authors propose DriveSafer, a failure-aware safety framework designed to explicitly steer generative E2E planners toward safe behaviors. The framework operates on a base model, DiffusionDrive, and introduces safety enforcement through two synergistic phases:

1. Safety-Aware Training

DriveSafer modifies the training objective by incorporating specialized, physics-informed, and safety-aware loss functions alongside the base loss ($L_{base}$). The total loss is defined as:
$$L_{total} = L_{base} + \lambda_{DAC}L_{DAC} + \lambda_{Col}L_{Col} + \lambda_{Comf}L_{Comf}$$

• Drivable Area Compliance ($L_{DAC}$): Penalizes predicted trajectories that leave the drivable area mask derived from the underlying map.
• Collision Avoidance ($L_{Col}$): Penalizes trajectories that come within a specific distance threshold ($d_{col}$) of surrounding agents.
• Comfort ($L_{Comf}$): Reduces sudden motion changes (jerk) to ensure passenger comfort.

To address the baseline model's inability to forecast future agent locations (which limits collision awareness during inference), the authors introduce a lightweight adapter layer. This layer predicts the motion of other road agents for future timesteps, enabling joint prediction and planning.

2. Inference-Time Safety Guidance

Recognizing the stochastic nature of diffusion models and potential sensor noise, DriveSafer employs a guidance strategy during inference to reject infeasible trajectories:

• Alternative Hypothesis Generation: The system generates alternate trajectories based on the top predicted mode from the diffusion model. These alternatives include:
  • Variations in heading and speed (5% variation).
  • Sub-meter lateral trajectory shifts.
  • Utilization of the second- and third-highest ranked modes from the diffusion model.
• Safety Scoring and Selection: Using predicted agent positions and the underlying map, these alternative trajectories are scored using safety metrics. The safest trajectory is selected as the final output, effectively "nudging" the vehicle away from unsafe behaviors.

Key Contributions

The paper outlines four primary contributions:

1. DriveSafer Framework: A failure-centric safety enforcement framework for E2E planning that targets catastrophic failures through synergistic training- and inference-time safety enforcement.
2. Safety-Aware Training Strategy: The introduction of specialized losses (drivable area, collision, comfort) and an integrated, lightweight forecasting module for joint prediction and planning.
3. Inference-Time Guidance: A method that generates and evaluates alternative trajectories to steer the planner toward safety-compliant outputs.
4. Performance Improvement: A demonstrated reduction in catastrophic failures on the NAVSIM benchmark compared to state-of-the-art baselines.

Experimental Results

The framework was evaluated on the NAVSIM benchmark, specifically the navtest split, which features dense urban traffic and complex scenarios.

• Catastrophic Failure Reduction: DriveSafer reduced the number of catastrophic failures (PDMS = 0) by 48% compared to the DiffusionDrive baseline (reducing cases from 653 to 340).
• Drivable Area Compliance: There was a >65% reduction in drivable-area compliance failures (from 448 cases to 154).
• Overall PDMS Score: The model achieved a PDMS score of 90.3, an improvement of 2.2 points over the DiffusionDrive baseline (88.1).
• Sub-metrics: Notable improvements were observed in Drivable Area Compliance (DAC) and Ego Progress (EP) sub-metrics.
• Qualitative Analysis: Visual examples demonstrate DriveSafer correcting scenarios where the baseline model drove off-road, violated intended directions, or collided with pedestrians and oncoming vehicles.

Significance and Claims

The paper posits that the practicality of E2E autonomous driving is currently limited by a significant number of catastrophic failures, regardless of average performance metrics. The significance of DriveSafer lies in its failure-centric view, prioritizing the mitigation of life-threatening scenarios over marginal improvements in average trajectory error.

The authors claim that explicit safety guidance in E2E systems should be geared toward the direct reduction of catastrophic failure cases. By combining training-time constraints with inference-time guidance, DriveSafer offers a pathway to safer real-world deployment. The authors state their future goal is to extend DriveSafer for real-world deployment on autonomous vehicles (AVs) to assess its impact in real-world conditions, with the ultimate aim of reducing catastrophic failures to zero.