Technical Summary: Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models

Problem Statement

The increasing adoption of end-to-end (E2E) learning for autonomous driving has led to state-of-the-art performance on benchmarks like NAVSIM. However, this architectural unification of perception, prediction, and planning into unified transformer-based models results in increased complexity and opacity. Unlike modular pipelines, E2E models lack clear interfaces for module-level testing, making internal decision-making processes difficult to analyze. This opacity complicates the identification of failure modes, leading to expensive debugging cycles driven by "best guesses" rather than systematic analysis. Specifically, the field lacks generalizing solutions to holistically describe model behavior, uncover erroneous learning (such as spurious correlations), and offer comprehensive insights into failure modes without compromising the model's representational capacity.

Methodology

The authors propose a structured, multi-step framework that integrates unsupervised dictionary learning via Sparse Autoencoders (SAEs) as a post-hoc interpretability module into state-of-the-art E2E driving models (specifically GTRS and iPAD). The framework operates through the following stages:

1. Latent Space Selection: The SAE is injected into the latent space representation immediately preceding the scoring module. This location is chosen because it contains a dense summary of scene understanding and directly informs the trajectory scoring, allowing for a link between extracted features and model outputs.
2. Disentangling Feature Directions: An SAE is trained on the frozen model's latent representations to decompose entangled activations into sparse, monosemantic features (concepts). The training minimizes reconstruction loss while enforcing sparsity via top-$k$ activation functions. To prevent "dead neurons," a reanimation term is added to the loss function. The authors evaluate various SAE architectures (TopK, Matryoshka, Archetypal) and find that TopK variants with moderate latent sizes (e.g., 256 neurons) offer the best balance between reconstruction fidelity, feature disentanglement, and avoidance of scenario memorization.
3. Semantic Assignment: The authors assign human-understandable meaning to SAE neurons using three strategies:
   • Activation/Attribution Maximization: Identifying reference dataset samples that maximally activate a specific neuron or attribution score.
   • Neuron Clustering: Grouping neurons with similar activation behaviors to identify shared semantic concepts (e.g., "car ahead," "intersection").
   • Concept Relevance Propagation (CRP): Using LRP-based methods to trace relevance from the trajectory score back through the SAE to the input space (camera images and ego status), generating attribution maps that visualize which input features drive specific neuron activations.
4. Circuit Discovery: The framework employs circuit analysis to map causal relationships between SAE neurons and specific model outputs (PDM sub-scores). This involves a four-stage pipeline:
   • Attribution Patching: Estimating causal influence via gradient-weighted activation differences.
   • Activation Patching: Performing exact causal interventions by patching neurons to their dataset mean.
   • ACDC Pruning: Sparsifying the influence matrix to retain only significant edges.
   • Neuron Specificity: Quantifying how specific a neuron is to a particular prediction head using the Gini coefficient.
5. Targeted Intervention: Based on the identified causal circuits, the authors perform targeted interventions by zeroing (ablating) the activations of specific SAE neurons associated with erroneous behavior. This allows for the manipulation of driving decisions at inference time without retraining the model.

Key Contributions

The paper makes four primary contributions:

1. Structured Interpretability Framework: It presents the first structured approach for interpreting E2E driving decisions, providing concept-level explanations for the scoring of individual candidate trajectories.
2. Concept-Based Decomposition: It introduces a decomposition of latent representations that identifies the most influential factors in driving scenarios, enabling fine-grained attribution of model decisions.
3. Direct Correspondence: It establishes a direct link between learned concepts and predicted quality scores (PDM sub-scores), revealing which concepts influence specific behaviors like collision avoidance, traffic light compliance, and progress.
4. Targeted Adaptation: It demonstrates that identified concepts can be leveraged to manipulate model inference at test time, enabling improvements in downstream performance without the need for costly retraining or dataset curation.

Results

The authors evaluated their approach on the GTRS and iPAD models using the NAVSIM dataset:

• SAE Performance: TopK SAEs with 256 neurons achieved high reconstruction fidelity (Cosine Similarity: 0.9975, Explained Variance: 0.9951) and strong alignment with ego-status features (Ego Correlation: 0.8454), outperforming Archetypal SAEs which tended to overfit to scenario-specific correlations.
• Concept Discovery: The framework successfully identified semantically meaningful clusters, such as neurons corresponding to "cars ahead," "lane markings," and "intersections." Circuit analysis revealed that while "car ahead" neurons influence all scoring heads, their specific influence varies; for instance, they correlate with traffic light compliance due to data co-occurrence (cars stopping at lights) rather than causal reasoning.
• Intervention Impact: By ablating three specific neurons (177, 59, 71) identified as having negative or spurious influences, the authors achieved a measurable improvement in the Extended Predictive Driver Model Score (EPDMS).
  • The manipulated model improved the EPDMS from 0.4963 (with unmodified SAE) to 0.5926.
  • This score surpassed the original baseline model without the SAE (0.524).
  • Significant improvements were observed in drivable area compliance (+0.1298) and driving direction compliance (+0.0702).
  • The intervention resulted in more conservative driving behavior, trading rapid progress for increased safety, as evidenced by a decrease in the ego progress score (-0.1000).

Significance and Claims

The paper claims that interpretability is not merely an analytical tool but a mechanism for targeted mitigation and performance enhancement. By decomposing the opaque latent space into semantically meaningful concepts, the authors demonstrate that:

• Opacity Reduction: Interpretability can effectively uncover erroneous behaviors and spurious correlations (e.g., associating a car ahead with traffic light compliance) that standard E2E models learn.
• Causal Understanding: The framework distinguishes between correlational artifacts and functionally relevant components, allowing for the isolation of specific decision-making logic.
• Performance Boost: Targeted interventions at the concept level can correct model behavior and improve overall driving metrics without retraining. The authors argue that this approach offers a pathway to more transparent, trustworthy, and controllable autonomous driving systems, addressing the critical challenge of model validation and error analysis in the field.

The authors remain modest regarding the scope, acknowledging that while the framework provides a foundation for concept-based analysis, fully understanding the semantic concepts and their complex interactions in the decision-making process remains an open research problem.