From Correlation to Causation in Lane Change Prediction for Automated Driving: A Causal Explanation Framework

Technical Summary: From Correlation to Causation in Lane Change Prediction for Automated Driving

Problem Statement
Lane-change prediction is a critical task for automated driving, enabling safer and smoother decision-making by anticipating surrounding vehicles' maneuvers. While existing approaches utilizing machine learning and deep learning have achieved high accuracy, they predominantly rely on statistical correlations between input variables (e.g., speed, gap, Time-To-Collision) and maneuver classes. A significant limitation of these correlation-driven models is their failure to explicitly model the physical and causal dependencies among the input variables themselves. In real-world driving, variables such as longitudinal gap, relative velocity, and Time-To-Collision (TTC) are not independent; they are physically linked (e.g., TTC is derived from gap and relative velocity). Consequently, standard models may predict maneuvers correctly but fail to distinguish whether a variable is a direct contributor, an upstream cause, a mediator, or merely a correlated feature. This lack of causal structure limits interpretability and prevents the system from understanding why a specific maneuver is favored over alternatives.

Methodology
The authors propose a causal-inference-based framework that moves beyond flat classification to mechanism-aware reasoning. The methodology consists of four primary stages:

1. Linguistic Feature Construction: Numerical driving data from the highD dataset (German highway trajectories) is converted into interpretable linguistic categories (e.g., "critical," "safe," "fast," "slow") using threshold-based discretization. This transforms continuous variables into 47 categorical features covering kinematics, lane context, surrounding vehicle occupancy, interaction measures, and traffic density.
2. Expert-Constrained Causal Discovery: A constraint matrix is constructed using expert driving knowledge and LLM-assisted review (via PyWhy-LLM) to define the search space for causal directions. This matrix blocks physically implausible causal relationships (e.g., preventing the future maneuver from causing current traffic density or TTC) while leaving plausible directions available for data-driven learning.
3. Deep Structural Causal Modeling (SCM): The framework employs Deep End-to-end Causal Inference (DECI) to learn a Structural Causal Model (SCM) represented as a Directed Acyclic Graph (DAG) from the observational data under the expert constraints. DECI jointly learns the causal graph structure and the structural mechanisms (non-linear functions) connecting variables, incorporating noise terms to handle uncertainty in categorical data.
4. Causal Analysis and Explanation:
   • Intervention-Based Effect Analysis: Using the do-calculus, the framework estimates the Average Treatment Effect (ATE) of changing specific feature states (e.g., changing TTC from "safe" to "critical") on maneuver probabilities. This distinguishes influential variables from weakly influential ones.
   • Refutation Testing: The DoWhy library is used to validate the robustness of estimated effects through placebo treatments, random common causes, and data-subset tests.
   • Contrastive Causal-Chain Explanation: The learned SCM is recursively traced to generate explanations. The system identifies direct contributors to the predicted maneuver, traces upstream causes and mediators, and explains why alternative maneuvers are less supported by identifying observed states that reduce their probability.

Key Contributions

• Mechanism-Aware Prediction Pipeline: The paper introduces a framework that integrates expert-constrained causal discovery with deep learning to model lane-change prediction as a causal chain rather than a flat correlation task.
• Expert-Constrained Learning: It proposes a method to incorporate domain knowledge via a constraint matrix that prevents the learning of physically impossible causal directions without forcing a fully manual graph, allowing the model to remain data-adaptive.
• Intervention and Robustness Analysis: The approach utilizes intervention analysis to quantify feature influence and employs refutation tests to ensure the stability of these causal estimates, moving beyond simple feature importance scores.
• Contrastive Explanations: The framework generates recursive causal-chain explanations that clarify not only why a maneuver is predicted but also why alternative maneuvers are suppressed, distinguishing between direct contributors, mediators, and confounders.

Results

• Prediction Performance: The DECI-based SCM achieves macro F1-scores above 95% (specifically 96.3%, 96.0%, and 95.4%) in the three seconds immediately preceding the lane-marking crossing event. It maintains predictive capability (F1 > 59%) up to 7–8 seconds before the event, demonstrating effective early anticipation.
• Causal Structure Validity: The learned graph respects temporal and physical constraints (e.g., maneuver nodes have no outgoing edges). It correctly identifies that contextual variables (e.g., rush hour) influence maneuvers indirectly through mediators like traffic density and gap, while interaction variables (e.g., relative velocity, TTC) act as direct contributors.
• Intervention Analysis: Lateral motion variables (velocity, acceleration) and interaction variables (relative velocity, TTC, gap) produced the strongest intervention effects on maneuver probabilities. Contextual variables showed smaller direct effects, consistent with their role as upstream causes.
• Robustness: Refutation tests (placebo, random common cause, data subset) confirmed that the estimated intervention effects for key variables (e.g., lateral velocity, TTC) remained stable and close to zero for placebo treatments, indicating the effects are not spurious.
• Diagnostic Validation: An ablation study showed that removing variables estimated to have weak intervention effects (lateral velocity and acceleration in a specific pre-lateral-motion setting) resulted in negligible degradation of prediction performance, validating the utility of the intervention analysis for feature selection.

Significance and Claims
The authors claim that this work provides a significant step toward interpretable autonomous driving systems by shifting from correlation-based classification to causal reasoning. The framework does not claim to discover definitive real-world causality but aims to provide a "mechanism-aware prediction framework" that can distinguish direct contributors, mediating variables, and upstream causes under a learned causal structure. The significance lies in the ability to generate contrastive causal-chain explanations that clarify the physical logic behind a prediction (e.g., explaining a left lane change as a result of a speed differential leading to a critical TTC) and to identify which variables are truly influential versus those that are merely correlated. This approach enhances trust and safety by ensuring that predictions are grounded in a structured understanding of driving dynamics rather than statistical coincidences.