MTA-RL: Robust Urban Driving via Multi-modal Transformer-based 3D Affordances and Reinforcement Learning

Technical Summary: MTA-RL

Problem Statement
Robust urban autonomous driving requires reliable 3D scene understanding and stable decision-making under dense, dynamic interactions. Existing approaches face a dichotomy: modular pipelines offer interpretability but suffer from error propagation across brittle interfaces, while end-to-end learning models often lack interpretability and struggle with sample efficiency when processing high-dimensional raw observations. Furthermore, existing affordance learning methods typically rely on monocular camera inputs, restricting representations to the 2D image plane and limiting geometric expressiveness, which leads to unstable policies in complex traffic. Additionally, pure end-to-end Reinforcement Learning (RL) often lacks explicit spatial reasoning, making failure analysis difficult and safety guarantees in novel environments hard to ensure.

Methodology
The paper proposes MTA-RL, a unified framework that bridges perception and control by coupling Multi-modal Transformer-based 3D Affordances with Reinforcement Learning. The architecture consists of two primary stages:

1. Transformer-Based Multimodal Affordance Prediction:

   • Input Fusion: The system fuses heterogeneous sensor inputs: three forward-facing RGB cameras (providing semantic cues like traffic lights and pedestrians) and LiDAR point clouds (providing precise geometric measurements).
   • Architecture: A transformer-based fusion network processes these inputs in a Bird's-Eye-View (BEV) space. RGB images are processed via a ResNet-34 backbone, while LiDAR points are projected onto a 32m × 32m BEV grid.
   • Output: The network predicts structured, geometry-aware 3D affordances. These are categorized into:
     • Discrete Affordances: Categorical predictions for traffic rules and safety (e.g., red traffic light, stop sign, hazard stop).
     • Continuous Affordances: Regression predictions for geometric relationships (e.g., lateral distance, relative angle, distances to leading vehicles, traffic lights, stop signs, and pedestrians).
   • These affordances form a compact, semantically structured latent vector that serves as the observation space.

2. Perception-Guided Reinforcement Learning (SAC):

   • Policy: A Soft Actor-Critic (SAC) agent operates purely on the predicted affordances and ego-vehicle state, rather than raw sensor data.
   • Action Space: Continuous control outputs for steering and longitudinal commands (throttle/braking).
   • Reward Shaping: The reward function is densely shaped to optimize multiple objectives: lane keeping, speed tracking, control smoothness, route progress, and crucially, traffic rule compliance. The rule compliance term explicitly penalizes late braking near constraints (lights, signs) and rewards full stops, ensuring strict adherence to safety boundaries.
   • Termination: Episodes terminate on safety violations (collisions, rule breaches) or lack of progress, with specific penalties defined for these events.

Key Contributions

• 3D BEV Affordance Prediction: The authors propose the first transformer-based multi-modal architecture with explicit distance-to-constraint heads to predict compact, geometry-aware affordances directly in BEV space, overcoming the 2D limitations of camera-only methods.
• RL on Structured Affordances: The framework enables RL policy learning to operate purely on predicted driving semantics. This decoupling provides a structured observation space that improves sample efficiency, stability, and interpretability compared to raw sensory inputs.
• Robust Performance: Extensive evaluations demonstrate that MTA-RL consistently outperforms state-of-the-art baselines in both affordance prediction accuracy and driving performance across diverse urban towns and traffic densities.

Experimental Results
Evaluations were conducted in the CARLA simulator across Town01, Town02, and Town03 with varying traffic densities (20–60 background vehicles).

• Affordance Prediction: MTA-RL achieved superior accuracy compared to baselines (DeepDriving, CAL, Affordance-RL). It attained the highest Intersection-over-Union (IoU) for categorical tasks (e.g., 99.13% for red traffic lights) and the lowest Mean Absolute Error (MAE) for continuous distance predictions (e.g., 0.011m for target vehicle distance). It was the only method to reliably estimate distances to traffic lights, stop signs, and pedestrians.
• Driving Performance:
  • Generalization: Trained solely on Town03, the model demonstrated strong zero-shot generalization to Town01 and Town02.
  • Metrics: In dense traffic (60 vehicles), MTA-RL achieved the highest Route Completion (RC) and Total Distance (TD) across all towns. For instance, in Town01, it achieved an RC of 0.764 compared to 0.630 for VLM-RL and 0.599 for Roach.
  • Safety: The method showed significant improvements in safety metrics, achieving an 83.7% improvement in Distance Per Violation (DPV) compared to baselines in certain scenarios. It maintained low collision speeds and high DPV, indicating fewer and less severe infractions.
• Ablation Studies:
  • Multi-modal Fusion: Replacing the transformer with geometric fusion or removing explicit LiDAR geometry (Latent TransFuser) resulted in degraded affordance prediction, confirming the necessity of attention-based multi-modal fusion and explicit geometric inputs.
  • Reward Shaping: Removing dense traffic-rule shaping and relying only on sparse terminal penalties led to slower convergence and lower RC, TD, and DPV, highlighting the critical role of intermediate feedback for policy refinement.

Significance
The paper claims that MTA-RL provides a principled balance between end-to-end learning and modular design for urban autonomous driving. By using multi-modal 3D affordances as an intermediate representation, the framework successfully decouples perception from control. This approach enhances interpretability and robustness while allowing the RL policy to leverage explicit spatial reasoning and traffic rules. The results suggest that this structured representation is essential for achieving stable, safe, and efficient decision-making in highly interactive and uncertain urban environments, addressing the limitations of both traditional modular pipelines and raw end-to-end RL.