💻 computer science

$AutoDrive\text{-}P^3$ : Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuning

This paper introduces **AutoDrive-P³**, a novel framework that unifies perception, prediction, and planning into a single coherent chain-of-thought process via a specialized dataset and hierarchical reinforcement learning, achieving state-of-the-art performance in autonomous driving by ensuring synergistic decision-making and balancing inference efficiency with dual thinking modes.

Original authors: Yuqi Ye, Zijian Zhang, Junhong Lin, Shangkun Sun, Changhao Peng, Wei Gao

Published 2026-03-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuqi Ye, Zijian Zhang, Junhong Lin, Shangkun Sun, Changhao Peng, Wei Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. In the past, we tried to teach the robot in two ways, and both had problems:

The "Gut Feeling" Driver: Some robots were told, "Look at the road, and just tell me where to steer." They skipped the thinking part. It was like asking a human to drive while blindfolded, guessing where the other cars are. They often made dangerous mistakes because they didn't "see" the danger first.
The "Siloed" Driver: Other robots were smart enough to say, "I see a truck," then "I think the truck will move left," and finally "I will turn right." But they did these three steps separately, like three different people shouting instructions to the driver without talking to each other. The result was a confused driver who didn't realize that the truck's movement was the reason for the turn.

AutoDrive-P 3 is the new solution. It teaches the robot to drive like a thoughtful human expert who never skips a step and always connects the dots.

Here is how it works, broken down into simple concepts:

1. The "Chain of Thought" (The Internal Monologue)

Instead of just giving an answer, AutoDrive-P 3 forces the robot to talk to itself out loud before acting. It uses a structure called P 3-CoT (Perception-Prediction-Planning Chain of Thought).

Think of it like a chess player before making a move:

Perception (The Eyes): "I see a red car ahead and a pedestrian on the sidewalk."
Prediction (The Crystal Ball): "The red car is slowing down, and the pedestrian looks like they might step off the curb."
Planning (The Strategy): "Because the car is slowing and the pedestrian is risky, I will gently brake and stay in my lane."

In previous systems, the robot might have jumped straight to "Brake!" without explaining why. AutoDrive-P 3 makes the robot write down the whole story first. This ensures the final decision is based on a solid understanding of the world.

2. The "Coach" (Reinforcement Fine-Tuning)

How does the robot learn to do this perfectly? The authors created a special training method called P 3-GRPO.

Imagine a driving instructor sitting in the passenger seat.

Old Way: The instructor only yelled "Good job!" or "Bad job!" after the car finished the whole lap. The robot didn't know if it crashed because it didn't see the obstacle or because it turned too late.
AutoDrive-P 3 Way: The instructor gives feedback at every single step.
- "You missed that pedestrian in your 'Perception' step! Try again."
- "Your 'Prediction' that the car would stop was wrong."
- "Your final 'Planning' to turn left was safe, but only because you fixed the earlier mistakes."

This "hierarchical" coaching ensures that if the robot makes a mistake in seeing the world, it learns to fix that first, which naturally leads to better driving decisions later.

3. The "Dual Mode" (Thinking Fast vs. Thinking Slow)

One of the coolest features is that the robot has two modes, just like humans do:

Detailed Thinking (The Slow, Careful Mode): When the road is crazy (rain, construction, crowds), the robot takes its time. It writes a long, detailed story about every car and pedestrian before moving. This is for safety.
Fast Thinking (The Quick Mode): When the road is empty and clear, the robot skips the long story and just gives the answer. It's like driving on a highway with no traffic; you don't need to narrate every breath, you just drive. This saves time and computing power.

The Result

By combining these ideas, AutoDrive-P 3 is like upgrading a robot driver from a nervous learner to a seasoned veteran.

It sees better (Perception).
It guesses better what others will do (Prediction).
It decides better how to move (Planning).

The paper shows that this method makes the car much safer (fewer crashes) and smarter in tricky situations than any other system currently available. It proves that if you teach a robot to think through the whole process, rather than just guessing the answer, it becomes a much better driver.

1. Problem Statement

Current Vision-Language Models (VLMs) applied to end-to-end autonomous driving face two critical limitations:

Lack of Structured Reasoning (CoT): Some VLMs directly output planning trajectories without intermediate reasoning, bypassing crucial perception and prediction stages. This creates a domain gap and compromises decision-making robustness, especially in long-tail scenarios.
Fragmented Multi-Task Synergy: Other VLMs can handle perception, prediction, and planning tasks but treat them as separate, isolated modules. This lack of synergy prevents the planning module from fully leveraging perceptual and predictive features, leading to suboptimal planning performance.
Incomplete Supervision: Existing reinforcement learning approaches (like GRPO) often optimize only the final planning metrics (e.g., L2 distance), leaving perception and prediction modules without direct reward guidance, resulting in superficial gains and poor interpretability.

The core issue is the failure to capture the staged Chain-of-Thought (CoT) process where accurate perception enables reliable prediction, and both are indispensable foundations for robust planning.

2. Methodology

The authors propose AutoDrive-P 3, a unified framework that integrates Perception, Prediction, and Planning through structured reasoning and hierarchical reinforcement learning.

A. P 3-CoT Dataset

To address the lack of unified reasoning data, the authors constructed the P 3-CoT dataset.

Construction: Based on nuScenes and NAVSIM datasets, they annotated key objects (not just all objects) with bounding boxes, predicted future actions, and ego planning trajectories.
Format: Unlike standard QA pairs, this dataset uses a unified CoT format where the model generates reasoning traces (<thinking>) followed by structured answers (<answer>) for three sequential stages:
1. Perception: Identifying and localizing key objects.
2. Prediction: Forecasting the future behavior of those objects based on perception.
3. Planning: Generating the ego vehicle's trajectory based on perception and prediction.
Scale: Includes ~25k frames from nuScenes and ~115k frames from NAVSIM.

B. Training Pipeline

The training process consists of two stages:

Supervised Fine-Tuning (SFT) for Cold-Start:
- The base VLM (Qwen2.5-VL-3B) is fine-tuned on the P 3-CoT dataset.
- This aligns the model with the autonomous driving domain and teaches it the structured P 3-CoT output format, establishing a baseline for reasoning.
P 3-GRPO (Group Relative Policy Optimization):
- A novel hierarchical reinforcement learning algorithm is introduced to optimize all three modules simultaneously.
- Multi-Component Reward Function: The total reward $R$ $R$ is a weighted sum of four components:
  - $R_{format}$ : Ensures output follows the CoT structure.
  - $R_{perc}$ : Rewards accurate object detection (IoU, Precision, Recall).
  - $R_{pred}$ : Rewards accurate behavior prediction, weighted by detection quality (linking perception to prediction).
  - $R_{plan}$ : Rewards trajectory quality (L2 distance, PDMS/EPDMS scores).
- Mechanism: By optimizing these rewards jointly, the algorithm ensures that improvements in planning are grounded in enhanced perception and prediction, fostering synergistic interactions.

C. Dual Thinking Modes

To balance inference efficiency with performance, the framework supports two modes:

Detailed Thinking: Generates full reasoning traces for all three stages (Perception, Prediction, Planning) before outputting answers.
Fast Thinking: Adheres to the P 3-CoT structure but skips the explicit reasoning text, outputting only the final answers for each module to reduce latency.

3. Key Contributions

Unified Framework (AutoDrive-P 3): An end-to-end VLM framework that explicitly captures the causal relationships between perception, prediction, and planning, resolving the "black box" nature of direct trajectory generation.
P 3-CoT Dataset: A high-quality dataset featuring unified reasoning chains and key-object annotations, bridging the gap between VLMs and autonomous driving requirements.
P 3-GRPO Algorithm: A hierarchical reinforcement learning method that provides progressive supervision across all three cognitive stages, significantly improving reasoning coherence and planning reliability compared to planning-only optimization.
Dual Thinking Modes: A practical mechanism to trade off between high-accuracy reasoning (Detailed) and real-time inference speed (Fast).

4. Experimental Results

The method was evaluated on nuScenes (open-loop) and NAVSIMv1/v2 (closed-loop) benchmarks.

nuScenes (Open-Loop):
- Achieved state-of-the-art (SOTA) performance in collision rates, reducing them by ~40% compared to previous SOTA methods (e.g., OmniDrive, AutoVLA).
- Achieved comparable L2 displacement errors with a significantly smaller model (3B parameters vs. 7B+ in competitors) and less training data.
NAVSIM (Closed-Loop):
- Achieved SOTA results on NAVSIMv1 (90.6 PDMS) and NAVSIMv2 (89.9 EPDMS) using vision-only inputs.
- Outperformed complex multi-modal baselines (e.g., DiffusionDrive, WoTE) that utilize LiDAR data.
Ablation Studies:
- Confirmed that P 3-GRPO significantly outperforms "SFT only" and "SFT + Planning-only GRPO," proving the necessity of joint optimization.
- Demonstrated that video inputs and historical trajectories are critical for performance.
- Showed that the "Detailed" mode offers higher safety/reliability, while the "Fast" mode maintains near real-time performance (1 Hz).

5. Significance

Interpretability: By forcing the model to articulate its reasoning (perception $\to$ prediction $\to$ planning), the system becomes more transparent and easier to debug, addressing a major safety concern in autonomous driving.
Synergistic Learning: The work proves that treating perception and prediction as core components of the planning loop, rather than byproducts, leads to superior decision-making in complex, long-tail scenarios.
Efficiency: The dual-mode approach offers a practical solution for deploying VLMs in real-time systems without sacrificing the safety benefits of deep reasoning.
Generalization: The approach demonstrates that large-scale VLMs, when properly fine-tuned with structured reasoning and hierarchical rewards, can outperform specialized, smaller end-to-end models in both open-loop and closed-loop settings.