🤖 AI

Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

This paper introduces the DEFT-RLVR framework and AD-MCQ benchmark to eliminate trajectory anchoring bias in autonomous driving VLMs by deferring ground-truth future trajectories from pre-decision inputs to post-decision verification targets, thereby enhancing causal reasoning and reducing hallucinations without compromising general visual capabilities.

Original authors: Zixuan Huang, Yang Zhou, Kaixuan Wang, Guli Zhang, Hongyan Xie, Yakun Zhu, Hao Geng, Yikun Ban, Deqing Wang

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zixuan Huang, Yang Zhou, Kaixuan Wang, Guli Zhang, Hongyan Xie, Yakun Zhu, Hao Geng, Yikun Ban, Deqing Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car. You want it to be smart, not just by memorizing roads, but by understanding why it should stop at a red light or turn left at a curve. This is the world of Autonomous Driving, where computers use cameras and sensors to see the world, and Vision-Language Models (VLMs) act as the robot's brain, translating what it sees into words and decisions. To make these brains smarter, scientists often use a trick called Chain-of-Thought (CoT). Think of this like asking a student to "show their work" on a math test. Instead of just giving the answer, the robot explains its reasoning step-by-step: "I see a stop sign, so I must stop." This helps the robot learn to think logically rather than just guessing.

However, there is a tricky problem in teaching these robots. Usually, when scientists create these "show your work" examples, they show the robot the correct answer (the exact path the car took) before asking it to explain why. It's like giving a student the answer key before they even read the question. The paper suggests this creates a bad habit called anchoring bias. The robot stops thinking about the road and starts justifying the answer it already sees, sometimes even making up fake reasons (hallucinations) to match the path it was shown. The researchers wanted to fix this to make self-driving cars safer and more trustworthy.

The Problem: Accessing the "Show Your Work" Test Early

The authors discovered that when they let the robot see the future path (the "Ground Truth") while it was trying to figure out its reasoning, the robot relied on that information. It wasn't actually analyzing the traffic, pedestrians, or road signs. Instead, it was looking at the future path and saying, "Oh, I see we are turning right, so there must be a right-turn sign!" even if no sign was there.

They call this Trajectory Anchoring Bias. It's like a detective who is told the suspect's name at the start of the case. Instead of looking for clues, the detective just invents evidence to fit the name they were given. The researchers found that when the robot saw the future path first, it produced reasoning that was full of made-up details (hallucinations) and wasn't truly based on what was happening on the road. In fact, in difficult driving situations, the robot's "explanations" were much less reliable when it was allowed to peek at the answer.

The Solution: The "Deferred Exposure" Game

To fix this, the team invented a new way to train the robot, which they call DEFT-RLVR (Deferred Exposure of Future Trajectories for Reinforcement Learning with Verifiable Reasoning). The core idea is simple: Don't show the answer until after the student has made their choice.

They created a game called AD-MCQ (Autonomous-Driving Multiple-Choice Question). Instead of asking the robot to draw a perfect, continuous line for where the car should go (which is hard and prone to errors), they gave it a few pre-drawn paths to choose from. Imagine a video game where you have to pick one of six different routes through a level.

Here is how the new training works in two steps:

Turn 1 (The Decision): The robot looks at the road without seeing any of the possible paths. It must look at the traffic, the signs, and the curves, and decide what it should do. It has to write down its reasoning and pick a direction (like "slow down and turn right") based only on what it sees.
Turn 2 (The Verification): Only after the robot has committed to its decision do the researchers reveal the six possible paths. The robot then has to match its decision to the correct path from the list.

This forces the robot to actually think about the scene first. It cannot rely on the path and make up a reason. It has to build a reason first, then find the path that fits.

What They Found

The results were impressive. When they trained the robots using this "deferred" method:

Better Thinking: The robots started giving much more honest and accurate reasons for their decisions. They stopped making up fake signs or invisible obstacles.
Fewer Mistakes: The robots became much better at picking the right path in tricky situations, like sudden stops or sharp turns.
Still Smart at Other Things: A big worry was that teaching the robot to drive might make it forget how to do other things, like describing a picture or solving a puzzle. But the researchers found that this new method actually improved the robot's general visual skills slightly, or at least kept them just as good as before.

They tested this on different robot "brains" and found that the method worked consistently. The robots learned to drive more safely and logically without losing their ability to understand the world around them.

Why This Matters

This paper doesn't just say "we made a better robot." It proves that the way we teach the robot matters just as much as the robot itself. By stopping the robot from peeking at the answer before it thinks, we get a much smarter, more honest driver. It's a reminder that in the race for self-driving cars, the goal isn't just to get the car to the destination, but to make sure it knows why it got there and didn't just guess its way through the traffic. The researchers have made their code and data available, so other scientists can use this "deferred exposure" trick to build even safer and smarter autonomous vehicles in the future.

Technical Summary: Deferred Exposure of Future Trajectories for Verifiable Reasoning in Autonomous Driving VLMs

Problem Statement

Recent Vision-Language-Action (VLA) models for autonomous driving (AD) increasingly rely on Chain-of-Thought (CoT) supervision to enhance the reasoning capabilities of their Vision-Language Model (VLM) components. However, existing annotation pipelines typically expose the teacher model to the logged ground-truth (GT) future trajectory before the model generates its reasoning.

The authors identify and empirically validate a phenomenon termed trajectory anchoring bias. When the GT trajectory is revealed, the teacher model tends to rationalize the known outcome rather than inferring a decision from scene evidence. This leads to:

Reduced Causal Faithfulness: The generated CoTs are geometrically consistent with the GT but fail to faithfully identify the scene evidence that supports the action.
Increased Hallucinations: Models are significantly more prone to fabricating causal cues (e.g., inventing traffic signs) to justify the revealed trajectory, particularly in causally challenging scenes.
Supervision Mismatch: The GT trajectory acts as a reasoning shortcut (a pre-decision anchor) rather than a decision target, causing the model to learn to reverse-infer explanations for known outcomes.

While hiding the GT trajectory eliminates this shortcut, it introduces a new challenge: open-ended trajectory generation entangles high-level decision-making with precise geometric synthesis and low-level dynamics, making the task difficult for VLMs and hard to verify.

Methodology

To address these issues, the paper proposes a framework consisting of a new benchmark formulation and a training strategy: AD-MCQ and DEFT-RLVR.

1. AD-MCQ: Autonomous-Driving Multiple-Choice Question

Instead of open-ended coordinate generation, the authors cast AD planning as a selection task among explicit trajectory candidates.

Discrete Trajectory Prototypes: A codebook of $K=8192$ trajectory prototypes is constructed using K-means clustering on a corpus of logged futures. This quantizes continuous trajectories into a finite motion vocabulary.
Candidate Construction: For each scene, the GT trajectory is mapped to its nearest prototype (the oracle). Distractors are constructed by retrieving prototypes that are geometrically similar but distinct (hard negatives), including scale-matched alternatives and constant-velocity extrapolations.
Verifiability: The task becomes selecting the correct option index from a shuffled set of $M$ candidates, providing an exactly verifiable decision target without requiring open-ended geometric synthesis.

2. DEFT-RLVR: Deferred Exposure of Future Trajectories for RLVR

Building on AD-MCQ, the authors propose DEFT-RLVR (Deferred Exposure of Future Trajectories for Reinforcement Learning with Verifiable Rewards) to mitigate anchoring bias during training.

Two-Stage Interaction:
- Turn 1 (Causal Decision Reasoning): The policy model receives only the scene context (visual history, ego state, navigation instruction) and must generate a high-level decision and reasoning trace without seeing the candidate trajectories.
- Turn 2 (Explicit-Trajectory Grounding): Only after the model commits to a scene-derived decision are the candidate trajectories revealed. The model then selects the candidate that best matches its committed decision.
Deferred Exposure: This design ensures that candidate geometry shapes the grounding of a decision, not the formation of the decision itself, preventing the model from taking shortcuts by reasoning around a visible answer.
Structured Rubric Rewards: To prevent the model from exploiting the MCQ format via random guessing or shortcut exploitation, the authors introduce process supervision.
- For rollouts with correct MCQ answers, the Turn-1 reasoning trace is evaluated against an instance-specific rubric.
- The rubric is generated offline by a large VLM based on the scene evidence.
- An online text-only judge evaluates the reasoning trace against the rubric criteria (e.g., grounding, absence of hallucination, causal coherence) without access to the visual input or candidate options.
- The final reward is the product of the exact-choice reward and the rubric reward ( $R = R_{MCQ} \times R_{RUB}$ ).

Key Contributions

Identification of Trajectory Anchoring Bias: The paper provides empirical evidence that exposing GT trajectories during CoT annotation induces post-hoc rationalization and severe hallucinations, degrading causal faithfulness in AD reasoning.
AD-MCQ Benchmark: A verifiable candidate-trajectory benchmark that preserves trajectory-level granularity (braking time, speed profile, lateral geometry) while replacing open-ended generation with exact selection, enabling scalable and controllable difficulty.
DEFT-RLVR Framework: A training paradigm that defers candidate exposure until after the policy commits to a decision. Combined with rubric-based process supervision, it improves AD reasoning while preserving the general visual capabilities of the base VLM.

Experimental Results

The authors evaluated their approach using Qwen3-VL-8B-Instruct and Qwen3.5-4B as base models, with Qwen3.5-397B-A17B serving as the teacher/judge.

AD Reasoning Performance:
- DEFT-RLVR significantly outperforms training-free baselines and ablations. On the Qwen3-VL-8B model, it achieved 77.9% accuracy on the AD-MCQ-500 test set, compared to 28.1% for the base model and 56.6% for training-free DEFT.
- It also improved Causal Faithfulness Score (CFS) from 0.431 (training-free DEFT) to 0.658, and High-Level Decision (HLD) consistency from 0.425 to 0.501.
- Compared to the JEFT (Joint Exposure) ablation, DEFT-RLVR showed consistent gains (e.g., +15.3% accuracy on Qwen3-VL-8B), confirming that deferred exposure reduces shortcut-driven optimization.
General Visual Capability:
- Unlike Supervised Fine-Tuning (SFT) distillation, which degraded general visual capabilities (dropping average general capability from 54.81% to 49.80%), DEFT-RLVR preserved and even enhanced general capabilities.
- On Qwen3-VL-8B, the average general visual capability score increased from 54.81% to 56.09%. The authors attribute this to the fine-grained nature of RL-based policy gradients and the shared visual-spatial reasoning required for the task.
Generalization:
- The method demonstrated strong out-of-distribution (OOD) transfer on the nuScenes domain, improving accuracy from 39.6% (training-free) to 49.5% with DEFT-RLVR.
- The gains were robust across different candidate-set constructions and distractor geometries.
Efficiency:
- The instance-specific rubric generation in DEFT-RLVR added only 0.5% overhead to training time compared to correctness-only training, while being 41.1% faster than online image-conditioned rubric grading variants.

Significance and Claims

The paper claims that DEFT-RLVR offers a scalable and flexible foundation for verifiable AD reasoning by:

Decoupling Decision and Verification: Transforming future trajectories from pre-decision anchors into post-decision verification targets, thereby forcing models to reason from scene evidence first.
Avoiding Dual Bottlenecks: By using candidate selection rather than direct trajectory token generation, the method avoids the bottlenecks of trajectory quantization error and the degradation of general visual capabilities often seen in direct generation tasks.
Enabling Verifiable RL: Providing a mechanism to apply reinforcement learning with verifiable rewards (RLVR) to complex, continuous control tasks like autonomous driving without requiring open-ended geometric synthesis.

The authors conclude that their approach improves generalizable AD reasoning while maintaining the model's general visual capabilities, addressing a critical gap in current VLA training pipelines where CoT supervision often relies on biased, outcome-revealed data.