Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning

This paper investigates the reliability of Vision-Language Models (VLMs) in autonomous driving by exposing their tendencies toward response inconsistency and weak temporal reasoning, and subsequently proposes the FutureVQA benchmark and a self-supervised chain-of-thought tuning method to enhance grounded future scene reasoning without requiring temporal labels.

The Gist

Imagine you are hiring a very smart, well-read co-pilot for your car. This co-pilot has read millions of driving manuals, watched countless traffic videos, and can describe a street scene in perfect detail. You ask them, "What's happening right now?" and they answer perfectly.

But then you ask, "Okay, what will happen four seconds from now?"

This is the problem researchers at DFKI and TU Delft tackled in their new paper. They discovered that while these AI "co-pilots" (called Vision-Language Models or VLMs) are great at describing the present, they are surprisingly bad at predicting the future. They often contradict themselves, guess randomly, or fail to understand how time moves forward.

Here is a simple breakdown of their findings and solution, using some everyday analogies.

1. The Problem: The "Amnesiac" Co-Pilot

The researchers found two main issues with current driving AIs:

• The "Shuffling" Trick (Inconsistency):
  Imagine you ask your co-pilot, "Is that a red truck?" and they say "Yes." Then, you ask the exact same question but swap the order of the multiple-choice answers on the screen. Suddenly, the AI says "No, it's a white bus."

  • The Metaphor: It's like a student who memorized the answer key but didn't learn the lesson. If you shuffle the order of the questions, they get confused and guess randomly. They aren't actually thinking; they are just pattern-matching.

• The "Time Travel" Failure (Temporal Reasoning):
  The AI can describe a car turning left right now. But if you ask, "Where will that car be in 4 seconds?", the AI might say it's still turning, or suddenly say it's driving straight, or claim it's on the other side of the road.

  • The Metaphor: Imagine watching a movie, but the AI only sees one single frame. It knows what a car looks like, but it doesn't understand the flow of the movie. It's like a person who has seen a photo of a runner but doesn't understand that the runner will keep moving forward. They treat every moment as a brand new, unrelated snapshot.

2. The New Test: "FutureVQA"

To prove this, the team created a new test called FutureVQA.

• The Analogy: Think of this as a "driver's ed" final exam, but with a twist. Instead of showing the student the answer key (the future video), they only show them the last 5 seconds of the video and ask, "What happens next?"
• The Result: Even the smartest AI models (like GPT-4o) failed this test. They got the "visual description" part right (e.g., "There is a red car"), but they failed the "prediction" part (e.g., "The red car will hit the intersection").

3. The Solution: "FutureAgent" (The Self-Taught Student)

The researchers didn't just point out the problem; they built a fix called FutureAgent.

• How it works: Instead of hiring a human to write thousands of "future prediction" labels (which is expensive and slow), they taught the AI to teach itself.
  1. They let the AI look at a video and describe the actual future (since it has the video).
  2. Then, they hid the future part of the video and asked the AI to predict it using only the past.
  3. If the AI's prediction matched what it saw in the "real" future, it got a gold star. If it was wrong, it learned from its mistake.
• The "Chain of Thought" Trick: They also taught the AI to "think out loud." Instead of jumping straight to the answer, the AI is forced to say, "First, the car will move a little. Then, it will turn. Finally, it will stop." This step-by-step reasoning helps the AI understand the flow of time.

4. The Big Takeaway

The paper concludes that seeing is not the same as understanding.

Just because an AI can describe a picture perfectly doesn't mean it understands how the world moves. A reliable self-driving car needs more than just a good eye; it needs a sense of time.

The Final Metaphor:
Current AI drivers are like a tourist with a camera. They can take a beautiful photo of a street and describe it perfectly. But if you ask them, "If I keep walking for 10 minutes, where will I end up?", they might guess wildly because they don't understand the concept of walking or time.

The new FutureAgent is like a local guide. They have learned to simulate the journey in their head, step-by-step, so they can actually tell you where you'll be in the future. This makes them much safer and more reliable for real-world driving.

Technical Summary

Here is a detailed technical summary of the paper "Probing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning."

1. Problem Statement

The paper addresses a critical gap in the deployment of Vision-Language Models (VLMs) for autonomous driving. While current VLMs excel at static scene understanding (e.g., identifying objects, traffic signs), their reliability as driving assistants is compromised by two fundamental issues:

• Response Inconsistency: VLMs often produce divergent answers for identical or semantically equivalent inputs. Minor perturbations, such as shuffling the order of multiple-choice options, cause significant accuracy drops, suggesting the models rely on memorized patterns or random guessing rather than genuine reasoning.
• Lack of Grounded Temporal Reasoning: VLMs struggle to predict future scenes based on past observations. They often fail to maintain coherent reasoning over time, producing contradictory descriptions of future events or failing to align sequential events correctly. Crucially, the paper finds that strong visual perception does not guarantee strong temporal reasoning; models with excellent image understanding often perform poorly on future prediction tasks.

The core hypothesis is that current VLMs lack temporal grounding—they do not truly "experience" the flow of time and thus cannot reliably infer how present actions influence future outcomes.

2. Methodology

A. Benchmark: FutureVQA

To rigorously evaluate these limitations, the authors introduce FutureVQA, a human-annotated benchmark dataset specifically designed for temporal reasoning in driving.

• Construction: Unlike existing datasets that rely on rule-based templates or GPT-generated data, FutureVQA uses human experts to create diverse, natural language Q&A pairs based on video clips from the OpenDV-YouTube dataset.
• Scope: It covers 2,700 question-answer pairs across 16 cities, focusing on predicting scenes 1 to 12 seconds into the future based only on the preceding 5 seconds of visual input.
• Categories: Questions are categorized into Hallucination, General, Traffic Understanding, Absolute Location, and Relative Position to test diverse reasoning capabilities.
• Evaluation Protocol: The benchmark employs a multi-trial evaluation where answer options are shuffled across multiple runs. This distinguishes between genuine understanding and random guessing or sensitivity to prompt formatting.

B. Proposed Solution: FutureAgent

To address the identified limitations, the authors propose FutureAgent, a self-supervised fine-tuning framework that enhances temporal reasoning without requiring explicit temporal labels.

• Self-Supervised Pseudo-Labeling: The method leverages a pre-trained VLM ($\psi$) to generate high-quality descriptions of future frames ($I_{t+\Delta t}$) when the actual future frame is available. These descriptions serve as pseudo-reference labels ($a_{ref}$).
• Training Objective: A new model ($\psi^*$) is initialized from the base model and fine-tuned to predict these descriptions using only past frames ($V_t$) as input. This forces the model to learn to "imagine" and temporally align future events based on context.
• Chain-of-Thought (CoT) Reasoning: The approach incorporates a temporal CoT strategy. Instead of jumping directly to the final prediction, the model is guided to articulate intermediate reasoning steps (e.g., predicting $t+1$, then $t+2$, etc.). This provides a structural prior for sequential evolution.
• Temporal Weighting: A time-aware weighting function $\lambda(\Delta t) = 2^{-\Delta t}$ is applied to the loss function, prioritizing short-term predictions while still supervising long-horizon reasoning.
• Efficient Visual Encoding: The system uses a Memory Decay Sampler (assigning fewer tokens to older frames) and an Adaptive Token Sampler (adjusting token count based on frame similarity) to reduce computational costs while maintaining performance.

3. Key Contributions

1. Critical Analysis of VLM Reliability: The paper provides empirical evidence that current VLMs (both open-source and commercial like GPT-4o) exhibit severe inconsistency and temporal misalignment in driving scenarios, even when they possess strong visual capabilities.
2. FutureVQA Benchmark: Introduction of a high-quality, human-annotated dataset that challenges models on time-specific future prediction, featuring diverse question types and a robust multi-trial evaluation protocol to filter out random guessing.
3. FutureAgent Framework: A novel self-supervised tuning method that significantly improves temporal consistency and future scene prediction without requiring expensive temporal annotations, outperforming video-based VLMs in this specific domain.

4. Experimental Results

The authors evaluated various models (LLaVA, Qwen-VL, CogVLM, GPT-4o) on the FutureVQA benchmark:

• Inconsistency: All tested models showed significant accuracy drops (up to 23.8% for CogVLM) when answer options were shuffled, confirming that many "correct" answers were likely due to memorization or bias rather than reasoning.
• Temporal Decay: Models exhibited a sharp decline in accuracy as the prediction horizon increased (from $t+1$ to $t+12$). For instance, GPT-4o, despite high visual understanding, suffered a 27.5% accuracy drop over 12 seconds.
• Performance of FutureAgent:
  • Consistency: The self-supervised approach significantly reduced the performance gap between single-trial and multi-trial evaluations, indicating more stable reasoning.
  • Accuracy: FutureAgent outperformed baseline models and even video-specialized VLMs (like LLaVA-Video) in future scene prediction tasks.
  • Metrics: The model achieved higher scores in BLEU, ROUGE, and CIDEr when comparing predicted future descriptions against reference descriptions generated with ground-truth future frames.
  • Ablation: Removing the CoT strategy or the self-supervised loop resulted in significant performance degradation, validating the necessity of both components.

5. Significance and Implications

• Safety Criticality: The findings highlight a major risk in deploying current VLMs for autonomous driving. A model that cannot consistently reason about the future or align events over time is unsafe for decision-making in dynamic environments.
• Decoupling Perception and Reasoning: The study proves that visual perception and temporal reasoning are distinct capabilities. Improving one does not automatically improve the other; specific architectural or training interventions (like the proposed self-supervised CoT) are required.
• Data Efficiency: The proposed method demonstrates that high-quality temporal reasoning can be achieved through self-supervised learning, bypassing the need for costly, large-scale human-annotated temporal datasets.
• Future Directions: The work suggests that future driving assistants must be explicitly trained on temporal dynamics and consistency, moving beyond static scene understanding to dynamic, time-grounded reasoning.