Benchmarking Deflection and Hallucination in Large Vision-Language Models

This paper introduces VLM-DeflectionBench, a dynamic benchmark and evaluation protocol designed to assess Large Vision-Language Models' ability to handle conflicting or insufficient multimodal evidence by generating appropriate deflections rather than hallucinating answers.

The Gist

Imagine you have a brilliant, super-smart robot assistant named "Visionary." Visionary can look at a picture, read a book, and answer almost any question you ask. But there's a catch: sometimes, Visionary gets too confident and makes things up when it doesn't actually know the answer. This is called hallucination.

On the other hand, sometimes Visionary is so scared of making a mistake that it refuses to answer even when it does have the right information. This is called deflection (or "abstaining").

The paper you shared introduces a new test called VLM-DeflectionBench. Think of it as a "stress test" or a "driving exam" for these AI robots to see if they can tell the difference between "I know this" and "I have no idea."

Here is the breakdown of what the researchers did, using simple analogies:

1. The Problem: The "Know-It-All" vs. The "Shy Kid"

In the past, researchers tested these robots by asking them questions where the answer was hidden in a pile of documents.

• The Old Way: They just checked if the robot got the answer right.
• The Flaw: If the robot didn't know the answer, it would either:
  • Hallucinate: Guess wildly and make up a story (like a student who guesses "42" on a math test just to get points).
  • Deflect: Say "I don't know" even when the answer was right in front of them (like a shy student who stays silent even though they raised their hand).

The old tests didn't care how the robot failed, only if it failed. But in the real world, we need robots that know when to speak up and when to stay quiet.

2. The Solution: A Dynamic "Trap" Test

The researchers built a new testing ground called VLM-DeflectionBench. Imagine this as a high-tech escape room with four different rooms:

• Room 1: The Memory Room (Parametric). No books, no pictures, just the question. The robot must rely only on what it learned in school.
  • Goal: If the robot doesn't know it, it should say, "I don't know."
• Room 2: The Perfect Library (Oracle). The robot is given the exact right book with the answer highlighted.
  • Goal: It should answer correctly. If it says "I don't know" here, it's being too shy.
• Room 3: The Messy Desk (Realistic). The robot gets the right book, but it's buried under 10 other books with wrong information.
  • Goal: It needs to find the right book and ignore the noise.
• Room 4: The Trap (Adversarial). The robot is given only the wrong books (distractors).
  • Goal: It should realize the books are lying and say, "I cannot answer this." If it tries to guess based on the wrong books, it fails.

3. The "Filter" Machine

One of the biggest problems with old tests is that as robots get smarter, they memorize the answers. A question that used to require looking up a book might now be answered from memory.

The researchers created a dynamic pipeline (a smart filter). Before a question becomes part of the test, they run it through a panel of other super-smart robots.

• If any robot can answer it from memory, the question is thrown out.
• Only the questions that truly require looking up new information are kept.
• This ensures the test stays hard and relevant, even as AI gets smarter in the future.

4. What They Discovered

They tested 20 of the smartest robots on the market (both open-source and expensive commercial ones). The results were surprising:

• The "Confident Liar" Problem: Even the best robots often hallucinate. When they were given misleading information (Room 4), they didn't say "I don't know." Instead, they confidently made up answers based on the lies.
• The "Text Bias": The robots are weirdly biased toward text. If they see a picture that says "A" but a piece of paper says "B," they often ignore the picture and believe the paper, even if the picture is the truth.
• The "Shy Kid" Problem: When the researchers told the robots to be very strict ("Only answer if you are 100% sure!"), the robots became too cautious. They stopped answering even when they had the right information, causing their accuracy to crash.

5. The Big Takeaway

The paper concludes that we need to stop just asking AI "Are you smart?" and start asking "Are you reliable?"

A truly reliable AI isn't just one that knows facts; it's one that knows when it doesn't know. It needs to be brave enough to guess when it's right, but humble enough to say "I don't know" when the evidence is missing or confusing.

In short: The researchers built a better "driver's license test" for AI. They found that while these robots are great at driving on empty roads, they tend to crash when the road is foggy (noisy data) or when they are tricked by a fake sign (distractors). They need to learn to pull over and ask for help instead of crashing the car.

Technical Summary

1. Problem Statement

Large Vision-Language Models (LVLMs) are increasingly deployed in real-world applications requiring knowledge-intensive multimodal reasoning. While existing benchmarks evaluate accuracy, they suffer from two critical limitations:

• Rapid Obsolescence: As LVLM training sets grow, models can answer many questions using parametric (internal) memory rather than retrieval, rendering retrieval-augmented benchmarks ineffective over time.
• Failure Mode Ambiguity: Existing benchmarks focus on accuracy but fail to distinguish between hallucination (generating unsupported answers) and deflection (abstaining from answering when evidence is insufficient or contradictory).
• Lack of Conflict Testing: Current datasets rarely test model behavior when visual and textual evidence conflict or when retrieved knowledge is noisy/misleading.

The core problem is that LVLMs often hallucinate confidently even when retrieved evidence is incomplete or contradictory, rather than deferring (deflecting) the query. There is a lack of a dynamic, robust benchmark to measure this trade-off.

2. Methodology: VLM-DeflectionBench

The authors introduce VLM-DeflectionBench, a dynamic benchmark designed to evaluate model reliability under varying knowledge conditions. The framework consists of three main components:

A. Dynamic Data Curation Pipeline

To prevent obsolescence, the authors propose a pipeline that filters out samples solvable by parametric knowledge:

1. Parametric Filtering: A pool of raw samples from six diverse KB-VQA sources (e.g., InfoSeek, WebQA, MRAG-Bench) is tested against a set of strong "gating" models (e.g., Gemma3, Qwen2.5-VL) in a parametric setting (no external context).
2. Retention Criteria: Only samples where all gating models fail (labelled as "Incorrect" or "Not Attempted") are retained. This ensures the remaining 2,775 samples genuinely require external retrieval.
3. Knowledge Augmentation:
   • Gold Context: Correct textual or visual evidence is paired with the sample.
   • Negative (Distractor) Context: The pipeline mines negative examples (irrelevant text or images) using retrieval indices (Wikipedia for text, image databases for visuals).
4. Quality Control: Samples are discarded if they are unsolvable even with gold evidence, or if distractors allow a gating model to answer correctly.

B. Four Evaluation Scenarios

The benchmark evaluates models across four distinct context configurations to disentangle memorization from retrieval robustness:

1. Parametric: No external context (tests internal knowledge; expected low accuracy).
2. Oracle: Only gold (correct) evidence provided (tests grounding capability).
3. Realistic: Gold evidence mixed with distractors (tests robustness to noise).
4. Adversarial: Only distractor (incorrect) evidence provided (tests ability to deflect/hallucinate).

C. Fine-Grained Evaluation Protocol

Using GPT-4o as an automated judge (validated with >92% human agreement), responses are categorized into:

• Accuracy: Correct answer based on evidence.
• Deflection: Explicit refusal to answer when evidence is insufficient.
• Hallucination: Incorrect answer generated despite insufficient or misleading evidence.

3. Key Contributions

1. Dynamic Curation Pipeline: A method to continuously filter datasets, ensuring benchmarks remain difficult as model capabilities improve by removing parametrically solvable questions.
2. VLM-DeflectionBench Dataset: A collection of 2,775 curated samples covering diverse multimodal retrieval settings, each paired with gold and distractor contexts.
3. Novel Evaluation Framework: The first KB-VQA benchmark to explicitly measure the hallucination vs. deflection trade-off across four controlled scenarios, moving beyond simple accuracy metrics.

4. Key Results

Experiments were conducted on 20 state-of-the-art LVLMs (both open-weight and proprietary, e.g., GPT-5, Claude-4, LLaVA, InternVL).

• General Failure to Deflect: Most models fail to deflect reliably. Even when provided with misleading or incomplete evidence, models tend to hallucinate rather than abstain.
  • In the Adversarial scenario (distractors only), top models like Mistral-Small-3.1 achieved ~83.8% deflection, but others like Qwen2.5-VL hallucinated on 83.9% of queries.
  • Proprietary models (e.g., Claude-Opus-4) showed better deflection (88.3%) but still hallucinated significantly in realistic settings.
• Grounding Bottleneck: In the Oracle scenario (perfect evidence), hallucination rates remained high (e.g., LLaVA-OneVision hallucinated 41.6% of the time), indicating that the primary bottleneck is not retrieval but the model's ability to ground its generation in the provided context.
• Modality Bias (Language over Vision): In the Realistic scenario, models exhibited a strong bias toward textual priors.
  • When textual distractors were present alongside visual gold evidence, accuracy collapsed (often to <1%), and deflection spiked (>80%).
  • Models struggled to trust visual evidence when contradicted by misleading text.
• Prompting Trade-offs: Increasing prompt strictness (forcing models to abstain) successfully reduced hallucinations in adversarial settings but caused over-deflection in realistic settings, drastically reducing accuracy even when valid evidence was present.
• Retrieval Noise Sensitivity: As the number of distractors increased (1 to 10), accuracy declined linearly while hallucination rose. Models did not increase deflection rates sufficiently to compensate, preferring to guess.

5. Significance and Impact

• Reliability over Accuracy: The paper argues that for real-world deployment, a model's ability to know when not to answer is as critical as its ability to answer correctly.
• Standard for RAG Evaluation: VLM-DeflectionBench provides a reproducible, extensible framework for evaluating Retrieval-Augmented Generation (RAG) systems, addressing the "calibration" gap in current LVLMs.
• Future-Proofing: By using a dynamic pipeline, the benchmark can evolve alongside stronger models, ensuring it remains a valid stress test for hallucination and deflection behaviors.
• Insight into Model Behavior: The results highlight that current LVLMs lack calibrated confidence; they often hallucinate to satisfy user requests rather than admitting uncertainty, a critical failure mode for safety-critical applications.

The authors conclude that while prompt engineering can force deflection, it often comes at the cost of utility. True reliability requires architectural improvements in how models weigh evidence and manage uncertainty, rather than just better prompting. All resources are set to be publicly available.