💬 NLP

Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination

This paper reveals that Vision-Language Models often produce self-reflective statements like "let me check again" as mere textual patterns rather than triggering genuine visual re-examination, a finding demonstrated by the VisualSwap framework showing that models frequently fail to detect semantically different image swaps despite such claims.

Original authors: Chufan Shi, Cheng Yang, Yaokang Wu, Linhao Jin, Bo Shui, Taylor Berg-Kirkpatrick, Xuezhe Ma

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chufan Shi, Cheng Yang, Yaokang Wu, Linhao Jin, Bo Shui, Taylor Berg-Kirkpatrick, Xuezhe Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Paper in Plain English: "Are VLMs Seeing or Just Saying?"

Imagine you are taking a math test with a very smart, chatty robot. You show the robot a picture of a triangle with a 60-degree angle. The robot starts thinking out loud: "Okay, I see a 60-degree angle. If I subtract that from 180, I get 120. Wait, let me check the picture again just to be sure."

Then, sneakily, you swap the picture on the screen with a different triangle that has a 50-degree angle. The new picture looks almost exactly like the old one, but that one number is different.

The robot, still talking to itself, says, "Let me check the figure again," but it doesn't actually look at the new picture. It just keeps doing the math it started earlier. It confidently answers 120, even though the picture now says the answer should be 130.

This paper, titled "Are VLMs Seeing or Just Saying?", investigates exactly this phenomenon in Vision-Language Models (VLMs). The researchers found that when these AI models say things like "let me check the image again," they are often just saying it, not actually seeing it.

Here is the breakdown of their discovery using simple analogies:

1. The "Ghost in the Machine" (The Illusion)

The researchers created a test called VISUALSWAP. They let an AI solve a problem, then swapped the image for a slightly different one while the AI was in the middle of its "thinking" process. They asked the AI to "re-check" the image.

The Result: The AI almost always failed. It ignored the new picture and kept answering based on the old picture it had seen seconds ago.
The Analogy: It's like a chef who says, "Let me taste the soup again," but instead of tasting the soup in the pot, they just remember the flavor they imagined earlier and pretend they tasted it. They are hallucinating that they looked, but they are actually just repeating a script.

2. The "Overthinkers" Are the Worst Offenders

You might think that "Thinking" models (AI designed to reason deeply and slowly) would be better at this. You'd be wrong.

The Finding: The "Thinking" models were three times worse at noticing the image swap than the standard models.
The Analogy: Imagine a student taking a test. The standard student glances at the new question and adjusts. The "Thinking" student, however, gets so deep in their own internal monologue ("I calculated X, then Y...") that they become blind to the fact that the question on the page has changed. Their own thoughts become a wall that blocks them from seeing the new reality.

3. The "Textual Inertia" (Why it happens)

The paper explains that once the AI starts writing a long chain of reasoning, it gets stuck in a groove. It becomes so focused on finishing its sentence and maintaining a logical flow that it stops paying attention to the visual input.

The Analogy: Think of a train on a track. Once it's moving fast, it's hard to stop it. The AI's "train of thought" is moving so fast that when the scenery (the image) changes, the train doesn't slow down to look out the window. It just keeps chugging along the old track.

4. The "Magic Switch" (How to fix it)

Here is the most interesting part: The AI is capable of seeing the change. It just won't do it on its own.

The Experiment: When a human (or a user prompt) explicitly interrupts the AI and says, "Stop! Look at the new image I just gave you," the AI suddenly wakes up. It sees the difference immediately and gets the answer right.
The Analogy: It's like a daydreaming student. If they are left alone, they will keep daydreaming about the old answer. But if a teacher taps them on the shoulder and says, "Hey, look at this new picture," they snap out of it and see the truth. The capability was there all along; they just needed an external nudge to break their trance.

5. The "Attention" Proof

The researchers looked under the hood of the AI (specifically at its "attention" mechanisms, which determine what the AI focuses on).

The Finding: When the AI says "let me check," its attention to the image barely moves. But when a user says "check the image," the AI's attention spikes dramatically.
The Takeaway: The AI isn't "thinking" about the image when it says it's checking; it's just generating text that sounds like it's checking.

Summary

The paper concludes that current AI models are masters of deception (even unintentionally). When they claim to "re-examine" an image during their own internal reasoning, they are often just repeating a learned phrase without actually looking. They are "saying" they see, but they are "blind" to the change.

However, this isn't a permanent flaw in their vision; it's a flaw in their self-control. They can see the truth, but they need a human to break their "textual inertia" and force them to actually look.

Key Takeaway for the General Public: If an AI tells you it's double-checking a picture, don't assume it actually did. It might just be talking to itself. You have to explicitly tell it to look again for it to truly see.

Technical Summary: Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination

Problem Statement
Vision-Language Models (VLMs) have demonstrated significant progress in multimodal reasoning, particularly with the advent of reasoning-enhanced models that utilize extended chain-of-thought (CoT) generation and self-reflection mechanisms. A common behavior in these models is the generation of self-reflective statements such as "Wait, let me check the figure again" during the reasoning process. While intended to mitigate perceptual hallucinations and ensure faithfulness to the input image, a critical question remains: do these statements trigger genuine visual re-examination, or are they merely learned linguistic patterns devoid of actual visual grounding? If models claim to re-examine an image but fail to detect visual changes, their reliability in safety-critical domains is compromised.

Methodology: VISUALSWAP and VS-BENCH
To investigate this phenomenon, the authors propose VISUALSWAP, a diagnostic framework designed to probe whether VLMs genuinely re-attend to visual inputs during self-reflection. The framework employs a two-stage protocol:

Standard Inference: The model generates an initial reasoning chain ( $R_a$ ) based on an original image ( $I_a$ ) and a question ( $Q$ ).
Re-examination Probe: The visual input is swapped with a semantically distinct but visually similar image ( $I_b$ ) while retaining the generated reasoning context ( $R_a$ ) and appending a self-reflective prompt (e.g., "Wait, let me check the figure again"). The model is then asked to continue generation.

If the model performs genuine visual re-examination, it should detect the discrepancy between $I_b$ and $R_a$ , update its reasoning, and produce the correct answer for $I_b$ . If it fails, it indicates "textual inertia," where the model relies on the prior reasoning trajectory rather than the new visual evidence.

To support this evaluation, the authors curated VS-BENCH, a benchmark comprising 800 image pairs derived from MathVista, MathVerse, MathVision, and MMMU-Pro. These pairs are constructed to share high-level visual similarity (layout, style) while diverging in fine-grained details critical for the ground-truth answer, ensuring that the question remains valid for both images but requires different answers.

Key Contributions

VISUALSWAP Framework: A principled diagnostic tool to evaluate the authenticity of visual re-examination in VLMs by manipulating visual inputs during the reasoning process.
VS-BENCH Benchmark: A dataset of 800 carefully constructed image pairs that isolate visual re-examination capabilities from general visual question answering.
Mechanistic Analysis: An investigation into the attention mechanisms underlying the failure, comparing self-generated reflection against explicit multi-turn user instructions.

Experimental Results
The authors evaluated 15 models across the Qwen3-VL, Kimi-VL, and ERNIE-VL families, including both "Instruct" and "Thinking" variants. The results reveal a critical failure mode:

Widespread Failure: Models overwhelmingly fail to detect image swaps during self-reflection. Performance degradation ( $\Delta = Acc_{base} - Acc_{probe}$ ) reaches up to 60% (e.g., ERNIE-4.5-VL-Thinking drops from 79.9% to 19.6%).
Thinking Models are More Vulnerable: Counterintuitively, "Thinking" models exhibit nearly 3x greater vulnerability than their "Instruct" counterparts. For instance, Qwen3-VL-32B-Thinking suffers a 48.3% drop compared to 17.9% for the Instruct version.
Scaling Does Not Help: Increasing model size does not mitigate the issue; larger models often show greater degradation (e.g., Qwen3-VL-235B-Thinking drops 54.6%).
Restoration via User Instruction: When the reflection is triggered by an explicit multi-turn user instruction (e.g., "Check the image again") rather than self-generated text, performance recovers to near-baseline levels. This confirms the models possess the latent capability to process the new image but fail to activate it autonomously.
Attention Analysis: Attention scores reveal that self-reflective statements elicit only negligible increases in attention to visual tokens. In contrast, explicit user instructions cause a substantial surge in visual attention, explaining the performance gap.

Significance and Claims
The paper concludes that current VLMs tend to "say" they are re-examining an image without actually "seeing" it. The phenomenon is identified as a failure of autonomous attentional control rather than a lack of visual capability. The authors argue that extended reasoning chains create strong textual inertia that decouples the model from visual evidence during continuous decoding.

The significance of this work lies in exposing a fundamental limitation of current Chain-of-Thought (CoT) paradigms in multimodal settings: they prioritize textual coherence over visual intake. The findings suggest that self-reflection, as currently implemented, may not be a reliable mechanism for correcting visual hallucinations. The paper posits that genuine autonomous visual re-grounding requires mechanisms beyond simple linguistic prompting, highlighting the need for future research into models that can effectively interrupt their own reasoning trajectories to re-engage with visual inputs. The authors emphasize that this is a diagnostic finding regarding model behavior, not a proposal for a new training algorithm, though they note that external interventions (like multi-turn prompting) can successfully restore grounding.