💬 NLP

Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models

This paper proposes a unified framework that uses pre-generation signals from visual-token representations and prompt-conditioned hidden states to predict and disentangle whether Vision-Language Model errors stem from perception/recognition bottlenecks or knowledge retrieval failures, enabling targeted interventions before answer generation.

Original authors: Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov, Timothy Baldwin, Yova Kementchedjhieva

Published 2026-07-13

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Khang Nhat Hoang Vo, Artem Vazhentsev, Artem Shelmanov, Timothy Baldwin, Yova Kementchedjhieva

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot friend who can look at a picture and answer any question about it. If you show it a crisp photo of a famous movie star and ask, "Who is this?", it's usually spot on. But if you ask, "Where was this actor born?" or "What's the natural home of this weird bug?", the robot sometimes stumbles. It might guess the wrong country or say the bug lives in the ocean when it actually lives in a forest.

For a long time, researchers treated these wrong answers like a single, messy blob of failure. They'd just say, "Oops, the robot got it wrong," and move on. But this new paper suggests that's like saying a car broke down without checking if it was the engine, the tires, or the fuel. The authors, a team of scientists, wanted to figure out exactly where the robot trips up before it even finishes its sentence.

The Two Big Mistakes: "I Can't See It" vs. "I Don't Know It"

The researchers built a special diagnostic tree to sort errors into four distinct buckets. Think of it like a detective trying to solve a mystery before the suspect even speaks.

The Visual Blur: Sometimes the picture is just too blurry, dark, or damaged. The robot literally can't see the details.
The Unknown Stranger: The picture is clear, but the robot has never met the person or animal in the photo. It's like looking at a celebrity from a culture the robot was never taught about.
The Forgotten Fact: The robot recognizes the person perfectly (e.g., "That's Tom Cruise!"), but it forgets the specific fact you asked for (e.g., "I can't remember his birth year").
The Missing Library: The robot knows who the person is, but the fact you asked for simply doesn't exist in its brain at all.

The paper argues against the idea that all these errors are the same. They explicitly show that you can't just use one generic "uncertainty meter" to catch all of them. A generic meter might tell you the robot is "unsure," but it won't tell you why.

The Magic of "Pre-Generation" Signals

Here is the coolest part: The authors found that they could predict which type of mistake was coming before the robot even started typing its answer. They didn't need to wait for the robot to say something wrong; they just needed to peek at its internal "thoughts" right after it looked at the picture and read the question, but before it generated a single word.

They discovered a fascinating split in how the robot thinks:

For Visual and Recognition Errors: If the robot is struggling to see the image or identify the object, the clue is hidden in the visual tokens. Imagine these as the robot's "eyes." If the eyes are confused, the signal is strongest right there in the visual part of the brain. The paper shows that looking at these visual signals is incredibly effective, with accuracy scores reaching as high as 97.0 (on a scale where 100 is perfect) for spotting visual evidence failures.
For Factual Errors: Once the robot has successfully identified the object, the clues for factual mistakes move to a different part of the brain: the prompt-conditioned hidden states. Think of this as the robot's "memory center" where it holds the question and the entity name together. If the robot knows the bug but doesn't know its habitat, the error signal lives here. The paper found that looking at the last eight tokens of the prompt (a specific chunk of the robot's internal thought process) was the best way to catch these errors, beating out other methods.

The paper explicitly rules out the idea that waiting for the answer to be generated helps much. They tested methods that look at the text after the robot speaks, and those were generally weaker than checking the internal signals before the robot speaks.

The "Fix-It" Experiment

To prove this idea works, the team set up a "repair shop." They used their pre-generation signals to diagnose the problem and then sent the robot to the right mechanic:

If the signal said "Visual Evidence Failure," they used a tool to "repair" the blurry image, making it clearer, and asked the robot to try again.
If the signal said "Unknown Entity," they gave the robot a hint about who the person or animal was.
If the signal said "Missing Fact," they fed the robot a quick fact sheet about the entity.
If the signal said "Unrecallable Fact," they rewrote the question to make the entity's name super clear, helping the robot retrieve the memory.

The results were impressive. For the iNaturalist dataset (questions about plants and animals), the robots started with very low accuracy, around 4.8% to 12.1%. After this targeted repair process, their accuracy jumped to 39.6% to 46.6%. That's a massive improvement, suggesting that knowing why a robot fails allows us to fix it much better than just hoping it gets lucky next time.

What This Means (and What It Doesn't)

The paper suggests that the way these Vision-Language Models fail is structured and predictable. It's not random chaos; it's a specific breakdown in either the eyes, the recognition, or the memory.

However, the authors are careful not to claim this is a perfect, solved problem. They note that their "repair" experiment was a proof-of-concept using a powerful tool manager (GPT-5) to do the fixing. They acknowledge this introduces additional cost and potential tool errors, and that a real-world system would need cheaper, specialized tools to do the same job. Also, their labels for "failure" are based on specific tests they designed, not necessarily on a perfect human understanding of every possible error.

But the core message is clear and exciting: We don't have to wait for a robot to give a wrong answer to know it's in trouble. By listening to its internal signals before it speaks, we can tell if it's squinting at a blurry photo, staring at a stranger, or just drawing a blank on a fact. And once we know the cause, we can send it to the right repair shop to get the answer right.

Technical Summary: Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models

Problem Statement

Vision-Language Models (VLMs) demonstrate high proficiency in answering questions based on high-quality images but frequently struggle when queries require knowledge extending beyond immediate visual perception. In these knowledge-intensive Visual Question Answering (VQA) scenarios, a single incorrect answer can stem from distinct failure modes: degraded visual evidence, unrecognized entities, missing parametric facts, or failed factual recall despite entity recognition.

Prior research has largely treated these errors as monolithic failures or investigated individual failure modes (e.g., image degradation or cultural bias) in isolation. Consequently, existing uncertainty quantification methods typically provide a binary signal of "likely failure" without diagnosing the specific source of uncertainty. This lack of granularity prevents targeted interventions, such as distinguishing between an image that needs repair and a query that requires external fact retrieval.

Methodology

Attribution Taxonomy

The authors propose a unified framework that decomposes VQA errors into four mutually exclusive uncertainty components, forming an attribution tree:

Visual Evidence Failure ( $Z_d$ ): The image is too degraded (e.g., noise, blur) to support recognition.
Unknown Entity ( $Z_u$ ): The entity is visually recognizable but unknown to the model (out-of-distribution or underrepresented).
Unknown Fact ( $Z_f$ ): The entity is recognized, but the required fact is missing from the model's parametric knowledge.
Unrecallable Fact ( $Z_r$ ): The entity is recognized and the fact is assumed available, but the model fails to retrieve or use it correctly.

Data Curation and Labeling

To train probes for these specific failure modes, the authors constructed a dataset from PopVQA (celebrities, landmarks, logos, paintings) and iNaturalist (plant and animal species). They applied a scalable, model-specific labeling procedure:

Unknown vs. Degraded Entities: They used a "verbalized recognition probe" with distractor prompts to distinguish between entities the model cannot recognize due to visual degradation (induced via Gaussian blur, noise, JPEG compression) versus entities it simply does not know.
Factual Access: To separate "Unknown Fact" from "Unrecallable Fact," they employed a two-hop test. If a model fails the original image-based question but succeeds when the entity name is explicitly provided in the prompt, the failure is attributed to Unrecallable Fact (cross-modal access failure). If it fails both, the fact is deemed Unknown.

Pre-Generation Feature Extraction

The core investigation focuses on whether internal model states before decoding can predict these failure sources. The authors extracted features after the image-question prompt was processed but before answer generation began:

VIS: Hidden states of the final image token (visual-prompt features).
EOP: Hidden state of the final prompt token.
LAST8: Concatenated hidden states of the final eight prompt tokens.
ATTN: Flattened attention weights from the final eight tokens to a lookback window.
ATTN+LAST8: A combination of the above.

These features were fed into lightweight probes (linear heads or small transformers) trained to predict the specific failure branch in the attribution tree.

Key Contributions

Attribution-Tree Taxonomy: The paper introduces a structured decomposition of knowledge-intensive VQA failures into visual evidence, entity recognition, factual recall, and knowledge availability, moving beyond binary error detection.
Scalable Labeling Procedure: A method to assign specific failure modes to VQA examples using controlled probes (distractors, entity-explicit rewrites, and synthetic image degradation) rather than relying on expensive human annotation.
Pre-Generation Failure Prediction: The demonstration that VLM internal representations can predict failure sources before an answer is generated. Crucially, the paper identifies that different signals are optimal for different failure types:
- Visual-token features (VIS) are the strongest predictors for recognition and visual-evidence failures.
- Prompt-conditioned hidden states (LAST8) are superior for predicting factual failures (unknown vs. unrecallable facts) once the entity is recognized.

Experimental Results

The study evaluated four VLM families (Gemma-3, Llama-3.2, Qwen2.5-VL, Qwen3-VL) on PopVQA and iNaturalist.

Failure Distribution: "Unknown Fact" was the dominant failure mode (36–64% of errors), followed by "Unknown Entity" and "Unrecallable Fact." This confirms that many errors occur even after successful entity recognition.
Feature Performance:
- For Visual Evidence and Entity Recognition, VIS features achieved the highest PR-AUC (e.g., 97.0 for visual evidence), significantly outperforming prompt-based features.
- For Answer Success and Factual Access, LAST8 features (particularly with a Transformer head) outperformed VIS and EOP, reaching average PR-AUCs of 62.7 and 53.5 respectively.
- Tree vs. Flat Classification: A tree-composed approach (using specialized probes for each binary decision) consistently outperformed a direct 5-label classifier (49.2 vs. 41.2 macro PR-AUC), suggesting that separating image-side and factual decisions is necessary.
Cross-Dataset Generalization: Probes trained on PopVQA transferred well to iNaturalist for visual and recognition tasks (high PR-AUC) but showed lower performance for factual tasks, indicating that factual signals are more domain-specific.
Probe-Guided Mitigation: Using the attribution tree to route examples to targeted interventions (image repair, entity identification, question rewriting, or external fact provision) improved final answer accuracy by 30.8–39.3 percentage points. Notably, "Unknown Fact" was the largest contributor to these gains, validating the utility of identifying missing knowledge early.

Significance and Claims

The paper claims that VLM failures in knowledge-intensive VQA are structured rather than random. By analyzing pre-generation internal states, it is possible to diagnose why a model is uncertain before it generates an answer.

The authors assert that this diagnostic capability enables efficient, targeted interventions. Instead of a binary "abstain or answer" decision, systems can route uncertain cases to specific repair mechanisms:

If the signal indicates visual degradation, the image can be repaired.
If the signal indicates unknown entity, the system can seek entity recognition support.
If the signal indicates missing knowledge, the system can trigger external retrieval.

The work suggests that the "two-hop" nature of knowledge-intensive VQA (visual recognition followed by fact retrieval) is reflected in the model's internal representations, with visual tokens encoding the first hop and prompt-conditioned hidden states encoding the second. This finding supports the development of more robust, self-correcting multimodal systems that can identify and address their own specific limitations.