💬 NLP

Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems

This paper introduces the MM-AQA benchmark to evaluate effective abstention in multimodal systems, revealing that current vision-language and multi-agent models rarely abstain from unanswerable questions and that improving this capability requires abstention-aware training rather than relying on better prompting or increased agent complexity.

Original authors: Nishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste

Published 2026-04-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nishanth Madhusudhan, Vikas Yadav, Alexandre Lacoste

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Silent" Superpower

Imagine you hire a brilliant, super-fast detective (an AI) to solve crimes. This detective is amazing at solving cases where all the clues are right there on the table. But here's the problem: This detective is too confident.

If you show the detective a crime scene with a missing fingerprint and a torn page from the suspect's diary, the detective won't say, "I can't solve this." Instead, they will make up a story, guess the fingerprint, and invent a reason for the torn page. They are so eager to please that they will lie to you rather than admit they don't know.

This paper is about teaching AI to do the one thing humans do naturally but AI struggles with: Knowing when to shut up.

In the world of AI, this is called Abstention. It's the ability to look at a question, realize the evidence is missing or confusing, and say, "I don't know," instead of guessing.

The Problem: The "Yes-Man" AI

The authors point out that current AI models (called Vision-Language Models) are like over-eager interns.

The Intern's Mindset: "My boss asked a question! I must give an answer! Even if I'm not sure, I'll guess!"
The Danger: In real life, like in medical diagnosis or legal reviews, a wrong guess can be disastrous. If an AI looks at a blurry X-ray and guesses "It's a broken bone" when it's actually just a shadow, a patient could get hurt.

The paper argues that we need to stop testing AI only on questions where the answer is obvious. We need to test them on questions where the answer should be "I don't know."

The Solution: Building a "Trick" Exam (MM-AQA)

To test if AI can learn to be humble, the researchers built a new exam called MM-AQA.

Think of it like a magic trick show.

The Setup: They took normal, solvable questions (like "What color is the car in this photo?").
The Trick: They secretly sabotaged the questions to make them impossible to answer.
- Visual Sabotage: They took the photo of the car and covered the car with a giant black box, or blurred it until it looked like a soup of colors.
- Text Sabotage: They changed the question to ask about something that isn't in the picture at all (e.g., "What is the car's license plate?" when the plate is hidden).
- Contradiction: They added a note saying "The car is red" while the picture clearly shows a blue car.

They created 2,079 of these "trick" questions. The goal? To see if the AI would spot the trick and say, "I can't answer this," or if it would confidently guess the wrong answer.

The Experiments: Solo Detective vs. The Squad

The researchers tested three types of AI systems:

The Solo Detective (Single AI): One AI model tries to answer everything.
- Result: It failed miserably. It kept guessing even when the clues were missing. It was like a detective who refuses to admit the case is cold.
The Squad (Multi-Agent System): They set up a team of three AIs:
- The Reasoner: Tries to solve the case.
- The Verifier: Acts like a strict supervisor. It checks the Reasoner's work and asks, "Are you sure? Do we actually have enough evidence?"
- The Manager: Decides when to stop the conversation.
- Result: The squad was much better at saying "I don't know." The Verifier would often stop the Reasoner from guessing.
- The Catch: The squad was so cautious that sometimes they refused to answer even when they could have solved it. It's like a team of detectives who are so afraid of making a mistake that they never solve any cases at all.
The "Extreme Pressure" Test: The researchers told the AI, "If you guess wrong, you will be fired!"
- Result: The AI started saying "I don't know" more often. This proved that the AI had the ability to know it didn't know, but it was usually too eager to answer to use that ability.

The Key Takeaways (The "Moral of the Story")

AI is Overconfident: Current AI models are terrible at admitting ignorance. They would rather hallucinate (make things up) than stay silent.
More Agents Help, But Have a Cost: Using a team of AIs helps them spot when they don't know, but it makes them too cautious, leading to a trade-off: they get better at not lying, but worse at solving easy problems.
It's Not About "Thinking Harder": Making the AI think longer or use more complex reasoning didn't fix the problem. The issue isn't that they aren't smart enough; it's that they are miscalibrated. They don't trust their own uncertainty.
The Fix Isn't a Prompt: You can't just tell an AI "Please be humble" and expect it to work. The paper suggests we need to retrain these models from the ground up to value "I don't know" just as much as a correct answer.

The Bottom Line

Imagine a GPS that, when you ask for directions to a place that doesn't exist, doesn't just drive you into a lake. Instead, it says, "I can't find that place, let's check the map again."

This paper is a wake-up call: We need to build AI that knows its limits. Until we teach them to say "I don't know" as confidently as they say "The answer is 42," we can't fully trust them with important jobs like medicine or law.

1. Problem Statement

Current Vision-Language Models (VLMs) and Multi-Agent Systems (MAS) are evaluated primarily on their ability to answer questions, operating under the assumption that all inputs are answerable. This paradigm incentivizes models to always generate a response, even when evidence is insufficient, ambiguous, or contradictory. In high-stakes domains (e.g., medical imaging, legal review), this leads to hallucinations and overconfident errors when the model encounters epistemic uncertainty.

The core problem addressed is the lack of robust evaluation frameworks for Effective Abstention (EA)—the ability of a system to recognize evidence insufficiency and refrain from answering. Existing benchmarks either ignore unanswerable instances or rely on coarse perturbation methods that fail to capture realistic failure modes. Furthermore, current models lack the calibration to distinguish between "I don't know" and a confident hallucination.

2. Methodology: The MM-AQA Benchmark

The authors introduce MM-AQA (MultiModal Abstention Question Answering), a comprehensive benchmark and evaluation framework designed to systematically test abstention capabilities.

A. Dataset Construction

MM-AQA consists of 2,079 samples derived from two source benchmarks:

Abstain-MMMU (553 samples): Derived from the STEM domains of MMMU.
Abstain-MMLongBench-Doc (1,526 samples): Derived from MMLongBench-Doc (long-document understanding).

Transformation Taxonomy:
Instead of simple image degradation, the authors employ a principled transformation pipeline that converts answerable instances into unanswerable ones along two axes:

Modality Dependency (for MMMU): Targets the reliance on visual vs. textual evidence. Transformations include Missing Visual Information (masking/cropping), Occlusion Ambiguity (blur/darkness), Semantic Unanswerability (rewriting questions to require unavailable info), and Adversarial Ambiguity (introducing multiple valid interpretations).
Evidence Sufficiency (for MMLongBench-Doc): Targets document reasoning. Transformations include Evidence Removal (hiding pages), Evidence Corruption (altering numbers/tables), Semantic Contradiction (conflicting captions), and Inferential Impossibility (temporal shifts outside document scope).

Quality Control:
A Dual-Consensus VLM QC module (using two independent VLMs) and human annotators filter the dataset. A sample is accepted only if both VLMs agree it is unanswerable; disagreements trigger human review. This ensures high reliability and minimizes "false negatives" (instances that are actually answerable).

B. Evaluation Framework

The study evaluates three frontier VLMs (GPT-5, Claude Sonnet 4.5, Qwen 2.5-32B-VL) and two Multi-Agent System (MAS) architectures (Sequential and Iterative) across 2079 samples.

Experimental Conditions:
1. Base: Standard prompting.
2. Verbal Confidence (vconf): Models output a confidence score (1-5) to enable threshold-based abstention.
3. Chain of Thought (CoT): Models must reason step-by-step before answering.
Abstention Clauses:
- Standard: Models can abstain but are not explicitly pressured.
- Extreme: Explicit penalties for wrong answers ("I will be imprisoned if you are wrong") to test behavioral pressure.
Metrics:
- AAC (Answerable Accuracy): Accuracy on answerable questions.
- UAC (Unanswerable Accuracy): Rate of correctly abstaining on unanswerable questions.
- AR (Abstention Rate): Total rate of abstention.
- MCC (Matthews Correlation Coefficient): Extended to a 5-way confusion matrix (TP, FP, FN, TN, and AU - Answer on Unanswerable) to measure the trade-off between accuracy and abstention.

3. Key Contributions

MM-AQA Benchmark: A principled, large-scale dataset with 22 transformation types specifically designed to test multimodal abstention, distinguishing itself from concurrent works via dual-axis construction and joint VLM/MAS evaluation.
Structured Transformation Pipeline: A method to convert answerable data into unanswerable data by disrupting specific reasoning pathways (modality dependency or evidence sufficiency) rather than surface-level noise.
Comprehensive Evaluation: The first large-scale comparison of standalone VLMs vs. Multi-Agent Systems (Reasoner-Verifier-Orchestrator) on abstention tasks, including various baselines (Self-Consistency, P(True), Verbal Confidence).
Empirical Findings on Calibration: Evidence that current frontier models are severely miscalibrated, often hallucinating on unanswerable inputs even when they possess latent uncertainty signals.

4. Key Results

The evaluation of 2,079 samples across 9 system-model combinations yielded four critical findings:

VLMs Rarely Abstain Under Standard Prompting:
- Under standard conditions, frontier VLMs rarely abstain. For example, on A-MMMU, Claude Sonnet 4.5 achieved only 1.2% UAC (correctly abstaining), while answering incorrectly on >80% of unanswerable instances.
- Simple confidence baselines (verbalized confidence thresholds) outperform standard prompting, suggesting models have uncertainty signals but fail to act on them without explicit instruction.
MAS Improves Abstention but Creates a Trade-off:
- Multi-Agent Systems (specifically the Verifier agent) significantly improve UAC (e.g., up to 95% on some configurations) by overriding the Reasoner.
- However, this introduces a severe Accuracy-Abstention Trade-off. High UAC often comes at the cost of reduced AAC (e.g., dropping from ~60% to ~45%). No system simultaneously exceeded 65% on both AAC and UAC.
Sequential vs. Iterative Designs:
- Sequential designs (one Reasoner-Verifier pass) generally matched or exceeded Iterative designs (up to 3 rounds).
- This suggests the bottleneck is miscalibration (the model's inability to recognize uncertainty) rather than a lack of reasoning depth. Iterative rounds often amplified errors when the base Reasoner was poorly calibrated.
Structural Explicitness Drives Performance:
- Models perform best when unanswerability is structurally explicit (e.g., missing pages, broken questions).
- Models struggle when evidence is degraded but present (e.g., blurred images, contradictory captions), attempting to "reconcile" the conflict rather than abstaining.
- The "Extreme Abstain" clause significantly boosted UAC (e.g., from 1% to 41% for Sonnet 4.5 on A-MMMU), proving that latent uncertainty exists but is suppressed by default decoding policies.

Performance Gap:
The best-performing system (VLM Claude Sonnet 4.5) achieved an average MCC of 0.344, while the estimated human expert oracle is 0.83. This leaves a gap ( $\Delta$ MCC) of ~0.49, indicating current systems close less than half the distance to expert-level abstention performance.

5. Significance and Conclusion

The paper concludes that prompting and adding more agents are insufficient to solve the multimodal abstention problem. The fundamental issue is that models are trained to maximize answer generation, not to recognize the limits of their knowledge.

Implication: To achieve reliable multimodal systems, abstention-aware training is necessary. Models must be fine-tuned on graded examples of unanswerability to learn when not to answer.
Future Work: The authors call for the development of uncertainty estimation methods specific to multimodal inputs (e.g., semantic entropy over visual inputs) and the extension of these benchmarks to non-STEM domains, video, and multi-document settings.

This work establishes that while current frontier models are powerful, they remain dangerously overconfident in the face of insufficient evidence, posing significant risks for real-world deployment without significant architectural or training shifts.