Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations

1. Problem Statement

Vision-Language Models (VLMs) are increasingly deployed in scenarios where users cannot directly observe the visual context, such as assisting blind and low-vision individuals or autonomous agents. In these settings, users rely on the model's natural language explanations to judge the reliability of its predictions.

However, current VLMs often suffer from hallucinations (generating non-existent visual details) and overconfidence. Prior research indicates that plausible-sounding natural language explanations can mislead users into trusting incorrect predictions, a phenomenon exacerbated by information asymmetry (the user cannot see the image to verify the claim). Existing explanation quality metrics (e.g., Simulatability, Informativeness, Plausibility) are designed for text-only Large Language Models (LLMs) and fail to account for the relationship between the explanation and the visual input, resulting in poor calibration with actual model correctness.

Core Challenge: How can we provide users with reliable signals to distinguish between accurate and inaccurate VLM predictions when they lack access to the visual context?

2. Methodology

The authors propose two novel, complementary explanation quality metrics specifically designed for VLMs: Visual Fidelity and Contrastiveness. These metrics are evaluated using training-free, LLM-as-a-judge pipelines.

A. Visual Fidelity (VF)

• Definition: Measures how faithfully an explanation reflects the actual contents of the image. It detects if the explanation relies on hallucinated details.
• Algorithm:
  1. Question Generation: An LLM ($m_{QGen}$) analyzes the explanation and generates a set of "Yes/No" verification questions ($Q_V$) corresponding to the visual details mentioned in the text.
  2. Verification: A verifier VLM ($m_{Verif}$) answers these questions based only on the image.
  3. Scoring: The VF score is the fraction of questions answered "Yes."
     $$S_{VF}(E) = \frac{\sum \mathbb{1}\{a^V_i = \text{"yes"}\}}{|Q_V|}$$

B. Contrastiveness (Contr.)

• Definition: Measures how well the explanation distinguishes the model's prediction from plausible alternative answers. It assesses if the explanation provides sufficient evidence to rule out other options.
• Algorithm:
  1. Masking: Mentions of specific answer choices in the explanation are masked to prevent label leakage.
  2. Hypothesis Generation: For every possible answer $a_j$ in the set of options $A$, a paraphraser LLM converts the (Question, Answer) pair into a declarative hypothesis $h_j$.
  3. Entailment: An entailment model ($m_{NLI}$) calculates the probability that the masked explanation entails each hypothesis.
  4. Scoring: The Contrastiveness score is the relative entailment probability of the predicted answer compared to all alternatives.
     $$S_{Contr.}(E) = \frac{P_{NLI}(P \text{ entails } h_0)}{\sum_{a_j \in A} P_{NLI}(P \text{ entails } h_j)}$$

C. Combination

The authors evaluate combining these scores via average, product, and minimum functions. The Product ($S_{Prod} = S_{VF} \times S_{Contr.}$) is found to be the most effective single metric.

3. Key Contributions

1. Novel Metrics: Introduction of Visual Fidelity and Contrastiveness, which explicitly bridge the gap between textual explanations and visual context.
2. Training-Free Evaluation: The proposed scoring functions do not require ground-truth quality annotations; they rely on off-the-shelf LLMs and VLMs for verification and entailment.
3. User-Centric Validation: A comprehensive user study demonstrating that these metrics directly improve human decision-making in "blind" VLM scenarios.
4. Calibration Analysis: Evidence that existing text-only metrics are poorly calibrated for VLMs, whereas the proposed metrics show strong correlation with prediction correctness.

4. Experimental Results

A. Metric Calibration (Automated Evaluation)

Evaluated on A-OKVQA, VizWiz, and MMMU-Pro using models LLaVA-v1.5, Qwen2.5-VL, and GPT-4o:

• Discriminability: The proposed metrics (especially the Product of VF and Contr.) achieved significantly higher discriminability scores than text-only baselines (Simulatability, Informativeness, Plausibility). This means they better separate correct predictions from incorrect ones.
• Expected Calibration Error (ECE): The proposed metrics showed lower ECE, indicating that the quality score is a more accurate proxy for the probability of the prediction being correct.
• Robustness: The metrics remained stable even when using smaller open-source models as verifiers instead of GPT-4o.

B. User Study (Human Evaluation)

• Setup: 30 participants evaluated 100 VLM predictions per dataset (A-OKVQA, VizWiz) without seeing the images. They were shown the question, the VLM's answer, the explanation, and optionally, quality scores.
• Key Findings:
  • Accuracy Improvement: Showing the Product of VF and Contrastiveness improved user accuracy in judging VLM correctness by 11.1% compared to showing explanations alone.
  • Reduced Over-Reliance: The rate of users falsely believing incorrect predictions (Over-Reliance) decreased by 15.4%.
  • Presentation Format:
    • Numeric vs. Descriptive: Descriptive text summaries of the scores performed comparably to or slightly better than raw numeric scores in reducing over-reliance.
    • Labeling: The specific label (e.g., "Visual Fidelity" vs. "AI Confidence") had negligible impact; the utility came from the distribution of the score itself.
  • Correlation: There was a strong negative correlation between the calibration error (ECE) of the score and user decision accuracy (higher ECE $\rightarrow$ lower user accuracy).

5. Significance and Future Directions

• Trust Calibration: The paper demonstrates that providing explanation quality scores is a viable strategy for "calibrating" human trust in AI, particularly for vulnerable populations (blind/low-vision users) who cannot verify visual inputs.
• Beyond Text-Only Metrics: It establishes that evaluation metrics for VLMs must inherently consider the visual modality; text-only plausibility is insufficient and potentially dangerous.
• Future Work: The authors suggest developing adaptive systems that dynamically decide when to show explanations and scores (using bandit algorithms) and exploring multilingual generalization.

Conclusion: The paper successfully argues that "believing without seeing" requires more than just a plausible explanation. By quantifying Visual Fidelity (is the detail real?) and Contrastiveness (does this rule out other answers?), we can provide users with reliable signals to avoid over-reliance on hallucinating VLMs.