Technical Summary: "Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMs"

Problem Statement

While Vision-Language Models (VLMs) have demonstrated remarkable capabilities in multimodal reasoning, their robustness against textual misinformation that conflicts with visual evidence remains under-explored. Existing research has largely focused on hallucinations (where models generate unfaithful text spontaneously) or low-level adversarial perturbations (imperceptible image noise). However, it is unclear how VLMs arbitrate between contradictory information from different modalities when faced with persuasive, human-readable textual inputs that deliberately contradict clear visual evidence. This gap is critical for real-world deployments in autonomous driving, content moderation, and medical diagnostics, where systems must reconcile conflicting user instructions or descriptions with their visual perception.

Methodology

1. Dataset Construction: CONTEXT-VQA

To address this gap, the authors introduce CONTEXT-VQA, a benchmark designed to systematically test VLM susceptibility to textual manipulation.

• Source: Derived from the A-OKVQA dataset, filtered to a "common subset" of 920 image-question pairs where 11 state-of-the-art VLMs initially achieved 100% accuracy. This ensures that any subsequent performance drop is due to manipulation, not intrinsic difficulty.
• Misinformation Generation: A semi-automated pipeline uses a strong VLM (Gemini 2.5-Pro) to generate persuasive prompts that conflict with the ground truth visual evidence. The generation targets a specific incorrect option (the "Target") and employs four distinct rhetorical strategies:
  • Repetition: Insistently restating the incorrect fact.
  • Logical: Constructing a seemingly rational argument by misinterpreting visual details (e.g., fabricating depth maps or pixel analysis).
  • Credibility: Posing as an expert or citing authoritative (but fake) sources.
  • Emotional: Using evocative language to create emotional bias toward the incorrect answer.
• Validation: Human annotators filtered out nonsensical or offensive generated content to ensure high-quality, rhetorically diverse misinformation.

2. Evaluation Framework

The authors propose a multi-round conversational testing framework to measure robustness:

• Stage I (Baseline): Establish initial accuracy (100% for the filtered subset).
• Stage II (Persuasion): Conduct iterative dialogues where the model is presented with the original image-question pair alongside the persuasive misinformation. The entire chat history is concatenated in each round to simulate continuous conversation.
• Stage III (Final Check): Measure the final accuracy and confidence shifts after up to 4 rounds of persuasion.
• Metrics: The study tracks Accuracy@n (accuracy after round n) and analyzes the shift in confidence scores for both correct and incorrect answers.

Key Contributions

1. First Investigation of Persuasive Misinformation in VLMs: This work is the first to systematically benchmark how persuasive conversations impact a VLM's arbitration between conflicting visual and textual modalities, moving beyond text-only misinformation studies.
2. CONTEXT-VQA Dataset: A curated dataset of 920 high-confidence VQA pairs augmented with systematically generated misleading prompts across four rhetorical strategies.
3. Comprehensive Benchmarking: A rigorous evaluation framework applied to 11 state-of-the-art VLMs (both open-source and proprietary), revealing significant vulnerabilities.

Experimental Results

1. High Susceptibility to Textual Manipulation

Despite clear visual evidence, VLMs are highly vulnerable to misleading text.

• Performance Drop: After a single round of persuasive conversation, the average accuracy across all 11 models dropped by 48.2%.
• Extreme Cases: Some models, such as Gemini-2.5-Flash under logical appeals, saw accuracy plummet to 10.5%.
• Confidence Shift: Models often adopt the incorrect answer with high confidence. For questions where the model flipped its answer, the confidence in the incorrect target rose significantly (e.g., from near 0% to ~28.7% in the highest confidence interval for InternVL-3-8B), indicating the model treats the misinformation as the new truth.

2. Capability Does Not Guarantee Robustness

A surprising finding is that high initial capability (general knowledge and V-L alignment) does not correlate linearly with robustness.

• Example: QwenVL-2.5-3B had high initial capability (86.7%) but the lowest robustness (dropping to 18.3% after persuasion).
• Scaling Benefits: While scaling up parameters generally improves both capability and robustness (e.g., InternVL-3-8B outperforms smaller variants), even the strongest proprietary models (like GPT-4o) are not immune, though they show greater resilience.

3. Strategy-Specific Vulnerabilities

Different models are susceptible to different rhetorical strategies:

• Logical Appeal: The most effective strategy overall, disproportionately swaying open-source models. These models appear particularly vulnerable to sophisticated, fabricated reasoning.
• Repetition: Surprisingly effective against proprietary models, suggesting they may be overly tuned for instruction-following and susceptible to persistent, simple claims.
• Diminishing Returns: The first round of persuasion causes the most significant damage; subsequent rounds yield diminishing returns.

4. Mitigation via Prompting

The authors tested a simple, training-free defense: adding an "alarm prompt" to the system instructions (e.g., "Please carefully examine the image and ensure your answer is consistent with what you actually see").

• Results: This significantly improved performance against Repetition strategies (e.g., +16.3% for GPT-4o-mini), suggesting that reminding models to prioritize visual evidence can counteract simple, persistent misinformation.

Significance and Claims

The paper argues that current VLMs possess a critical limitation: they frequently prioritize conflicting textual input over clear visual evidence, even when the text is demonstrably false. This "obedience bias" toward text poses a severe risk for real-world applications where VLMs act as autonomous agents (e.g., in autonomous driving or medical diagnostics).

The authors conclude that:

1. Reliability is Compromised: The ability of VLMs to "see" is insufficient without the critical ability to discern when they are being told not to believe their eyes.
2. Security Gap: Textual misinformation represents a form of "social engineering" for AI, exploiting instruction-following capabilities to override perception.
3. Future Directions: While simple prompt-based defenses offer some mitigation, the paper calls for more sophisticated solutions, such as improved model architectures and fine-tuning schemes that explicitly train models to handle contradictory multimodal inputs.

The work serves as a warning that as VLMs are deployed in high-stakes environments, their susceptibility to semantic attacks must be addressed to ensure safe and reliable operation.