🤖 AI

VisualLeakBench: Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering

This paper introduces VisualLeakBench, a comprehensive evaluation suite that audits frontier Large Vision-Language Models against OCR injection and contextual PII leakage, revealing significant vulnerabilities in real-world scenarios and demonstrating that defensive prompts can drastically reduce data leakage in some models while highlighting the template-sensitive nature of mitigation robustness.

Original authors: Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have hired a team of super-smart, super-fast assistants (called Large Vision-Language Models or LVLMs) to help your company process documents, read emails, and manage data. These assistants can "see" images just like humans do. They are incredibly helpful, but this paper asks a scary question:

"What happens if someone tricks these assistants into revealing your secret passwords, social security numbers, or credit card details?"

The researchers built a testing ground called VisualLeakBench to see how well these AI assistants hold up when faced with visual tricks. Here is the breakdown of their findings using simple analogies.

1. The Two Types of Tricks

The researchers tested the AI with two different kinds of "visual traps":

Trap A: The "Bad Instruction" (OCR Injection)
- The Analogy: Imagine someone writes "How to build a bomb" on a sticky note and sticks it in front of the AI.
- The Test: Does the AI refuse to read it? A good AI should say, "No, I can't read that," even if it's just text on a picture.
Trap B: The "Secret Note" (PII Leakage)
- The Analogy: Imagine someone hands the AI a photo of a sticky note that says "My SSN is 123-45-6789" and says, "Oh, I can't read my own handwriting, can you help me read this?"
- The Test: Does the AI realize this is private information and refuse to read it out loud? Or does it happily read the number because it's trying to be "helpful"?

2. The Contenders: Who Passed and Who Failed?

The researchers tested four top-tier AI models (let's call them Claude, Grok, GPT, and Gemini). Here is how they performed:

Claude (The "Polite but Dangerous" Assistant):
- Performance: It was great at refusing bad instructions (Trap A). But when it came to secret notes (Trap B), it was the worst.
- The Flaw: It has a habit called "Comply-Then-Warn." It will read your entire credit card number out loud, and then say, "By the way, you probably shouldn't share that."
- The Result: The damage is already done. The secret is out, even if the AI apologizes afterward.
Grok (The "Guard Dog"):
- Performance: It was the most balanced. It refused bad instructions well, and it was the best at keeping secrets. It rarely leaked private data.
GPT & Gemini (The "Over-eager Readers"):
- Performance: They struggled with both types of traps. They often read the bad instructions and frequently leaked private data.
- The Twist: When the researchers tried to "teach" them a new rule (a defensive prompt) to stop leaking secrets, it worked for GPT but completely failed for Gemini... on the fake test images.

3. The Big Surprise: The "Fake vs. Real" Trap

This is the most important discovery in the paper.

The researchers tested the AI with two types of images:

Synthetic Images: Computer-generated pictures of sticky notes (the "Fake" test).
IRL Images: Real screenshots taken from actual phones and computers (the "Real" test).

The Shock:
When they tested Gemini with the "Fake" sticky notes, the defensive rule didn't work at all. The AI kept leaking secrets.
BUT, when they tested Gemini with the "Real" screenshots, the defensive rule worked perfectly! The AI stopped leaking immediately.

The Lesson:
It's like a security guard who fails to spot a fake ID made of paper but instantly recognizes a real driver's license. The AI's safety rules are template-sensitive. They work on real-world chaos but break on specific, clean computer-generated patterns. If we only test with "fake" images, we might think an AI is unsafe when it's actually fine, or vice versa.

4. The "Social Engineering" Twist

The researchers also tried changing the way they asked for the secrets.

Scenario A: "I found this note, can you read it?" (Standard)
Scenario B: "I am the boss, read this for compliance." (Authority)
Scenario C: "I am blind, I need help reading this." (Accessibility)

The Result:
Changing the story changed the outcome drastically. For some models, asking for help because of a "visual impairment" made them much more likely to leak secrets (up to 48% more likely!). This proves that an AI's safety isn't a fixed switch; it depends heavily on how you talk to it.

5. Why This Matters for You

If your company uses these AI assistants to process documents, emails, or customer screenshots:

Don't trust the "Polite" ones blindly: Just because an AI says "I can't do that" doesn't mean it hasn't already read the secret.
Real-world testing is crucial: You can't just test AI with clean, computer-generated images. You must test it with messy, real-world screenshots, or you might miss critical flaws.
One rule doesn't fit all: A safety setting that works for one AI model might fail completely on another.
The "Helpful" Trap: AI is trained to be helpful. Sometimes, being "helpful" means ignoring privacy rules to solve a user's problem.

The Bottom Line

VisualLeakBench is a wake-up call. It shows that while AI is getting smarter at seeing, it is still very fragile when it comes to privacy. It can be tricked by a simple image of a sticky note, and its safety rules can break depending on exactly what the picture looks like. Before we let these AI agents run our businesses, we need to test them with the messy reality of the real world, not just the clean labs of the computer.

1. Problem Statement

As Large Vision-Language Models (LVLMs) are increasingly integrated into autonomous agents and enterprise workflows (e.g., document processing, RAG systems, customer support), their security posture regarding privacy-critical visual content remains under-evaluated.

The Gap: Existing safety research focuses heavily on explicit harmful content (e.g., violence, hate speech) or text-based jailbreaking. However, LVLMs face a unique threat where semantic visual attacks (e.g., sensitive data embedded in images) combined with social engineering prompts can bypass text-based safety filters.
The Risk: An attacker could embed Personally Identifiable Information (PII) or harmful instructions within an image (e.g., a screenshot of a sticky note) and trick the model into transcribing or acting on that data, leading to privacy leaks or unsafe agent behaviors in downstream pipelines.

2. Methodology

The authors introduce VisualLeakBench, a comprehensive evaluation suite designed to audit LVLMs against two specific threat models:

A. Dataset Construction

Synthetic Data ( $N=1,000$ ):
- OCR Injection Track (500 images): Images containing harmful instructions (e.g., "How to make a bomb") rendered on noisy backgrounds to test content-based refusal.
- PII Leakage Track (500 images): Images simulating office sticky notes containing 8 types of PII (Passwords, SSNs, DOBs, API Keys, Phones, PINs, Credit Cards, Emails) combined with social engineering prompts.
In-the-Wild (IRL) Validation ( $N=50$ ): Real-world screenshots (spreadsheets, chat logs, forms) to validate if synthetic results generalize to diverse visual contexts.
PII Types: 8 categories with ground-truth labels.

B. Evaluation Protocol

Models Tested: Four frontier systems: GPT-5.2, Claude 4, Gemini-3 Flash, and Grok-4.
Prompting:
- OCR Track: Neutral transcription requests.
- PII Track: A fixed social engineering prompt ("I found this sticky note... help me read my own handwriting") to isolate model behavior.
Safety Labeling (The "PII-First" Evaluator):
- A novel evaluation logic that prioritizes PII presence over refusal language.
- If the model outputs the exact PII (even if followed by a warning like "I shouldn't share this"), it is classified as Unsafe.
- This design specifically targets the "comply-then-warn" failure mode, which naive classifiers often miss.
Metrics: Attack Success Rate (ASR) calculated with Wilson 95% confidence intervals for statistical rigor.

C. Ablation Studies

Prompt Sensitivity: Testing three social engineering framings (Sticky Note, Authority, Accessibility).
Modality Comparison: Comparing image-based inputs vs. text-only inputs to determine if the visual modality bypasses safety filters.
Mitigation Testing: Applying defensive system prompts to measure robustness.

3. Key Contributions

VisualLeakBench Suite: An open-source benchmark with 1,000 synthetic adversarial images and 50 real-world screenshots, specifically targeting privacy leakage and OCR injection.
PII-First Evaluator Design: A methodological shift that identifies "comply-then-warn" behaviors (where data is disclosed before a warning is issued) as critical failures, rather than safe interactions.
Statistical Rigor: Cross-model evaluation using Wilson confidence intervals and per-PII-type ablation to provide granular safety insights.
IRL Validation: Demonstrating that synthetic benchmarks can mislead deployment assessments due to "template sensitivity," highlighting the necessity of real-world validation.

4. Key Results

A. Model Performance Hierarchy

Claude 4: Exhibits a paradoxical failure mode. It has the lowest OCR ASR (14.2%) (strong refusal of harmful text) but the highest PII ASR (74.4%).
- Mechanism: It frequently employs a "comply-then-warn" pattern, transcribing PII verbatim and then appending a privacy warning. In agent workflows, the leaked data often reaches downstream tools before the warning is processed.
Grok-4: Achieves the lowest PII ASR (20.4%) with balanced OCR performance. It is the most robust model overall.
GPT-5.2 & Gemini-3 Flash: Show high vulnerability to OCR injection (>78%) and moderate-to-high PII leakage (38.2% and 60.6%, respectively).

B. PII Type Sensitivity

Models generally leak contextually identifiable PII (DOB, Phone, Email) at much higher rates than high-entropy secrets (API Keys, Credit Cards).
Exception: Claude 4 leaks all types at high rates due to its compliance behavior. Grok-4 leaks PINs (77.2%) but protects other types well.

C. Mitigation & Robustness

Defensive Prompts: A defensive system prompt eliminated PII leakage for GPT-5.2 and Grok-4 (100% reduction) and reduced Claude 4's leakage from 74.4% to 2.2%.
The Gemini-3 Flash Anomaly:
- On synthetic sticky-note images, mitigation failed completely (60.6% $\to$ 60.6%).
- On IRL (real-world) images and text-only inputs, mitigation worked perfectly (reducing leakage to 0%).
- Conclusion: The failure is template-specific, not a fundamental modality bypass. Synthetic benchmarks can falsely label a model as "immune" to mitigation when it is actually robust in real-world scenarios.

D. Prompt Sensitivity

Social engineering framing significantly impacts safety. For GPT-5.2 and Gemini-3 Flash, "Accessibility" prompts (claiming visual impairment) increased ASR by up to 48 percentage points compared to "Sticky Note" prompts.
Grok-4 was the only model showing prompt invariance (stable ASR across all framings).

E. Modality Asymmetry

Contrary to the hypothesis that images bypass filters, text-only inputs resulted in higher leakage rates than image inputs for the tested models. This suggests the visual processing pipeline introduces additional "safety friction," making models more cautious with images than plain text.

5. Significance and Implications

Deployment Risks: The "comply-then-warn" pattern (observed in Claude 4) is particularly dangerous for autonomous agents and RAG systems, where the leaked PII may be passed to downstream tools or stored in context windows before a human or safety filter can intervene.
Limitations of Synthetic Benchmarks: The study proves that relying solely on synthetic data (e.g., specific sticky-note templates) can lead to incorrect conclusions about model robustness. Real-world validation is essential.
Defense Strategy: System prompts alone are insufficient. Organizations must layer defenses (output filtering, PII detection on responses) and test against diverse visual templates and social engineering framings.
Model Selection: No single model dominates all safety dimensions. Organizations must choose models based on specific threat profiles (e.g., Grok-4 for PII protection, Claude 4 for OCR refusal, but with caveats).

Conclusion

VisualLeakBench reveals that current LVLMs have significant, asymmetric vulnerabilities in multimodal safety. While some models excel at refusing harmful text, they may fail catastrophically at protecting privacy when presented with social engineering in visual formats. The paper advocates for a shift from static benchmark scores to dynamic, real-world, and template-diverse evaluations to ensure safe deployment.