Technical Summary: VisualLeakBench

Problem Definition

Vision-language agents (VLAs) are increasingly integrated into agentic workflows where they process screenshots, documents, and user interfaces to execute downstream actions such as saving notes, sending emails, or querying search engines. While prior safety research has focused on response-level leakage (what the model says to a user), this paper identifies a distinct and critical failure mode: action-boundary propagation.

This failure occurs when a vision-language agent extracts sensitive or unsafe visible text from an input image and propagates it directly into the arguments of a tool call. Unlike a refusal failure where the model simply declines a request, this failure is often invisible to response-level inspection because the model's visible output may be benign, while the dangerous content is embedded in the structured tool arguments (e.g., save_note(content="...API_KEY...")). This allows sensitive data to persist in memory, leave the local context, or become input to other systems without the user's explicit knowledge.

Methodology

The authors introduce VisualLeakBench, a reproducible benchmark designed to measure and diagnose these propagation failures.

Benchmark Design

• Dataset: A diversified set of 500 images spanning five visual families: UI, chat, document, form, and dashboard.
• Tracks:
  1. PII Track: Contains eight synthetic Personally Identifiable Information types (SSNs, credit cards, emails, API keys, etc.).
  2. Rendered Unsafe-Text Track: Contains synthetic harmful requests (e.g., phishing, fraud, sabotage) rendered as visible text within ordinary visual contexts.
• Evaluation Protocol: The study evaluates a stratified 100-image subset using four production VLM systems (GPT-5.4, Claude Opus 4.7, Gemini 3.1 Flash Lite, and Grok 4.20).
• Workflows: Two single-step tool-use scenarios are tested:
  1. Note Capture: The agent calls save_note(content).
  2. External Handoff: The agent calls send_email(to, body).
  3. Web Research: A third tool surface (search_query) was tested to evaluate tool-surface specificity.
• Metrics: The primary metric is the tool propagation rate, defined as the fraction of non-error traces where the normalized target string appears in the tool arguments. The evaluation distinguishes between "response-only leakage," "tool-only propagation," and "response-plus-tool leakage."

Mitigation Strategy

The authors test a lightweight defensive system prompt instructing models not to transcribe PII and to refuse social-engineering requests. Additionally, an offline labeled-target oracle diagnostic is used to simulate a perfect guard at the tool boundary, blocking any tool call containing the known synthetic target string. This serves as an upper-bound diagnostic to isolate the value of controlling the tool boundary.

Key Results

High Propagation Rates at Baseline

Without defensive prompting, action-boundary propagation is severe:

• PII: Propagated in 78.8% of non-error traces. Rates varied by model (e.g., up to 95.7% for Claude in note capture) and scenario (higher in note capture than external handoff).
• Rendered Unsafe Text: Propagated in 85.5% of non-error traces.

Asymmetric Mitigation Efficacy

The defensive prompt yielded highly asymmetric results:

• PII: Propagation dropped significantly from 78.8% to 2.0%. However, the authors note this was largely achieved by suppressing tool use entirely (94.2% of traces resulted in "no tool call") rather than sanitizing the arguments while preserving utility.
• Rendered Unsafe Text: Propagation remained high at 52.6%. The mitigation failed to prevent the majority of unsafe text from crossing the tool boundary, suggesting that models treat rendered harmful text as content to be preserved or forwarded rather than as a safety violation to be refused.

Trace Diagnostics and Tool Surface Specificity

• Tool-Only Dominance: The majority of failures (78.1% for PII, 83.4% for unsafe text) were "tool-only" propagations, where the visible response was safe but the tool arguments contained the target. This confirms that response-level inspection underestimates agent risk.
• Tool Surface Dependence:
  • Search Tools: Search-like tools (web research) almost never propagated PII (1.0% baseline), likely because search queries are keyword-oriented and do not naturally invite preserving full contact details.
  • Note/Handoff Tools: These tools naturally invite preserving visible content, leading to higher propagation rates.
  • Unsafe Text Persistence: Even with search tools, rendered unsafe text remained a residual risk (54.5% baseline), indicating that harmful instructions can cross action boundaries regardless of tool semantics.

Oracle Diagnostics

The oracle diagnostic revealed that blocking tool arguments containing the target string removes the vast majority of propagation failures (e.g., 335/341 unsafe traces caught). However, a small residual risk remains from response-side leakage, highlighting that the tool boundary is a critical, but not exclusive, intervention point.

Significance and Claims

The paper positions VisualLeakBench as a controlled failure primitive for studying agentic safety, rather than a claim that all deployed agents lack memory filters or human checkpoints.

1. Redefining the Failure Unit: The work shifts the unit of analysis from response-level chat safety to action-boundary propagation. It demonstrates that a model can be "safe" in its visible response while simultaneously executing an unsafe action by embedding sensitive data into tool arguments.
2. Diagnostic Granularity: By separating trace classes (response-only vs. tool-only), the study reveals that standard response inspection is insufficient for agent safety evaluation.
3. Mitigation Asymmetry: The results highlight that recognizable PII forms are easier to suppress (often via tool suppression) than rendered unsafe text, which models often misinterpret as benign content to be preserved.
4. Reproducibility: The authors provide a comprehensive artifact including per-trace logs, normalization rules, and scripts to regenerate aggregate tables, allowing for precise attribution of failures to specific visual families, models, and tool surfaces.

The authors conclude that future agent evaluations must treat tool arguments as first-class safety artifacts and report trace-level diagnostics alongside response-level outcomes to fully capture the risk landscape of vision-language agents.