💻 computer science

One Perturbation, Two Failure Modes: Probing VLM Safety via Embedding-Guided Typographic Perturbations

This paper reveals that multimodal embedding distance strongly predicts the success of typographic prompt injection attacks on Vision Language Models by simultaneously influencing text readability and safety alignment, and leverages this insight to develop a model-agnostic red teaming tool that optimizes perturbations to bypass safety filters.

Original authors: Ravikumar Balakrishnan, Sanket Mendapara

Published 2026-04-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ravikumar Balakrishnan, Sanket Mendapara

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot assistant (a Vision-Language Model, or VLM) that can look at pictures and read the text inside them. You can give it instructions by writing them on a piece of paper and taking a photo. This is how it works in a safe world.

But what if someone tries to trick the robot? They write a harmful command on the paper, but they make the text look weird—maybe it's tiny, blurry, or turned sideways. This is called a Typographic Prompt Injection. The goal is to sneak a bad instruction past the robot's safety filters by hiding it in plain sight within a messy image.

This paper asks a simple question: Why do some messy pictures trick the robot, while others don't? And can we use that knowledge to test how safe the robot really is?

Here is the breakdown of their findings, using some everyday analogies:

1. The "Distance" Meter

The researchers discovered a hidden ruler that predicts whether a trick will work. They call it Embedding Distance.

Think of the robot's brain as having two different "languages": one for seeing pictures and one for reading words.

When the text in the image is clear, the robot's "picture language" and "word language" are standing right next to each other, holding hands. They understand each other perfectly.
When the text is blurry, tiny, or rotated, the robot gets confused. Its "picture language" and "word language" drift far apart in a mental space.

The paper found a strong rule: The farther apart these two languages are, the less likely the robot is to follow the command.

High Distance (Far apart): The robot thinks, "I can't read this," or "This doesn't make sense," and it ignores the command.
Low Distance (Close together): The robot thinks, "Ah, I see what this says," and it follows the instruction—even if that instruction is dangerous.

2. The Two Ways to Break the Robot

The researchers found that when they tried to "fix" the messy images to make the robot's two languages stand closer together again, two different things happened. It's like trying to fix a broken radio; sometimes you just need to turn up the volume, and other times you need to convince the radio to ignore its "Do Not Play" list.

Mode A: The "Can You Hear Me?" Fix (Readability)
Sometimes the text is so blurry or tiny that the robot literally cannot read it. It's like trying to read a sign through a thick fog.

The Fix: The researchers used a special math trick to nudge the pixels just enough so the robot's "eyes" could finally make out the letters.
The Result: The robot suddenly realizes, "Oh, I can read this now!" and follows the command. This happened a lot with heavy blur and tiny fonts.

Mode B: The "Don't Say No" Fix (Safety Bypass)
Sometimes the robot can read the text perfectly fine, but it refuses to do it because its safety rules say, "No, that's a bad request."

The Fix: The researchers nudged the image slightly. This didn't make the text clearer (it was already clear), but it changed the "vibe" or "shape" of the image in the robot's mind just enough to confuse its safety guard.
The Result: The robot still sees the bad request, but its safety filter gets confused and says, "Okay, I guess I can do this." This happened a lot with rotated text.

3. The "Red Teaming" Tool

The authors built a tool (a "Red Team" tool, which is like a professional hacker hired to test security) that uses this "Distance Meter."

Instead of guessing how to trick the robot, the tool automatically tweaks the messy images to bring the robot's "picture language" and "word language" closer together. It does this without needing to see the robot's internal code or safety rules. It just tries to make the image "look more like the text" to a set of other AI models acting as stand-ins.

What they found:

When they ran this tool on different robots (like GPT-4o, Claude, Mistral, and Qwen), it successfully made the robots follow commands they previously refused.
The Catch: For some robots, the tool worked by making the text readable. For others, it worked by confusing the safety filter. It depends on how strong the robot's safety guard is and how messy the original image was.

The Bottom Line

The paper concludes that safety isn't just about what the image looks like to a human. It's about how the image is represented inside the robot's brain.

If you can make a messy image look "closer" to the text in the robot's internal math, you can often trick it into ignoring its safety rules. This means future safety systems need to be tough not just against blurry pictures, but against these subtle internal "distances" that confuse the robot's understanding.

Limitations: The researchers only tested this on specific types of text (black on white) and a specific set of bad instructions. They also noted that their tool takes a long time to run and requires powerful computers. They didn't test if this method works against defenses designed to catch hackers.

1. Problem Statement

Vision-Language Models (VLMs) are increasingly used to power autonomous agents, creating a security vulnerability known as Typographic Prompt Injection (TPI). In TPI attacks, malicious instructions are rendered as text within images. Because the instruction arrives via the visual modality, it can bypass traditional text-based safety filters.

While prior research has focused on maximizing the Attack Success Rate (ASR) through novel attack designs, there is a lack of understanding regarding why certain visual renderings (e.g., specific font sizes, rotations, or blurs) bypass safety alignment while others fail. The authors aim to bridge this gap by characterizing the relationship between visual degradation, model embeddings, and safety failures.

2. Methodology

The paper proposes a two-pronged approach: an empirical study to identify predictors of attack success, and an optimization framework to exploit these predictors.

A. Empirical Study: Embedding Distance as a Predictor

The authors conducted a controlled study across four VLMs (GPT-4o, Claude Sonnet 4.5, Mistral-Large-3, and Qwen3-VL) using 1,000 prompts from SALAD-Bench.

Variables: They tested 12 font sizes (6–28px) and 10 visual transformations (rotation, blur, noise, etc.).
Metric: They calculated the L2 distance between the multimodal embedding of the rendered image and the embedding of the source text prompt using off-the-shelf models (JinaCLIP and Qwen3-VL-Embedding).
Hypothesis: A smaller distance in embedding space (meaning the image "looks" more like the text to the encoder) correlates with higher attack success.

B. Optimization Framework: Embedding-Guided Red Teaming

Based on the empirical findings, the authors developed a red-teaming tool to stress-test VLM safety without access to the target model's internal weights or safety classifiers.

Objective: Maximize the cosine similarity between a degraded typographic image and the source text embedding under bounded $\ell_\infty$ perturbations.
Algorithm: They adapted CWA-SSA (Common Weakness Attack with Spectral Simulation Augmentation).
- Ensemble: The optimization uses an ensemble of 4 surrogate embedding models (Qwen3-VL-Embedding, JinaCLIP, OpenAI CLIP, SigLIP) to ensure the perturbations are robust and transferable.
- Process: The algorithm iteratively adjusts pixel values to minimize the embedding distance, effectively "pushing" the degraded image into a region of the embedding space where the VLM is more likely to comply.

3. Key Contributions

Discovery of a Strong Correlation: The paper establishes that multimodal embedding distance is a strong, interpretable, and model-agnostic predictor of TPI success.
- Correlation coefficients ( $r$ ) range from -0.71 to -0.93 (negative correlation: smaller distance = higher ASR).
- This provides a proxy metric for attack success without needing to query the target VLM repeatedly.
Identification of Two Co-occurring Failure Modes: The optimization reveals that reducing embedding distance improves ASR through two distinct mechanisms:
- Perceptual Readability Recovery: The perturbation makes the text semantically parseable to the model when it was previously unreadable due to degradation (e.g., heavy blur or small font).
- Safety Alignment Evasion: The perturbation alters the holistic image representation enough to bypass the model's safety reasoning, even if the text was already readable.
A Model-Agnostic Red Teaming Tool: The proposed method allows security researchers to probe VLM safety robustness using only surrogate embedding models, requiring no access to the target model's gradients or safety filters.

4. Experimental Results

The authors tested their optimization on five degradation settings (6px font, 8px font, 90° rotation, triple degradation, and heavy blur) across the four target VLMs.

Quantitative Gains:
- Claude Sonnet 4.5 showed the most significant improvements, particularly in "Heavy Blur" scenarios, where ASR increased by +28% (from 0.7% to 28.5%).
- GPT-4o showed gains in rotation scenarios (+22%) and heavy blur (+10%), though it remained more robust than others.
- Mistral and Qwen (which had higher baseline ASR) sometimes saw a decrease in performance after optimization, suggesting the perturbation inadvertently triggered safety mechanisms that were previously dormant.
Qualitative Analysis (Failure Modes):
- Readability Dominance: In extreme degradation (6px font, heavy blur), the primary gain came from the model finally being able to "read" the text. For example, on GPT-4o with 6px text, readability failures dropped from 35 to 10, but safety refusals rose, indicating the safety filter caught the now-legible malicious request.
- Refusal Reduction Dominance: In moderate degradation (90° rotation, 8px), the gains were driven by the model failing to refuse the request even though it could read the text. This suggests the perturbation successfully confused the safety alignment logic.
- Model Dependency: The dominant failure mode depends on the model's inherent safety filter strength. Stronger filters (like GPT-4o) tend to fail on readability first; weaker filters fail on alignment.

5. Significance and Implications

Safety Mechanism Vulnerability: The findings suggest that VLM safety alignment is brittle in representation space. Small perturbations that do not significantly change the visual appearance (pixel domain) can drastically alter the model's internal representation, leading to safety bypasses.
Defense Strategy: Current defenses relying on visual degradation (e.g., blurring text to hide it) are insufficient. Defenses must be robust in the embedding space, ensuring that the model refuses harmful instructions regardless of whether the text is perfectly legible or slightly perturbed.
Red Teaming Standard: The paper provides a new, efficient methodology for evaluating VLM safety that does not require black-box access to the target model's training data or internal architecture, relying instead on embedding alignment.

Limitations: The study is limited to a single dataset (SALAD-Bench) and a specific rendering style (black text on white). The optimization is computationally expensive (~2.5 hours per sample) and was not evaluated against detection-based defenses.