🤖 machine learning

Visual Token Compression Enhances Robustness of MLLMs

This paper proposes a visual token pruning method that enhances the robustness of Multimodal Large Language Models against jailbreak attacks and hallucinations by identifying and removing misaligned, out-of-distribution visual tokens, while simultaneously reducing inference costs.

Original authors: Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu, Richang Hong

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Shishen Gu, Jiequan Cui, Wenbo Hu, Zenglin Shi, Zhenzhen Hu, Richang Hong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers don't just read words but also "see" pictures, combining them to answer questions, tell stories, or solve problems. These are called Multimodal Large Language Models (MLLMs). Think of them as super-smart students who have read every book in the library and can also look at a photo to understand the context. However, just like any student, they can get confused. Sometimes, if you show them a tricky picture or ask a sneaky question, they might get tricked into saying something dangerous (a "jailbreak") or make up facts that sound real but are completely wrong (a "hallucination").

To understand why this happens, we need to look at how these models process information. They take a picture, chop it up into tiny digital pieces called "visual tokens," and mix them with text tokens. The problem is that the computer's "eyes" and "ears" don't always speak the same language perfectly. Sometimes, a visual token is so out of place or confusing that it acts like static noise in a radio signal. This noise can throw the model off balance, making it vulnerable to attacks or causing it to dream up false information. Scientists have long known that cleaning up this noise helps, but they usually thought it was just about making the computer faster, not safer.

This paper introduces a clever new trick called Visual Token Compression, specifically a method named OOD-VTP (Out-of-Distribution Visual Token Pruning). The researchers discovered that by identifying and removing the "weird" visual tokens—the ones that don't fit well with the text—the model actually becomes much tougher against attacks and less likely to lie. It's like a bouncer at a club who kicks out the rowdy guests who don't belong, leaving the party safe and orderly. The study shows that this isn't just a theory; when they tested it on seven different popular benchmarks, the model became significantly harder to trick and more accurate, all while running faster.

The Story of the Noisy Room

Imagine you are in a giant, noisy room where a group of friends (the text) is trying to have a serious conversation. Suddenly, a bunch of strangers (the visual tokens) burst in. Most of the strangers are helpful; they bring snacks and point out things in the room that match the conversation. But a few of them are pranksters. They are wearing masks, shouting nonsense, or holding signs that say things that don't make sense with what your friends are saying.

In the world of AI, these pranksters are the misaligned visual tokens. Because the computer's vision and language systems aren't perfectly synced, these tokens act as Out-of-Distribution (OOD) inputs. In plain English, they are "out of place." When the AI tries to process them, it gets confused. This confusion is what allows hackers to "jailbreak" the AI (tricking it into ignoring its safety rules) or causes it to "hallucinate" (making up facts).

The Bouncer's Strategy

The authors of this paper realized that previous methods tried to speed up the AI by kicking out random tokens or just the ones that seemed "redundant" (like kicking out the quietest people). But they found that this didn't make the AI safer. In fact, sometimes it made things worse!

Their new method, OOD-VTP, acts like a super-smart bouncer. Here is how it works:

Measuring the Distance: The bouncer checks every single visual token (every piece of the image) and asks, "How far is this person from the conversation?" They measure the distance between the visual token and the "language feature space" (the group of text tokens).
Identifying the Pranksters: The tokens that are furthest away from the text—the ones that are totally out of sync—are flagged as the OOD tokens. These are the ones causing the trouble.
The Perfect Timing: The researchers found that you can't just kick people out at any time. There are specific "robust-pruning layers" (think of these as specific moments in the conversation) where removing the bad tokens works best. If you remove them too early or too late, it doesn't help. But if you do it at the right moment (like layer 13 or 17 in their tests), the AI suddenly becomes much more robust.

The Results: Safer, Smarter, and Faster

The team tested this bouncer strategy on two popular AI models: LLaVA-OneVision and Qwen-2.5-VL. They used seven different benchmarks to see how well it worked.

Stopping Jailbreaks: When they tried to trick the AI into doing bad things (like explaining how to make a bomb or embed malware), the OOD-VTP method was a huge success. On average, it improved the model's ability to say "No, I won't do that" by 13.29%. For the Qwen-2.5-VL model specifically, the "Refuse-to-Answer Rate" (how often it successfully blocked a bad request) jumped from 20.38% to 33.67%.
Stopping Hallucinations: The method also helped the AI stop making things up. On the HallusionBench, the accuracy improved by 8.00% for the Qwen model.
Speed: As a bonus, because they removed some of the tokens, the AI ran faster. It processed fewer tokens (dropping from 16,128 to 10,512) and used less computing power (TFlops dropped from 4.29 to 2.78), making it more efficient without losing its smarts.

What They Didn't Do (and What They Ruled Out)

It's important to note what this paper didn't do. They didn't retrain the AI from scratch. They didn't add new safety rules or change the model's brain weights. They simply pruned (cut out) the bad visual tokens during the process.

They also explicitly ruled out the idea that any token pruning makes an AI safer. In fact, they showed that if you prune the wrong tokens—specifically the ones that are well-aligned and helpful—the AI becomes less safe. They tested this by pruning the "smallest distance" tokens (the good ones), and the model's safety performance crashed. This proves that it's not just about cutting things down; it's about cutting out the specific noise that causes the trouble.

The Bottom Line

This paper suggests that the key to making Multimodal AI safer might be as simple as cleaning up the visual noise. By identifying and removing the visual tokens that don't fit with the text, the model becomes more stable, less prone to being tricked, and less likely to make up facts. It's a "plug-and-play" solution, meaning it can be added to existing models without needing a massive overhaul. While the paper shows these results are measured and consistent across multiple tests, it remains a method that enhances robustness rather than a magic bullet that solves every security problem in the world. But for a curious teenager (or a safety-conscious AI developer), it's a fascinating look at how a little bit of pruning can make a big difference in keeping our digital friends safe.

Technical Summary: Visual Token Compression Enhances Robustness of MLLMs

Problem Statement

Multimodal Large Language Models (MLLMs) face significant security and reliability challenges, specifically vulnerability to jailbreak attacks and the generation of hallucinations. While Large Language Models (LLMs) trained on text data often demonstrate robust safety mechanisms, the introduction of visual modalities in MLLMs can bypass these safeguards. The authors hypothesize that this vulnerability stems from representation misalignment between vision and language modalities. Specifically, visual tokens that are not perfectly aligned with the language feature space act as Out-of-Distribution (OOD) inputs. These OOD tokens disrupt the model's internal representations, leading to unpredictable outputs, successful jailbreaks, and factually incorrect responses (hallucinations).

Existing defense strategies, such as instruction tuning, Reinforcement Learning from Human Feedback (RLHF), or prompt engineering, often incur high computational costs or suffer from limited generalization against unseen adversaries. Furthermore, standard visual token pruning methods, designed primarily to reduce inference costs by removing "redundant" tokens, rarely enhance model robustness. This suggests that redundancy and OOD status are not synonymous; pruning the wrong tokens can actually degrade safety.

Methodology: OOD-VTP

The paper proposes Out-of-Distribution Visual Token Pruning (OOD-VTP), a training-free, plug-and-play method designed to enhance robustness by selectively removing misaligned visual tokens. The core components of the method are:

Visual Token Distance Measurement:
The method quantifies the "OOD-ness" of a visual token by measuring its distance from the textual feature space.
- Feature Space Representation: The textual feature space ( $T$ ) is represented by the last $p\%$ of textual tokens (e.g., the final 50%). These tokens are chosen because, through causal attention mechanisms, they have integrated the most semantically informative and visually grounded information from preceding tokens.
- Distance Metric: For each visual token $v^k_j$ at layer $k$ , the distance to the textual space is defined as the minimum distance to any token in the selected textual subset $S_k$ . The distance metric $D(\cdot, \cdot)$ is instantiated as the negative mean attention score across all attention heads in the LLM. A larger distance indicates a greater misalignment (higher OOD probability).
Robust-Pruning Layers:
The authors identify that not all layers are equally effective for pruning. They define robust-pruning layers as specific intermediate layers where pruning OOD tokens yields the maximum safety gain.
- Empirical analysis reveals that pruning at specific layers (e.g., layers 13–17 in Qwen-2.5-VL) significantly improves robustness, whereas pruning at other layers offers limited benefit or degrades performance.
- The optimal layer and pruning ratio are determined via a grid search on a validation set, though the paper notes that a unified fixed layer (e.g., Layer 14) can achieve strong results across multiple benchmarks without per-benchmark tuning.
Pruning Strategy:
At the identified robust-pruning layer, the top $r\%$ of visual tokens with the largest distances (i.e., the most misaligned/OOD tokens) are pruned. This process can be applied iteratively across multiple layers.

Key Contributions

Novel Insight: The paper is the first to demonstrate that visual token compression can substantially improve the robustness of MLLMs against both jailbreak attacks and hallucinations, challenging the notion that compression solely serves efficiency.
OOD-VTP Method: The authors introduce a specific pruning technique that targets visual tokens misaligned with the language feature space, utilizing a distance metric based on attention scores and identifying specific "robust-pruning layers."
Comprehensive Validation: The method is validated on seven diverse benchmarks, showing significant robustness gains while maintaining strong performance on general tasks.
Complementarity: The approach is shown to be complementary to existing safety methods (e.g., AdaShield), further boosting safety when combined.

Experimental Results

The authors evaluated OOD-VTP on LLaVA-OneVision and Qwen-2.5-VL models across seven benchmarks:

Jailbreak Defense:
- On SafeBench, HADES, and MM-SafetyBench, OOD-VTP achieved an average improvement of 13.29% in the Refuse-to-Answer Rate (RAR) over the vanilla Qwen-2.5-VL baseline.
- Specifically, on Qwen-2.5-VL, the average RAR improved from 20.38% (Vanilla) to 33.67%, and the Harmless Output Rate (HOR) improved to 50.09%, outperforming other compression methods (FastV, PDrop, etc.) which showed negligible or negative impacts on safety.
- When combined with AdaShield, the RAR on SafeBench reached 93.80%.
- The method also demonstrated robustness against perturbation-based image attacks (PGD).
Hallucination Mitigation:
- On HallusionBench, OOD-VTP improved accuracy by 8.00% (from 59.73% to 67.99% on the best layer) for Qwen-2.5-VL.
- On COCO2014, it reduced the CHAIR metric (indicating fewer hallucinations) to 0.59% for Qwen-2.5-VL, outperforming the vanilla baseline.
General Performance & Efficiency:
- General Benchmarks: OOD-VTP maintained or slightly improved performance on general datasets like MME (perception and cognition scores) compared to the vanilla model.
- Efficiency: The method reduced computational cost (TFlops) and increased Tokens Per Second (TPS) compared to the vanilla model and other compression baselines like FastV.
- Trade-off: A slight drop in performance was observed on OCRBench, attributed to the loss of fine-grained image information due to token pruning, which the authors deem a reasonable trade-off for the substantial safety gains.

Significance and Claims

The paper claims that OOD-VTP offers a novel, efficient, and effective direction for securing MLLMs. By addressing the root cause of vulnerabilities—misaligned visual tokens acting as OOD inputs—the method enhances model robustness without requiring expensive retraining or alignment tuning.

The authors emphasize that:

Robustness is highly sensitive to which tokens are pruned; removing well-aligned tokens destroys safety, while removing OOD tokens enhances it.
Robustness is also sensitive to where (at which layer) pruning occurs, necessitating the identification of robust-pruning layers.
The method serves as a "plug-and-play" module that can be integrated with existing safety defenses to create a more robust system, balancing safety, efficiency, and general utility.