💻 computer science

Auditing Inference-Time Defense Evaluation for Multimodal Large Language Models

This paper presents a traceable comparative audit of multimodal large language model inference-time defenses, revealing critical provenance failures in benchmark data and evaluation protocols that invalidate prior mass-refusal claims while highlighting the need for rigorous data integrity standards in future safety assessments.

Original authors: Bulat Nutfullin, Vladimir Evgrafov, Dmitry Namiot

Published 2026-07-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Bulat Nutfullin, Vladimir Evgrafov, Dmitry Namiot

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the head chef at a very fancy restaurant that serves two things at once: a picture on a plate and a spoken order. Your goal is to make sure the kitchen never cooks anything dangerous, even if someone tries to trick the chef by hiding a "poisonous" instruction inside the picture or whispering it in a weird way. This is the world of Multimodal Large Language Models (MLLMs). These are super-smart computer brains that can "see" images and "read" text, then chat back to you. But because they can see, bad actors can try to sneak in harmful commands by drawing them in pixels or hiding them in the background of a photo.

To stop this, scientists have invented Inference-Time Defenses. Think of these as security guards who stand right at the kitchen door. They don't change the chef's recipe (the model's brain); instead, they check the order before it goes in, or they check the final dish before it goes out. Some guards add a polite warning note to the order, while others might shake the picture a little bit to blur out any hidden tricks. The big question for everyone in the AI safety world is: Which guard is actually doing the best job? If we can't trust the guards, we might accidentally serve a dangerous meal, or worse, we might be so scared of danger that we refuse to serve any food at all.

The Great Kitchen Audit: When the Scorecard Was Wrong

A team of researchers from Moscow decided to play detective. They didn't build new guards; instead, they went into the archives of a recent experiment that claimed to test three different security guards (inspired by methods called RapGuard, AdaShield, and SmoothVLM) across eight different AI chefs. They wanted to see who was the best. But as they dug through the files, they found a mess of broken links, missing ingredients, and confused scorecards.

Here is what they discovered, served up with a side of reality checks.

1. The "Missing Ingredient" Mystery

The researchers found that the experiment was supposed to test 9,000 different orders (inputs) across seven different safety tests. However, when they checked the provenance (the paper trail of where the data came from), they realized three of the seven tests were completely broken.

The "Wrong Plate" Error: One test, meant to check if the AI could spot hidden instructions in images, was actually serving the wrong picture entirely. It was like trying to test a fire alarm by spraying water on a toaster instead of a smoke detector.
The "Text-Only" Cheat: Another test was supposed to be a mix of pictures and text, but the system accidentally let the AI ignore the pictures and just read the text. It was a "text-only fallback," meaning the AI never actually had to look at the image to pass the test.
The "Random Patch" Glitch: A third test claimed to use a special "adversarial patch" (a tiny, tricky sticker on an image to confuse the AI). But the computer didn't generate the tricky sticker; it just drew a random rectangle.

Because of these errors, the researchers had to throw out the results for those three tests. They couldn't just "fix" the numbers; the data simply wasn't there. This left them with only 4,820 valid inputs from four specific tests (FigStep, JailBreakV, SALAD-Bench, and HarmBench-Simple) to work with.

2. The "Keyword" Trap

The original experiment used a very simple way to decide if the AI was safe: it looked for specific "magic words."

If the AI said "I can't do that" or "I'm sorry," it was marked as Safe (a refusal).
If the AI said "Here is how to..." or "Sure, I can help," it was marked as Unsafe (a harmful answer).

The researchers found a huge problem with this method. The system counted empty answers (where the AI said nothing at all) as "Safe" simply because the legacy code didn't have a special rule for them; since the empty string matched neither the "harmful" list nor the "refusal" list, the default logic counted it as safe. It also counted answers that started with a polite warning but then gave the dangerous instructions as "Safe" because the system only looked for the presence of a refusal phrase to override a harmful one, and sometimes the logic got tangled.

When they re-checked the data, they found that the "keyword" method was a terrible judge. It was like a teacher grading a test by only looking for the word "the" and ignoring whether the student actually answered the question. In one audit of 246 tricky answers, the keyword system missed 13 harmful answers that a smarter human-like judge (a "Haiku-family judge") correctly identified as dangerous. In technical terms, the keyword system labeled these 13 cases as "Safe" when they were actually "Harmful" (false negatives).

3. The "Mass Refusal" Myth

The original study seemed to suggest that the security guards were causing the AI to refuse everything (a "mass refusal"). The researchers re-audited 38,500 safe, normal questions to see how often the AI actually said "no" when it didn't need to.

The result? The AI was actually very polite and helpful.

The estimated rate of unnecessary refusals was only 0.52%.
Even in the worst-case scenario (the largest single group of data), the rate was only 3.24%.
The researchers noted that the "sampling-only" uncertainty (the margin of error) could reach 10.92%, but even that is far from the "mass refusal" the original study implied.

The confusion came from the "defensive preamble." The security guards often added a long, polite warning at the start of the answer (e.g., "I must warn you that this is dangerous..."). The old keyword system thought this warning was the refusal, but the AI actually went on to give the answer anyway. It was like a bouncer saying, "I'm not letting you in," but then opening the door and letting you walk right in. The keyword system mislabeled these preambles as refusals, inflating the numbers.

4. The Cost of Being Safe

Finally, the researchers looked at how much time these security guards wasted.

The "Gaussian" guard (which blurs the image five times and votes on the answer) took 5.45 to 12.76 times longer to process a request than doing nothing.
The full stack of three guards took 4.36 to 16.59 times longer.

This means that while the guards might be trying to help, they are making the kitchen incredibly slow.

The Bottom Line

This paper doesn't tell us which security guard is the "best." In fact, it tells us that we can't trust the previous rankings because the data was broken and the scoring method was flawed.

What they ruled out: They proved that the original claims about "mass refusal" were wrong and that three of the seven safety tests were invalid.
What they found: The security guards (specifically the prompt wrappers) do change the AI's behavior, but the effect varies wildly depending on the AI model and the type of test. Sometimes they help, sometimes they hurt, and sometimes they just slow things down.
The Lesson: You cannot just run a test and look at a score. You need to make sure the ingredients (inputs) are real, the scorecard (evaluation) actually measures safety, and you have the full recipe (raw text) to check the work.

The authors conclude that before we can compare different security guards, we need to build a better, more honest system for testing them. Until then, we can't say for sure who is the hero and who is just making the kitchen slower.

Technical Summary: Auditing Inference-Time Defense Evaluation for Multimodal Large Language Models

Problem Statement

Multimodal Large Language Models (MLLMs) introduce new attack surfaces via visual channels, including instructions embedded in images, split across modalities, or hidden in pixel-space perturbations. While inference-time defenses (modifying inputs, prompts, or outputs without retraining weights) offer a cost-effective mitigation strategy, their evaluation suffers from critical integrity issues. The authors identify that comparative safety results often rely on flawed data provenance: result files may not correspond to the actual image payloads processed, "safe" labels may be derived from empty strings or pipeline votes rather than semantic refusals, and benchmark loaders may silently fallback to text-only modes when images are missing. Consequently, existing evaluations often conflate pipeline artifacts with semantic safety, failing to validate whether a defense actually prevents harmful generation or merely alters the output format.

Methodology

The authors conducted a traceable audit of an experimental archive covering two fixed prompt wrappers (inspired by AdaShield and RapGuard) and a Gaussian image-perturbation adapter (inspired by SmoothVLM). The study evaluated these adapters across eight MLLMs from the InternVL and Qwen-VL families.

The methodology involved three primary audit phases:

Input and Output Provenance Audit:
The team examined the execution logs and input files for seven planned safety benchmarks (totaling 9,000 inputs). They verified that the image payloads matched the intended benchmarks and that the loader did not admit text-only fallbacks. Three branches were found to fail these checks:
- MM-SafetyBench: The local renderer used the wrong payload field.
- JailBreakV-MM: The loader admitted text-only fallbacks for missing images.
- Adversarial-patch branch: The runner generated random rectangles instead of optimized adversarial patches.
  These branches were quarantined, leaving four corpora (FigStep, JailBreakV, SALAD-Bench, HarmBench-Simple) with 4,820 inputs per configuration for analysis.
Semantic and Keyword Re-evaluation:
The original safety scores relied on a legacy keyword classifier ( $C_{kw}$ ) that counted empty strings as "safe" and used a majority vote for Gaussian adapters. The authors re-audited 274 adversarial responses using a separate LLM-judge protocol. They found that the keyword rule had a high false-negative rate (13 out of 246 comparable responses) and that "safe" labels often masked defensive preambles or empty outputs.
Benign Refusal Audit:
To measure over-refusal (false positives on benign queries), the authors analyzed 38,500 stored outputs from benign MMBench-derived queries. They employed a dedicated LLM-judge to distinguish between direct system refusals, defensive preambles followed by answers, and actual refusals. They calculated pooled refusal rates and sampling intervals, explicitly excluding the "mass-refusal" interpretation derived from the legacy keyword protocol.

Key Contributions

The paper makes four specific contributions:

Traceable Audit Framework: A method to distinguish between intended benchmarks, actual payloads, returned text, pipeline proxies, and semantic judgments.
Restricted Comparative Analysis: A re-evaluation of prompt wrappers (Ada, Rap) and their combination (A+R) against a Gaussian adapter (Smooth) on the retained corpora (FigStep, JailBreakV, SALAD-Bench, HarmBench-Simple).
Benign Refusal Re-estimation: A corrected refusal rate estimate based on 38,500 outputs, demonstrating that the legacy keyword protocol significantly overestimated refusal rates due to defensive preamble contamination.
Operational Cost Measurement: Quantification of batch processing time and the frequency of defensive preamble emission, separating these costs from unmeasured task accuracy.

Results

Data Integrity: Three of the seven planned benchmark branches were invalidated due to input-provenance failures. The remaining results are descriptive aggregates of a legacy keyword protocol, not semantically validated harmlessness rates.
Safety Scores: The legacy keyword scores showed no uniform ordering across models or adapters. For example, on the 8B InternVL3.5 model, the score varied from 0.882 (no adapter) to 1.000 (Rap wrapper), but the semantic audit could not validate these as genuine safety improvements.
Keyword vs. Semantic Discrepancy: In a nonrandom search of 246 adversarial responses, the keyword rule missed 13 harmful cases (false negatives) that a semantic judge identified. Precision was 41.7% and recall was 69.8% on this sample.
Benign Refusal: The pooled refusal point estimate for benign queries was 0.52%, with the largest estimable cell at 3.24%. This contradicts the original "mass-refusal" interpretation. The keyword protocol had inflated this rate by approximately 54-fold because it counted defensive preambles (emitted by the Rap wrapper in 20–78% of outputs) as refusals.
Operational Costs: The Gaussian adapter increased processing time by 5.45–12.76 times, and the full three-adapter stack increased it by 4.36–16.59 times.

Significance and Claims

The authors explicitly state that this work does not claim one defense or adaptive router is universally superior. Instead, the significance lies in:

Methodological Correction: Highlighting that safety evaluations require immutable manifests binding payloads to results, preservation of user-visible text separate from pipeline votes, and evaluators that measure the claimed construct (semantic safety vs. keyword matching).
Evidence of Variability: The data shows that defense performance is highly dependent on the specific model and corpus, justifying the profiling of candidate pipelines on target traffic before deployment.
Provenance Requirements: The paper establishes a set of provenance requirements for future MLLM defense evaluations, arguing that without these gates (valid inputs, stored raw responses, semantic judging), comparative rankings are invalid.

The paper concludes that the archived data does not support the earlier mass-refusal interpretation or a definitive ranking of defenses. It calls for future evaluations to use versioned manifests and denser independent semantic judging before comparing fixed and adaptive defense pipelines.