💻 computer science

Misguiding BLIP with Adversarial Patches: Multi-level Attacks with Cross-modal Attention Graphs in BLIP

This paper proposes MAGA, a novel multi-level attack framework that exploits cross-modal attention vulnerabilities in BLIP models by constructing and disrupting cross-modal attack graphs, resulting in significant performance degradation and linguistically plausible but visually inconsistent adversarial captions.

Original authors: yingying hu, minghao mo, peng zhang

Published 2026-09-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: yingying hu, minghao mo, peng zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the rapidly evolving world of artificial intelligence, a new generation of computer systems has emerged that can see and read simultaneously. These systems, known as vision-language models, are trained on millions of pairs of images and text, learning to understand how a picture of a dog relates to the word "dog." They have become the foundation for tools that describe photos, answer questions about scenes, and search for images using natural language. For these machines to work well, they must constantly connect specific words to specific parts of an image, deciding which pixels belong to the concept of "horse" and which belong to "grass." This process relies on a complex internal mechanism that acts like a dynamic map, linking every word in a sentence to the relevant visual details in a photograph. If this connection breaks, the machine loses its ability to understand what it is looking at, potentially leading to confusion or incorrect descriptions.

Researchers at Guangzhou University have discovered a subtle weakness in how these systems maintain those connections. They found that by placing a small, carefully designed pattern on an image, they could trick the model into rearranging its internal map of word-to-picture links. This pattern, which looks like a simple patch of color or texture, does not need to be a complex disguise. Instead, it acts as a silent signal that causes the model to ignore the most important parts of an image and focus on irrelevant background areas. When this happens, the model can still speak fluently and grammatically, but what it says no longer matches the picture. It might describe a field of flowers when the image clearly shows a horse, or claim a person is holding a bowl of ramen when the photo contains only a living room. The researchers call this phenomenon "natural but wrong," highlighting a vulnerability where the machine's confidence remains high even as its understanding collapses.

The team developed a method to create these deceptive patterns, which they named a multi-level attack. Unlike previous attempts that tried to simply blur an image or scramble its colors, this approach targets the specific way the model connects text to vision. The researchers built a system that maps out the strength of the links between words and image parts, essentially creating a graph that shows which words are paying attention to which pixels. They then trained their attack to maximize the difference between the graph of a clean image and the graph of the same image with the patch on it. By doing this, they forced the model to sever the strong connections between key words and their visual evidence, while simultaneously creating new, false connections to random background areas. This process was combined with other goals to ensure the attack remained hidden and the resulting text sounded natural, even though it was factually incorrect.

The results of their experiments were striking. When they applied their generated patch to images from a standard dataset, the model's ability to find the correct image for a given text dropped dramatically. In tests where the system was asked to match a sentence to the right picture, its success rate fell from over seventy percent to just nine percent. The attack was so effective that it worked even on images the system had never seen before, suggesting that the flaw was fundamental to how the model processes information rather than a simple mistake with specific pictures. In tasks where the model had to describe an image, the quality of the description plummeted. The system's scores for accuracy dropped significantly, yet the sentences it produced remained grammatically correct and diverse, avoiding the repetitive gibberish that often signals a broken system. This confirmed that the model was not just failing to speak; it was confidently describing a reality that did not exist.

The researchers also tested the system on visual questions, such as counting objects or identifying spatial relationships. The attack caused the model to fail at these logical tasks as well, with success rates dropping from nearly eighty percent to just twelve percent. In many cases, the model would answer a question about a dog with a description of a cat, or claim there were three animals when there was only one. What made these failures particularly notable was that the model's internal attention map had been completely rewired. Visualizations showed that the model, which normally focused its attention on the main subject of the photo, began to ignore the subject entirely and instead focused on empty sky or grass. The patch did not hide the object; it simply made the model look away from it.

This work suggests that the current generation of these powerful AI systems is more fragile than previously thought. While earlier security tests focused on whether an image could be made to look like something else to the human eye, this research shows that the danger lies in how the machine organizes its own understanding. The attack does not require the machine to be fooled into seeing a different object; it only needs to be convinced that the object it sees is irrelevant. By disrupting the internal graph that links words to pixels, the researchers proved that a small, static pattern could cause a cascade of errors across different tasks, from searching for images to answering complex questions. The study concludes that as these models become more integrated into daily life, understanding and protecting these internal connection maps will be just as important as protecting the images themselves. The findings serve as a reminder that even the most advanced systems can be led astray not by changing what they see, but by changing how they think about what they see.

Technical Summary: Misguiding BLIP with Adversarial Patches (MAGA)

Problem Statement
Vision-Language Pretraining (VLP) models, particularly fusion-based architectures like BLIP, rely on cross-modal attention mechanisms to establish fine-grained interactions between textual tokens and visual patches. While existing adversarial attacks on these models primarily focus on pixel-level perturbations (modifying image pixels directly) or feature-level disruptions (separating image-text embeddings in latent space), the vulnerability of the internal cross-modal attention structures remains underexplored. The authors posit that the dense Cross-modal Attention Graphs (CAGs) within fusion encoders are a critical point of failure. Current attacks often fail to explicitly target the topological rewiring of token-to-patch associations, which is essential for grounding textual concepts in visual evidence. Consequently, there is a need for an attack method that specifically targets the attention-level vulnerability to induce "natural but wrong" hallucinations—outputs that are linguistically fluent but semantically inconsistent with the visual input.

Methodology: Multi-level Attention-Graph Attack (MAGA)
The paper proposes MAGA, a universal adversarial patch attack framework designed to disrupt BLIP through a multi-level optimization strategy. The method operates under a white-box threat model where the attacker has access to the target model's parameters and intermediate attention matrices during training, but only the learned patch is deployed.

The framework consists of three core components:

Cross-Modal Attention Graphs (CAGs):
The authors construct CAGs to represent the attention relationships between textual tokens and visual patches. By averaging attention scores across heads and applying a threshold $\delta_g$ , they transform the attention score matrix into a weighted bipartite graph. Nodes represent tokens and patches, while edges represent attention weights. A shared Graph Convolutional Network (GCN) encodes both clean and adversarial CAGs into graph-level representations ( $g_{clean}$ and $g_{adv}$ ).
Multi-Level Objective Function:
MAGA jointly optimizes a universal adversarial patch $p_\theta$ by minimizing a total loss function ( $L_{total}$ ) that combines three distinct objectives:
- Attention-Level Objective ( $L_{attention}$ ): This is the novel core of the method. It maximizes the cosine distance between the clean and adversarial CAG representations. By forcing the graph structures to diverge, the attack disrupts the specific token-to-patch connections (e.g., weakening the link between "horse" and the visual region containing the horse), effectively rewiring the model's internal reasoning.
- Feature-Level Objective ( $L_{feature}$ ): This component ensures the attack remains a universal patch by including a visual fidelity constraint (minimizing pixel distortion and structural dissimilarity via SSIM) and an alignment disruption term. It reduces the similarity between the adversarial image and its paired text while maintaining consistency with the clean image's visual features.
- Caption-Level Objective ( $L_{caption}$ ): To prevent degenerate or repetitive outputs, a semantic dispersion term is introduced. It maximizes the distance between the generated adversarial caption and the ground-truth caption, while also encouraging diversity among captions within a mini-batch. This promotes the generation of fluent, diverse, yet factually incorrect descriptions.
Optimization and Deployment:
The patch generator is trained using a fixed latent code and a spatial mask. During optimization, the target BLIP model parameters remain frozen; only the patch generator and the auxiliary graph encoder are updated. Once trained, the universal patch can be applied to unseen images without further access to the target model or its gradients.

Key Contributions

Identification of Attention Vulnerability: The work investigates and demonstrates that the cross-modal attention distributions in BLIP fusion encoders are a distinct and exploitable vulnerability, separate from pixel or global feature vulnerabilities.
CAG-Based Attack Formulation: The authors introduce Cross-modal Attention Graphs (CAGs) as a structural representation of token-patch associations. They propose a method to maximize the discrepancy between clean and adversarial CAGs, effectively "rewiring" the model's internal attention.
Multi-Level Attack Framework (MAGA): The paper proposes a unified framework that jointly optimizes attention-level disruption, feature-level alignment, and caption-level semantic dispersion.
Stealthiness via Diversity Regularization: By introducing a diversity regularization term, the method successfully induces "natural but wrong" hallucinations, where the model produces linguistically plausible captions that are semantically detached from the visual content.

Experimental Results
The authors evaluated MAGA on MS COCO, Pascal Sentences, and BLIP-VQA benchmarks across image-text retrieval, image captioning, and Visual Question Answering (VQA) tasks.

Retrieval: MAGA significantly outperformed baselines (including PGD, UAP, AdvPatch, and AdvCLIP). On MS COCO, it reduced Image-to-Text Recall@1 from 72.3% (clean) to 9.0%, compared to 20.0% for the strongest baseline (AdvCLIP). It also demonstrated strong transferability to the out-of-distribution Pascal Sentences dataset.
Captioning: The attack caused a drastic drop in CIDEr scores (from 82.0 to 12.8) while maintaining high lexical diversity (Distinct-2 score of 0.40) and reasonable perplexity. This confirmed the "natural but wrong" phenomenon.
VQA: MAGA reduced overall VQA accuracy from 78.5% to 12.4%, with Attack Success Rates (ASR) exceeding 80% across Yes/No, Counting, and Spatial question categories.
Ablation Studies: Experiments confirmed that the attention-level objective (CAG disruption) provides a significant boost over feature-level or caption-level attacks alone, validating the necessity of the multi-level approach.

Significance and Claims
The paper claims that MAGA highlights a critical vulnerability in the reasoning mechanisms of multimodal foundation models. By demonstrating that disrupting the topology of cross-modal attention is more effective than perturbing global features, the work underscores the importance of evaluating the robustness of internal attention structures, not just input-output mappings. The ability to generate fluent but visually inconsistent outputs suggests that current VLP models may rely heavily on internal language priors when visual grounding is disrupted, posing risks for the reliable deployment of these models in safety-critical applications. The authors conclude that robustness evaluation must extend to the fine-grained interactions within fusion encoders to ensure the reliability of future multimodal systems.

Technical Summary: Misguiding BLIP with Adversarial Patches (MAGA)

More like this