Can Visual Mamba Improve AI-Generated Image Detection? An In-Depth Investigation

Technical Summary: Can Visual Mamba Improve AI-Generated Image Detection? An In-Depth Investigation

Problem Statement
The rapid advancement of generative AI, including Generative Adversarial Networks (GANs), diffusion models, and Vision-Language Models (VLMs), has led to the creation of increasingly photorealistic synthetic images. While beneficial for content creation, this technology poses significant risks regarding misinformation, identity theft, and privacy violations. Consequently, the development of robust detection mechanisms is critical. While Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and VLMs have been extensively studied for this task, the potential of Vision Mamba architectures—based on State Space Models (SSMs)—remains largely unexplored. This study addresses the gap in literature by systematically evaluating whether Vision Mamba models can effectively detect AI-generated images, particularly focusing on their accuracy, efficiency, and generalization capabilities across diverse generative sources.

Methodology
The authors conducted a comprehensive empirical study benchmarking three distinct Vision Mamba variants against established State-of-the-Art (SOTA) baselines, including CNNs (ResNet, Xception), Transformers (ViT, DeiT), and VLM-based detectors (AntifakePrompt, Bi-LORA).

• Models Evaluated:
  • Vim: The first pure SSM-based vision model utilizing bidirectional SSM paths and positional embeddings.
  • MambaVision: A hybrid architecture combining CNNs for local features with a redesigned Mamba block and self-attention for global dependencies.
  • VSSD (Visual State Space Duality): A non-causal variant of the State Space Duality model designed to handle 2D image data without causal constraints, utilizing multi-scan strategies.
• Datasets: Experiments were performed on three diverse datasets:
  • AntifakePrompt: Contains real images from MS COCO and Flickr, and synthetic images from various text-to-image models (SD2, SDXL, DALL-E 2, etc.) and attack scenarios (adversarial, backdoor, data poisoning).
  • Bedroom: Comprises real images from LSUN Bedroom and synthetic images generated by unconditional and text-to-image diffusion models (LDM, ADM, DDPM, SDv1.4, GLIDE).
  • UniversalFakeDetect: A large-scale dataset with real images from LSUN, ImageNet, and LAION, and synthetic images from 20 different generative models (GANs, Diffusion, DeepFakes).
• Experimental Protocol: Models were fine-tuned or trained from scratch on specific training splits (e.g., MS COCO vs. SD2+LaMa for AntifakePrompt) and evaluated on held-out test sets containing images from unseen generative models to assess cross-generator generalization. Performance was measured using Accuracy (ACC).

Key Contributions

1. Systematic Evaluation of Vision Mamba: This paper provides the first rigorous benchmark of Vision Mamba architectures (Vim, MambaVision, VSSD) specifically for the task of AI-generated image detection.
2. Comparative Analysis: The study offers a detailed comparison of SSM-based models against CNNs, Transformers, and VLMs across multiple datasets and generative sources.
3. Identification of Generalization Limits: The research highlights a critical limitation of current Vision Mamba models: while they perform competitively when training and testing occur on the same generative distribution, their performance degrades significantly when applied to unseen generators.
4. Architectural Insight: The authors analyze the structural reasons for these limitations, attributing them to the sequential nature of SSMs, which struggles to capture the global spatial dependencies and fine-grained artifacts necessary for distinguishing synthetic content compared to the global receptive field of Transformers.

Results

• Generalization Performance: Vision Mamba models exhibited substantial drops in accuracy when tested on images from unseen generative models. For instance, on the AntifakePrompt dataset, while models like VSSD achieved near-perfect accuracy on the training distribution (SD2), their performance collapsed on unseen models like SDXL (0.23%) and DALL-E 2 (14.00%).
• Comparison with SOTA:
  • VLMs and Large Models: Models with billions of parameters, such as AntifakePrompt and Bi-LORA, demonstrated superior generalization, maintaining high accuracy across diverse unseen generators and adversarial attacks.
  • Transformers: Attention-based models (ViT, DeiT) showed moderate to strong robustness, often outperforming Vision Mamba models in cross-generator scenarios.
  • Vision Mamba: Despite their computational efficiency and lower parameter counts, Vision Mamba models generally achieved lower average accuracy (around 70%) compared to Transformer and VLM counterparts (often >90%). They struggled particularly with unconditional diffusion models (e.g., ADM, DDPM) and adversarial perturbations.
• Feature Visualization: t-SNE visualizations revealed that while Vision Mamba detectors could separate real and fake images for specific generators (e.g., ProGAN), the feature distributions for unseen generators overlapped significantly, indicating a failure to learn generalizable artifacts.

Significance and Claims
The paper concludes that while Vision Mamba architectures offer promising efficiency and speed, they currently face significant hurdles in the domain of AI-generated image detection. The authors claim that the sequential, causal nature of standard SSMs limits their ability to model the complex, non-causal spatial relationships inherent in visual data, which is crucial for detecting subtle generation artifacts.

The significance of this work lies in its honest assessment of Vision Mamba's current capabilities. It establishes that, unlike Transformers and VLMs which leverage global attention or large-scale multimodal pretraining to achieve robust generalization, Vision Mamba models are currently ill-suited for detecting AI-generated images across diverse, unseen distributions without substantial architectural modifications. The authors suggest that future research must focus on hybrid SSM-Attention architectures, improved 2D scanning strategies, and frequency-domain components to bridge this performance gap. The study serves as a cautionary benchmark, indicating that efficiency gains from Mamba architectures should not be pursued at the cost of the generalization robustness required for real-world security applications.