When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm

1. Problem Statement

The field of text-to-image (T2I) generation is transitioning from Diffusion Models (e.g., Stable Diffusion) to Multimodal Large Language Models (MLLMs) (e.g., Janus, Bagel). While MLLMs offer superior semantic understanding, reasoning, and the ability to follow complex, conversational instructions, this paper posits that these advancements introduce novel and potentially greater safety risks.

The core problem is that existing safety defenses and fake image detectors are primarily optimized for diffusion models. It remains unclear whether the enhanced semantic capabilities of MLLMs:

1. Increase the likelihood of generating unsafe content (e.g., violence, sexual content, hate speech) by better interpreting abstract or coded prompts.
2. Produce fake images that are harder to detect, thereby undermining existing forensic tools.

2. Methodology

The authors conducted a systematic empirical comparison between two diffusion models and five state-of-the-art MLLMs.

Models Evaluated:

• Diffusion Models: SD3.5 Large, SD3.5 Large Turbo.
• MLLMs: Bagel, Janus, Janus Pro, TokenFlow, VILA-U.

Evaluation Dimensions:
The study focused on two critical safety dimensions:

1. Unsafe Image Generation:

   • Dataset: 1,184 unsafe prompts aggregated from four sources (Lexica, 4chan, Template, I2P) and an expanded "TemplateLong" dataset with richer descriptive details.
   • Protocol: Each model generated 10 images per prompt (totaling 82,880 images). External safety filters were disabled to measure the model's intrinsic tendency.
   • Metric: An "Unsafe Score" calculated based on the proportion of images flagged as NSFW by the OpenAI Moderation API (validated against human annotators).
   • Analysis: Included qualitative case studies on abstract prompts, non-English (Chinese) prompts, and gender bias in neutral sexual prompts.

2. Fake Image Detection:

   • Dataset: 2,000 benign prompts from MSCOCO and Flickr30k, generating 14,000 images.
   • Detectors: Four detectors were tested: two commercial (Winston.AI, Illuminarty) and two open-source research models (DE-FAKE, AIorNot-SigLIP2).
   • Experiments:
     • Baseline: Testing off-the-shelf detectors.
     • Fine-tuning/Training: Retraining detectors on multi-source data (mixing diffusion and MLLM outputs) and single-source data to test generalization.
     • Prompt Extension: Testing detection accuracy as prompts become increasingly detailed (v0: original, v1: moderate detail, v2: high detail) to see if MLLMs' ability to interpret context makes images more "realistic" and harder to detect.

3. Key Contributions & Findings

A. Unsafe Image Generation

• Higher Risk in MLLMs: MLLMs consistently produced higher unsafe scores than diffusion models across all datasets. For example, on the TemplateLong dataset, Janus Pro had an unsafe score of 0.613, while diffusion models scored below 0.28.
• The "Comprehension" Factor: The primary driver for this gap is not necessarily weaker safety alignment, but superior semantic comprehension.
  • Diffusion Models: Often fail to interpret abstract or complex prompts, resulting in "damaged" or garbled images (e.g., text artifacts, black patches). These failures are often misclassified as "safe" by detectors, artificially lowering their unsafe scores.
  • MLLMs: Successfully interpret abstract, metaphorical, or slang-based prompts (e.g., "sh*t hole" interpreted as a run-down environment rather than literal text) and generate coherent, realistic unsafe images.
• Cross-Lingual Vulnerability: MLLMs successfully generated unsafe images from Chinese prompts, whereas diffusion models failed to interpret the language entirely, resulting in zero unsafe scores (due to generation failure). This expands the attack surface to non-English adversaries.
• Gender Bias: MLLMs exhibited significant gender bias in unsafe generation. When given gender-neutral sexual prompts, models like Bagel generated female depictions 80% of the time, reinforcing harmful stereotypes.

B. Fake Image Detection

• Evasion of Detectors: Images generated by MLLMs are significantly harder to detect as fake compared to diffusion models. Commercial detectors showed accuracy drops of up to 30-40% for MLLMs compared to diffusion models.
• Training Bias: The detection gap is largely due to training data bias. Detectors trained exclusively on diffusion outputs fail to recognize MLLM artifacts.
  • Mitigation: Retraining detectors on paradigm-inclusive data (mixing both diffusion and MLLM outputs) significantly improved accuracy (often >90%), proving the gap is learnable but requires specific data.
  • Generalization: Detectors fine-tuned on a single MLLM (e.g., Janus) failed to generalize well to other MLLMs, indicating high architectural diversity among MLLMs makes unified detection difficult.
• Prompt Extension Attack: As prompts became more detailed and context-rich, MLLMs produced increasingly realistic images that evaded detection (accuracy dropped from ~0.98 to ~0.88). Diffusion models remained largely unaffected by prompt complexity, maintaining high detectability.

4. Significance and Implications

• Paradigm Shift in Risk: The paper establishes that the shift to MLLMs is not just a usability improvement but a security regression. The very feature that makes MLLMs useful (deep semantic understanding) is the same feature that makes them dangerous (interpreting abstract unsafe intent and generating high-fidelity forgeries).
• Limitations of Current Defenses: Current safety filters and fake image detectors are ill-equipped for the MLLM era. Keyword-based filters are easily bypassed by MLLMs' ability to understand metaphorical language.
• Need for Adaptive Defenses:
  • Safety mechanisms must evolve from simple keyword blocking to semantic reasoning to understand intent behind complex prompts.
  • Fake image detectors require paradigm-inclusive training and must be resilient to "prompt extension" attacks where adversaries use richer descriptions to fool detectors.
• Ethical Concerns: The findings highlight new vectors for misinformation, harassment, and non-consensual imagery, particularly given the cross-lingual capabilities and gender biases observed in MLLMs.

Conclusion

The paper concludes that while MLLMs represent a technological leap in generative AI, they introduce a "double-edged sword" scenario: their advanced understanding capabilities significantly lower the barrier for generating harmful content and simultaneously raise the bar for detecting synthetic media. The research community and industry must urgently develop adaptive, semantically aware safety protocols to mitigate these emerging risks.