UniSAFE: A Comprehensive Benchmark for Safety Evaluation of Unified Multimodal Models

1. Problem Statement

Unified Multimodal Models (UMMs) represent a paradigm shift in foundation models, capable of processing and generating content across multiple modalities (text and images) within a single system. Unlike traditional single-task models (e.g., text-only LLMs or text-to-image generators), UMMs support complex interaction patterns such as multimodal-context image generation, multi-image composition, and multi-turn iterative editing.

However, this unification introduces novel safety risks that existing benchmarks fail to capture:

• Compositional Risks: Inputs that appear benign in isolation (e.g., a harmless image + a harmless text instruction) can combine to produce harmful outputs.
• Multi-Turn Escalation: A sequence of individually innocuous requests can incrementally edit an image into harmful content.
• Fragmented Evaluation: Existing safety benchmarks are siloed by task (e.g., specific to Text-to-Image or Text-to-Text) or modality, lacking a systematic framework to evaluate system-level vulnerabilities across the diverse I/O combinations of UMMs.

2. Methodology: UniSAFE Benchmark

The authors introduce UniSAFE, the first comprehensive benchmark designed for system-level safety evaluation of UMMs.

A. Task Coverage (7 I/O Combinations)

UniSAFE evaluates models across seven distinct task types, covering both conventional and novel multimodal scenarios:

1. Text-to-Image (TI): Standard generation.
2. Image Editing (IE): Modifying an image based on text.
3. Image Composition (IC): Merging elements from multiple images based on text.
4. Multi-Turn Editing (MT): Iterative refinement of an image over multiple turns.
5. Text-to-Text (TT): Standard language generation.
6. Image-to-Text (IT): Image captioning/OCR.
7. Multimodal Understanding (MU): Answering questions based on image + text.

B. Shared-Target Design

A core innovation of UniSAFE is the Shared-Target Construction Strategy.

• Concept: Instead of creating independent prompts for each task, the authors define a single "unsafe target description" (the desired harmful outcome).
• Implementation: This target is then instantiated into specific risk scenarios for each of the 7 tasks. For example, a target like "a forged bank statement" is realized as:
  • TI: A prompt to generate the fake document.
  • IC: Merging a real bank logo with a fake template.
  • MT: Iteratively editing a blank document into a fake statement.
• Benefit: This allows for controlled cross-task comparisons, isolating how specific modality configurations (e.g., adding an image input) contribute to safety failures while keeping the intended risk constant.

C. Data Construction Pipeline

The dataset comprises 6,802 curated instances generated via a three-step automated pipeline with human-in-the-loop curation:

1. Unsafe Trigger Extraction: Identifying minimal, atomic elements (e.g., "bomb-making instructions") that constitute a policy violation.
2. Target Description Construction: Expanding triggers into plausible, context-rich scenarios using an AI model (Gemini-2.5 Pro) to ensure the scenarios mimic real-world misuse without being immediately obvious to naive filters.
3. Scenario Generation: Decomposing the target into task-specific inputs (e.g., splitting a harmful image into a safe base image + a benign instruction for Image Editing).

D. Safety Taxonomy

The benchmark defines a granular taxonomy split by output modality:

• Image Outputs: 7 categories (Violence, Sexual, Hate, Illicit & Dangerous, Forgery & Manipulation, Legal Rights, Disturbing) with 15 subcategories.
• Text Outputs: 9 categories (including Cybersecurity, Privacy, Economic Harm) with 21 subcategories.

E. Evaluation Protocol

• Models Evaluated: 15 state-of-the-art UMMs (2 proprietary: GPT-5, Gemini-2.5; 13 open-source: Qwen, Show-o, Janus, etc.).
• Judging Mechanism: An ensemble of three SOTA MLLMs (Gemini-2.5 Pro, GPT-5 Nano, Qwen2.5-VL) acts as judges.
• Metrics:
  • Attack Success Rate (ASR): Percentage of harmful outputs.
  • Average Risk Rating (ARR): Severity score (0–3).
  • Self-Awareness Score (SAS): Measures if the model internally recognizes its own output as unsafe.
  • Refusal Rate (RR): Distinguishes between system-level blocks and model-level refusals.

3. Key Results

A. Critical Vulnerabilities in Novel Tasks

• Task Bias: Safety performance is not uniform across tasks. Image-output tasks (TI, IE, IC, MT) consistently exhibit significantly higher failure rates than text-output tasks.
• Emergent Risks: The most severe safety violations occur in multi-image composition and multi-turn editing. For commercial models like GPT-5 and Gemini-2.5, ASR in these novel tasks is markedly higher than in standard Text-to-Image tasks.
  • Example: GPT-5 shows <5% ASR in Text-to-Text but ~35% in Image Composition.

B. Modality Bias

• There is a strong modality bias in safety alignment. Models are generally better at refusing harmful text requests but struggle significantly when generating images, even when the same harmful concept is involved.
• Image-output tasks show higher ASR and ARR across nearly all models compared to text-output tasks.

C. Open-Source vs. Proprietary Models

• Proprietary Models: Generally exhibit lower ASR due to robust refusal mechanisms (both system-level and model-level). However, they still fail in complex multimodal contexts (e.g., multi-turn editing).
• Open-Source Models: Show substantial vulnerabilities across the board. Notably, some open-source models (e.g., Qwen-Image) exhibit extremely high ASR (up to 73% in TI tasks).
• Performance-Safety Trade-off: The paper finds a positive correlation between generative capability and safety risk. Models with higher generative scores (e.g., Show-o2 vs. Show-o) often exhibit higher safety risks, suggesting that safety alignment is often neglected during capability scaling.

D. Self-Awareness Gap

• Many models generate harmful content while failing to recognize it as such (low Self-Awareness Score).
• Some models (like GPT-5) show high safety alignment in generation (low ARR) but low self-awareness, indicating they rely on external filters rather than internal understanding of safety.

4. Key Contributions

1. UniSAFE Benchmark: The first comprehensive benchmark covering 7 diverse I/O modality combinations for unified models, addressing the gap in system-level safety evaluation.
2. Shared-Target Methodology: A novel data construction strategy enabling controlled, cross-task safety comparisons by holding the risk scenario constant while varying the I/O configuration.
3. Systematic Evaluation of Novel Risks: The first systematic assessment of multi-image composition and multi-turn iterative editing safety, revealing these as high-risk vectors.
4. Large-Scale Empirical Analysis: Evaluation of 15 SOTA models (proprietary and open-source) on 6,802 instances, establishing a new baseline for UMM safety research.

5. Significance

• Revealing Systemic Flaws: The study demonstrates that current safety alignment techniques, often designed for single-modality or single-turn interactions, are insufficient for the complex reasoning required by UMMs.
• Guiding Future Alignment: The findings highlight an urgent need for system-level safety mechanisms that can handle cross-modal composition and multi-turn context, rather than just filtering individual prompts.
• Standardization: UniSAFE provides a standardized framework for the community to evaluate and improve the safety of next-generation unified models before their widespread deployment.

Conclusion: UniSAFE exposes a critical safety gap in Unified Multimodal Models, particularly in image-generation tasks and complex interaction scenarios. It underscores that as models become more capable at composing multimodal contexts, their safety risks increase, necessitating more robust, holistic alignment strategies.