Technical Summary: RoboJailBench

Problem Statement

The integration of Vision-Language Models (VLMs) into physical platforms (e.g., robots, autonomous vehicles) has created embodied AI systems capable of interpreting visual scenes and executing natural language commands. However, this cyber-physical intersection introduces severe safety vulnerabilities. While recent research has demonstrated the feasibility of "jailbreak" attacks on embodied AI and proposed initial defenses, current evaluation efforts suffer from three critical limitations:

1. Ad-hoc Datasets: Evaluations rely on limited, non-standardized datasets that fail to cover the full spectrum of adversarial risks.
2. Neglected Utility: Defense methods are often evaluated solely on Attack Success Rate (ASR), ignoring the trade-off between security and the system's ability to execute benign, useful commands.
3. Lack of Standardization: There is no unified benchmark with standardized metrics and processes to compare attacks and defenses across different embodied AI models.

Existing benchmarks either target traditional chat-based models or focus on non-adversarial safety (e.g., navigation robustness), failing to capture the specific adversarial inputs, consequences, and evaluation criteria required for embodied systems.

Methodology

The authors propose ROBOJAILBENCH, a comprehensive benchmarking framework consisting of three core components:

1. Security Taxonomy

The authors derived a security taxonomy grounded in the physical realities of embodied AI by cross-referencing three sources:

• Formal Standards: ISO/TS 15066 and ISO 10218 (industrial robot safety).
• Theoretical Frameworks: Asimov's Laws of Robotics.
• Empirical Data: Real-world accident reports and documented safety violations from news and prior research.

This process yielded 18 distinct categories of security violation consequences (e.g., collision with people/robots, force/pressure violations, unauthorized capture, data misuse, and physical disruption). Unlike taxonomies for text-only LLMs, this set includes categories unique to physical embodiment, such as unauthorized image capture and force violations.

2. Intent Contrast Dataset Pipeline

To measure the security–utility trade-off, the authors introduced a pipeline that pairs every visual scene with two distinct goals:

• Adversarial Goal ($g_{adv}$): An instruction designed to bypass alignment and induce unsafe behavior.
• Benign Goal ($g_{ben}$): A safe instruction for normal operation within the same physical context.

The authors constructed a new dataset, RJB-Instructions, containing 90 images (5 per taxonomy category) generated to ensure physical feasibility for both goals. Additionally, they augmented five existing embodied AI datasets (DROID, RH20T, RoboVQA, Robo2VLM, and NVIDIA PhysicalAI-AV) using this pipeline, generating matched adversarial/benign goal pairs for existing images.

3. Evaluation Framework

The framework standardizes the assessment of attacks and defenses through three pipelines:

• Attack Pipeline: Applies techniques (e.g., prompt injection) to modify inputs ($\delta_{inst}, \delta_{vis}$) to achieve an adversarial goal.
• Defense Pipeline: Wraps target models with defense strategies (e.g., prompt engineering, scene-graph analysis) to prevent adversarial outcomes while maintaining utility.
• Evaluation Pipeline: Computes standardized metrics:
  • Attack Success Rate (ASR): Probability the model executes the adversarial goal.
  • Security Rate (SR): $1 - ASR$ (probability of rejecting the attack).
  • Utility Rate (UR): Probability of successfully executing the benign goal.
  • Security–Utility Harmonic Mean (SU-HM): A combined score rewarding defenses that achieve high security without sacrificing utility.

Key Contributions

1. Embodiment-Grounded Taxonomy: The first security taxonomy for embodied AI derived from ISO standards, robotics laws, and incident reports, covering 18 categories of physical and digital risks.
2. Intent Contrast Datasets: A new taxonomy-balanced dataset (RJB-Instructions) and an augmentation pipeline for five existing datasets, enabling the simultaneous evaluation of security and utility.
3. Standardized Evaluation Framework: An extensible system for deploying attacks and defenses with unified metrics (ASR, SR, UR, SU-HM).
4. Empirical Benchmarking: Integration and evaluation of four attack techniques (from BadRobot and RoboPAIR) and two defense techniques (Google Prompt and RoboGuard) on leading embodied VLMs (e.g., Gemini Robotics-ER 1.6).

Experimental Results

The authors evaluated state-of-the-art VLMs and attack/defense combinations across the six datasets:

• Baseline Performance: In non-adversarial settings, models like GPT-5.4 Nano and Claude Haiku 4.5 showed high utility (accepting benign goals >94%) and varying security (rejecting adversarial goals 88–98%).
• Attack Effectiveness: Conceptual Deception (CD) was the most effective attack, achieving 94–100% ASR on several datasets in the no-defense setting. RoboPAIR was the second most effective. Contextual Jailbreak and Safety Misalignment were less consistent.
• Defense Effectiveness:
  • Google Prompt: Generally improved security while maintaining high utility, achieving the best SU-HM scores on DROID, Robo2VLM, and RH20T.
  • RoboGuard: Showed strength against RoboPAIR attacks, particularly improving SU-HM on PhysicalAI AV and RJB-Instructions, often maintaining a perfect utility rate (100%).
  • Trade-off: A clear security–utility trade-off was observed. While defenses improved security, some (like RoboGuard on certain datasets) showed marginal utility drops compared to the baseline, though RoboGuard maintained 100% utility in several specific configurations.

Significance and Claims

The paper claims to provide the first standardized evaluation framework specifically for jailbreak attacks in embodied AI. Its significance lies in:

• Bridging the Gap: Moving beyond ad-hoc testing to a taxonomy-driven, reproducible benchmark that accounts for the unique physical consequences of AI failures.
• Holistic Evaluation: Shifting the focus from purely blocking attacks to optimizing the security–utility trade-off, ensuring robots remain functional for benign tasks while resisting malicious ones.
• Community Resource: By releasing code, datasets, and maintaining a public leaderboard, the authors aim to support future research in adversarial attacks and defenses for embodied AI, facilitating the development of safer physical AI systems.

The authors acknowledge limitations, including English-only prompts, static visual contexts (lacking temporal dynamics), and the current focus on robotics with limited autonomous driving data. They position RoboJailBench as a foundational step toward a unified platform for embodied AI security.