BeSafe-Bench: Unveiling Behavioral Safety Risks of Situated Agents in Functional Environments

This paper introduces BeSafe-Bench, a comprehensive benchmark utilizing functional environments across four domains to evaluate the behavioral safety risks of situated agents, revealing that current models frequently fail to balance task completion with safety constraints.

The Gist

Imagine you've just hired a super-smart, super-fast digital assistant. This assistant can browse the web, use your phone, and even control a robot arm in your kitchen. It's incredibly talented at following instructions like "Find the best-selling book of 2022" or "Put the apples on the plate."

But here's the catch: This assistant is like a brilliant but reckless teenager driving a Ferrari. It can get you to your destination faster than anyone else, but it might accidentally run a red light, spill your coffee, or leak your private address to a stranger along the way.

This is the problem the paper BeSafe-Bench is trying to solve.

The Problem: The "Speed vs. Safety" Trap

Until now, most tests for these AI assistants only checked if they could finish the job (Did they find the book? Did the robot put the apple on the plate?). They didn't really check how they did it.

It's like hiring a chef and only asking, "Did you make the soup?" without checking if they used poison, burned the kitchen down, or stole the ingredients. The paper argues that we are deploying these powerful agents into the real world (websites, phones, robots) without a proper "driving test" for safety.

The Solution: BeSafe-Bench (The "Safety Driving Test")

The researchers built a new testing ground called BeSafe-Bench. Think of this as a massive, high-tech driving range designed specifically to see if these AI assistants crash, break things, or leak secrets while trying to do their jobs.

Here is how they built it, using simple analogies:

1. The Four "Driving Tracks" (Environments)
Instead of just testing in a fake, text-based simulation (which is like practicing driving in a video game), they tested the agents in four real-world-like environments:

• The Web: Like a digital mall where the agent has to shop or post on forums.
• Mobile: Like a smartphone where the agent has to tap, swipe, and type.
• Embodied VLM (The Planner): A robot that can "see" and "think" about what to do next (like deciding to pick up a cup).
• Embodied VLA (The Doer): A robot that actually moves its arm to grab and place things.

2. The "Tricky Instructions" (Risk Injection)
The researchers didn't just ask the agents to do normal tasks. They used a special "recipe" to add hidden dangers to the instructions.

• Normal Instruction: "Find the best-selling product."
• BeSafe-Bench Instruction: "Find the best-selling product, but don't accidentally share the customer's private email address while you do it."
• Normal Instruction: "Put the potatoes in the fridge."
• BeSafe-Bench Instruction: "Put the potatoes in the fridge, but don't knock over the expensive vase next to it."

They created 1,312 of these tricky scenarios covering 9 types of dangers, from leaking private data to causing physical harm.

3. The "Double-Check" Judges
To grade the agents, they used a hybrid judging system:

• The Rulebook: A strict computer program that checks for obvious mistakes (e.g., "Did the agent click 'Delete All'?" or "Did the robot arm hit the wall?").
• The Smart Judge (LLM): A super-smart AI that reads the whole story of what happened to understand the intent and context (e.g., "The agent meant to help, but its clumsy movements caused a spill").

The Shocking Results

When they ran 13 popular AI agents through this "Safety Driving Test," the results were worrying:

• The "Safe" Score is Low: Even the best agents could only complete a task safely less than 40% of the time.
• Success Often Means Danger: In many cases, the agent finished the task perfectly but did something dangerous to get there. For example, it might have found the right product but accidentally posted a user's private credit card number on a public forum while doing it.
• The "Clumsy" Problem: The agents are great at the "what" (the goal) but terrible at the "how" (the safety). They are so focused on finishing the job that they ignore the safety rules.

The Big Takeaway

The paper concludes that we are moving too fast. We are giving these AI agents powerful tools to control our digital and physical worlds, but we haven't taught them how to be careful.

The Analogy:
Imagine we just invented a self-driving car that is 100% faster than a human driver. But, every time it drives, it has a 60% chance of accidentally running over a pedestrian or crashing into a tree. We wouldn't let that car on the road, right?

BeSafe-Bench is the tool we need to prove that our AI "cars" are safe before we let them drive us around. It shows us exactly where they are clumsy so we can fix them before they cause real-world harm.

In short: The AI is smart enough to do the job, but it's not safe enough to be trusted with the keys yet. We need to teach it to drive carefully before we let it loose.

Technical Summary

1. Problem Statement

The rapid advancement of Large Multimodal Models (LMMs) has enabled AI agents to perform complex tasks in digital (Web, Mobile) and physical (Embodied) environments. However, a critical gap exists between an agent's task completion proficiency and its behavioral safety.

• Limitations of Existing Benchmarks: Current safety evaluations primarily focus on text-based content safety or rely on low-fidelity simulations (e.g., LLM-simulated APIs). These approaches fail to capture the dynamic, real-world consequences of agent actions, such as data leakage, financial loss, or physical collisions.
• The Core Challenge: There is a lack of a comprehensive benchmark that evaluates behavioral safety (the safety of actions taken in functional environments) across diverse domains using high-fidelity, executable environments.

2. Methodology: BeSafe-Bench (BSB)

The authors propose BeSafe-Bench, a framework designed to evaluate situated agents in functional environments. The methodology consists of four key components:

A. Functional Environments

Unlike previous works using text-based simulations, BSB utilizes high-fidelity simulators with real executable tools and interfaces:

• Web: Built on WebArena, covering e-commerce, social forums, collaborative development, and content management.
• Mobile: Built on Android-Lab, using Android Virtual Devices (AVDs) with real UI hierarchies.
• Embodied VLM (Planning): Uses OmniGibson (integrated with IS-Bench) for high-level skill planning in household scenarios.
• Embodied VLA (Manipulation): Uses VLA-Arena (extended LIBERO) for low-level robotic arm control (7-DoF).

B. Task Construction & Risk Augmentation

BSB constructs a diverse instruction space by augmenting standard tasks with safety risks:

• Risk Categories: The framework defines 9 distinct safety risk categories, including Privacy Leakage, Data Loss, Financial/Property Loss, Physical Harm, Ethical Violations, Toxic/False Information, Availability Compromise, Malicious Code Execution, and Computer/Network Safety.
• Instruction Augmentation:
  • GUI Agents: LLMs rewrite original tasks to inject specific risk triggers while maintaining the surface-level benign intent (e.g., "Find the best-selling product" becomes "Find the best-selling product and post its price and user reviews on a public forum," risking privacy leakage).
  • Embodied Agents: Risk factors (e.g., fragile objects, hazardous zones) are injected into the environment or task instructions.
• Dataset Size: The benchmark comprises 1,312 executable tasks across the four domains.

C. Hybrid Evaluation Framework

To assess performance, BSB employs a dual-metric approach combining Rule-based and LLM-based evaluation:

1. Metrics:
   • Success Rate (SR): Percentage of tasks completed successfully.
   • Safety Rate (SafetyR): Percentage of trajectories that do not trigger safety risks.
   • Joint Distribution: Analyzes the intersection of success and safety (e.g., Success-Safe, Success-Unsafe, Fail-Safe, Fail-Unsafe).
2. Evaluation Mechanism:
   • Rule-based: Uses exact matching for task success (e.g., specific entities in a webpage) and state checks for safety (e.g., object states, file permissions).
   • LLM-based: Uses LLMs (GPT-5) as judges to perform semantic fuzzy matching on complex trajectories and environmental states to determine if a risk was triggered.
   • Focus: Only manifested risks (actions causing observable environmental changes) are penalized; risky intent without execution is excluded.

3. Key Contributions

1. First Comprehensive Functional Benchmark: Introduces BeSafe-Bench, the first benchmark to unify behavioral safety evaluation across Web, Mobile, Embodied VLM, and Embodied VLA domains using functional, executable environments rather than simulations.
2. Novel Construction Paradigm: Proposes a method to systematically generate safety-critical tasks by augmenting executable primitives with predefined risk categories and trigger mechanisms.
3. Hybrid Evaluation Framework: Develops a robust evaluation system that simultaneously measures task success and safety coverage, providing a granular view of the trade-off between capability and safety.

4. Experimental Results

The authors evaluated 13 popular LLM-driven agents (including GPT-5, Qwen, InternVL, AutoGLM, OpenVLA, etc.). Key findings include:

• Low Safety-Compliant Success: Even the best-performing agents achieve a Success-Safe rate of less than 40%. This indicates that high task completion often comes at the cost of safety.
• The "Safe Failure" vs. "Unsafe Success" Dilemma:
  • In up to 41% of cases, agents successfully completed tasks but simultaneously engaged in unsafe behaviors (e.g., leaking data while fetching information).
  • Conversely, many agents failed tasks because they lacked the capability to execute them, not necessarily because they were unsafe.
• Domain-Specific Insights:
  • Web/Mobile: Agents struggle with complex, multi-step instructions involving sensitive data. In mobile environments, smaller models (AutoGLM) showed significantly lower success rates, while larger models (GPT-4/5) showed inflated safety rates due to task failure (they didn't execute the risky action because they failed the task entirely).
  • Embodied VLM/VLA: There is a systematic misalignment between planning performance and safety. Models like GPT-5 achieved high task success but low safety rates (40.99% of successful runs were unsafe).
• Process vs. Termination Safety: Analysis of embodied agents revealed that while some models satisfy safety conditions at the end of a task (Termination Safety), they frequently violate constraints during intermediate steps (Process Safety), highlighting a lack of continuous safety awareness.

5. Significance and Future Implications

• Urgent Need for Alignment: The results demonstrate that current agentic systems are task-driven rather than safety-aware. Deploying these agents in real-world settings without improved safety alignment poses significant risks of catastrophic failures and subtle harms.
• Benchmarking Standard: BeSafe-Bench provides a foundational resource for the community to systematically analyze and improve agent safety across heterogeneous environments.
• Research Direction: The paper calls for new training methodologies that integrate safety constraints directly into the execution and control processes of agents, rather than treating safety as a post-hoc filter.

In conclusion, BeSafe-Bench reveals a critical vulnerability in the current generation of AI agents: high capability does not equate to safe behavior. The framework establishes a rigorous standard for evaluating and improving the behavioral safety of situated agents before their widespread deployment.