Technical Summary: OOPSIEVERSE

1. Problem Statement

Despite rapid advancements in robotic manipulation, physical safety remains a critical barrier to deploying household robots. Current simulation-based training and evaluation paradigms suffer from a fundamental disconnect: they define success solely by task completion, ignoring the physical consequences of unsafe interactions. In standard simulators, a robot can "succeed" by breaking objects, spilling liquids, or overheating components, behaviors that would be catastrophic in the real world. Existing safety benchmarks often rely on task-specific constraints, hand-designed reward shaping, or abstract cost functions (e.g., collision counts) that do not model the nuanced, physically grounded nature of real-world damage (mechanical, thermal, or fluid). This leads to policies that appear successful in simulation but are unsafe or destructive upon real-world deployment.

2. Methodology

The authors introduce OOPSIEVERSE, a unified simulation framework and benchmark designed to bridge the gap between simulation and real-world safety by explicitly modeling damage. The framework consists of two core components: DAMAGESIM and OOPSIEBENCH.

A. Theoretical Framework: Damage-Aware POMDP

The authors reformulate the standard Partially Observable Markov Decision Process (POMDP) into a Damage-Aware POMDP ($M_{DA}$).

• State Augmentation: The state space is expanded from $S$ to $S_{DA} = S \times S_h$, where $S_h$ represents a "health" state for each object and the robot.
• Mechanism: This health state is updated at every timestep based on physical interaction signals (forces, temperature, fluid contact). It influences observations, rewards, and termination conditions, allowing the agent to learn policies that avoid damage, not just achieve goals.

B. DAMAGESIM: Damage Detection and Quantification

DAMAGESIM is a simulator-agnostic plugin that detects and quantifies damage by converting physical simulator states into a unified "health" metric (ranging from 100 to 0). It models three primary damage modalities:

1. Mechanical Damage: Captures harm from impulsive events (collisions, drops) and sustained/constrained contacts (compression, tension). It decomposes contact forces into parallel (impulsive) and orthogonal (sustained) components relative to link acceleration. A unified load proxy $\epsilon_{mech}$ is computed, and damage accumulates if this load exceeds a material-specific threshold ($E_{mech}$).
2. Thermal Damage: Models harm from exposure to temperatures outside a safe operating range (melting, burning, freezing). Damage accumulates linearly based on the deviation from thresholds ($T_{hot}, T_{cold}$) and material-specific scaling factors. Note: Currently implemented in OmniGibson due to its temperature tracking capabilities.
3. Fluid Damage: Models harm from liquid exposure (corrosion, short-circuiting). Damage is calculated based on the number of liquid particles in contact with an object link exceeding a tolerance threshold ($C_{fluid}$). Note: Currently implemented in OmniGibson.

The system tracks health per object link, with the object's total health defined as the minimum health of its constituent links.

C. OOPSIEBENCH: The Benchmark Suite

OOPSIEBENCH comprises 32 household manipulation tasks across two simulators: OmniGibson (Nvidia Omniverse) and RoboCasa (MuJoCo).

• Task Design: Tasks are designed to expose agents to specific damage modes (mechanical, thermal, fluid) and distinguish between "goal achievement" and "safe execution."
• Scenarios: Includes short-horizon tasks isolating specific damage types and long-horizon composite tasks requiring sustained safety-aware decision-making.
• Dataset: Includes 90 human-collected demonstrations across five tasks, split between those collected with real-time health feedback and those without.

3. Key Contributions

1. Unified Damage Framework: A general, physics-grounded method to represent damage as an explicit, task-agnostic signal (health) derived from standard simulator outputs (forces, temperature, fluid contact).
2. Cross-Platform Implementation: DAMAGESIM is instantiated in two distinct physics backends (Omniverse and MuJoCo), demonstrating portability.
3. Comprehensive Benchmark: OOPSIEBENCH provides a standardized suite of 32 tasks and a dataset of demonstrations to evaluate safety-performance trade-offs.
4. Versatile Utility: The framework supports multiple use cases: real-time teleoperation feedback, imitation learning (IL) data curation, reinforcement learning (RL) reward shaping, and VLA benchmarking.

4. Experimental Results

The authors evaluated OOPSIEVERSE across four use cases:

• Imitation Learning (IL):
  • Real-time Feedback: Human operators provided with real-time health bars and color-coded damage cues produced significantly safer demonstrations.
  • Data Curation: Policies trained on data filtered by health (removing episodes or timesteps with damage) achieved much higher "Safe Task Completion Rates" compared to policies trained on unfiltered data, with minimal drop in overall task completion. For example, on the "Wipe Countertop" task, unfiltered policies had a 100% task completion rate but only 35% safe completion, whereas health-filtered policies significantly improved safety.
• Reinforcement Learning (RL):
  • Using the damage signal as a negative reward (penalty) allowed agents to learn safer behaviors.
  • Fine-tuning: Pre-trained policies (Flow-matching and Gaussian) fine-tuned with damage penalties showed substantial improvements in safe success rates (e.g., increasing from 20% to 100% on "Move Cup of Water").
  • Training from Scratch: RL agents trained from scratch using only the damage penalty achieved >80% safe task completion, compared to near 0% when trained on task rewards alone.
• VLA Benchmarking:
  • Evaluating the state-of-the-art NVIDIA GR00T model revealed a significant gap between task success and safety. While GR00T achieved high task completion rates (e.g., 92% on "Open Microwave Door"), its Safe Completion Rate was extremely low (4%), and average environment health was severely degraded. This highlights that standard VLA evaluation misses destructive behaviors.
• Sim-to-Real Transfer:
  • Policies trained in OmniGibson with damage-aware filtering were transferred to a Franka Panda robot.
  • The damage-aware policy reduced unsafe behaviors (e.g., spilling water on laptops, toppling bottles) by 60% compared to a baseline policy trained without health feedback, while maintaining comparable task completion rates.

5. Significance and Claims

The paper claims that OOPSIEVERSE provides a general, practical, and scalable foundation for research into safe robot manipulation. Its primary significance lies in:

• Bridging the Sim-Real Gap: By explicitly modeling physical damage, it prevents the "simulation gap" where policies learn to succeed by destroying the environment.
• Standardization: It offers a unified, physics-grounded signal for safety that is independent of specific task rewards or hand-crafted constraints, enabling consistent evaluation across diverse environments and robots.
• Enabling Safer Learning: It demonstrates that safety can be learned effectively through both imitation (via better data collection and curation) and reinforcement learning (via damage-aware rewards).
• Real-World Applicability: The successful sim-to-real transfer of safer policies validates that simulation-based damage modeling can directly improve physical robot safety.

The authors acknowledge limitations, noting that their damage models are approximations of complex physical processes (fracture mechanics, fluid dynamics) and require manual parameterization for object materials. However, they argue these approximations are sufficient to capture the qualitative differences between safe and unsafe behaviors necessary for training and evaluation.