Ensuring Safe Physical AI in Urban Mobility via Hazard-Informed Synthesized Envelopes

Technical Summary: Ensuring Safe Physical AI in Urban Mobility via Hazard-Informed Synthesized Envelopes

Problem Statement
The integration of heterogeneous robotic systems—ranging from sidewalk delivery robots to autonomous robotaxis and road-maintenance vehicles—into public urban spaces presents a critical safety challenge. Unlike controlled industrial environments, urban mobility involves complex, dynamic interactions with human agents. Current safety paradigms are often fragmented: sidewalk robots rely on low-speed "yield-to-all" behaviors, robotaxis utilize multi-layered sensor redundancies and deterministic envelopes, and maintenance robots depend on geofencing and V2X communication.

The core problem identified is that safety is frequently treated as a static constraint isolated within individual software modules. However, safety is inherently a system-level feature that evolves as information flows through different layers of system design (from symbolic rules to physical control). There is a lack of a unified framework that bridges systematic hazard analysis with runtime enforcement, ensuring that safety objectives remain consistent across varying levels of abstraction.

Methodology
The authors propose a unified framework centered on Hazard-Informed Synthesized Envelopes and a Physical AI Harness. The methodology operates through three primary components:

1. Hazard-Informed Safety Envelopes:
   Instead of training models solely on normal driving data, the framework utilizes a pipeline to generate synthetic training scenarios specifically around known safety-critical situations. These scenarios are derived from explicitly defined assets, exposure modes, and hazard scenarios. This approach allows machine learning models to learn the gradual transition from normal operation to hazardous conditions, rather than just rare collision events. The resulting models continuously estimate the proximity to safety constraint violations, enabling predictive safety assessment.

2. Cross-Layer Safety Transformation:
   The framework models safety across a hierarchy of three world models, where safety is represented differently at each level but maintains a consistent underlying objective:

   • Symbolic Layer: Represents the environment using entities, relationships, and rules (e.g., traffic laws, ethical constraints). Safety is defined by permissible actions (e.g., yielding to pedestrians).
   • Spatial Layer: Translates symbolic rules into geometric motion plans. Safety is expressed through collision probabilities, clearance distances, and predicted occupancy of dynamic obstacles.
   • Dynamic Layer: Governs physical interactions, focusing on actuator limits, vehicle dynamics, and stability. Safety is defined by the physical feasibility of control commands.

   The framework facilitates bidirectional flow: forward transformation converts high-level rules into low-level physical commands, while feedback transformation updates higher-level representations based on real-world execution deviations. The "Urban Safety Envelope" emerges from the consistent intersection of constraints across all three layers.

3. Runtime Enforcement via Physical AI Harness:
   To ensure that AI-generated decisions remain consistent with safety constraints during execution, the paper introduces a Physical AI Harness based on the Projection, Isolation, and Transfer (PIT) mechanism:

   • Projection: Constrains ML model outputs by projecting commands onto a predefined region of acceptable behavior (the safety envelope). Violating commands are modified or rejected before reaching the controller.
   • Isolation: Allocates dedicated computational resources to prevent AI workloads from degrading the performance of safety-critical functions, ensuring deterministic behavior.
   • Transfer: Provides a controlled mechanism to hand authority from the AI model to a verified fallback controller (e.g., emergency braking or rule-based control) when safety margins fall below acceptable thresholds.

Key Contributions

• Unified Safety Framework: The paper presents a cross-layer approach that bridges systematic hazard analysis with runtime enforcement, moving beyond static, module-specific safety constraints.
• Hazard-Informed Synthetic Data: It proposes a methodology for generating training data specifically around safety-critical edge cases to improve model interpretability and predictive safety assessment.
• Hierarchical Safety Representation: It formalizes the transformation of safety objectives from symbolic rules to spatial plans and finally to dynamic control commands, ensuring consistency across abstraction levels.
• Runtime Safety Mechanism: The introduction of the Physical AI Harness (PIT) provides a concrete mechanism for enforcing safety semantics during the operational lifecycle of autonomous systems.

Results and Validation Approach
The paper does not present empirical experimental results from a deployed fleet. Instead, it outlines a proof-of-concept implementation currently in development. The authors claim to be validating the framework using a closed-loop pipeline integrating three NVIDIA platforms:

• NVIDIA Isaac Sim and Isaac Lab: For physically accurate robotics simulation and learning.
• NVIDIA Cosmos: For generating high-quality synthetic data and simulating diverse urban scenarios, including safety-critical edge cases.
• NVIDIA Omniverse: For building high-fidelity digital twins of urban environments.

The goal of this implementation is to systematically evaluate the urban safety envelope across thousands of scenario variations and validate the PIT runtime enforcement mechanisms under controlled yet realistic conditions.

Significance and Claims
The authors claim that this framework offers a rigorous and scalable foundation for safe urban mobility. By treating safety as a cross-layer property that evolves with abstraction levels, the system can anticipate boundary conditions before violations occur. The integration of hazard-informed synthesis with runtime enforcement ensures that safety is preserved not only during model development but throughout the complete operational lifecycle. The paper positions this approach as a necessary evolution from static safety constraints to dynamic, predictive, and enforceable safety systems capable of handling the complexity of heterogeneous robotic systems in public spaces.