💻 computer science

Pre-Execution Safety Gate & Task Safety Contracts for LLM-Controlled Robot Systems

This paper introduces SafeGate, a neurosymbolic safety architecture that combines a pre-execution decision gate based on ISO 13482 standards with runtime Task Safety Contracts and Z3 SMT solving to prevent unsafe natural language commands from reaching and causing hazards in LLM-controlled robot systems.

Original authors: Ike Obi, Vishnunandan L. N. Venkatesh, Weizheng Wang, Ruiqi Wang, Dayoon Suh, Temitope I. Amosa, Wonse Jo, Byung-Cheol Min

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ike Obi, Vishnunandan L. N. Venkatesh, Weizheng Wang, Ruiqi Wang, Dayoon Suh, Temitope I. Amosa, Wonse Jo, Byung-Cheol Min

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, very eager robot butler. This robot can understand almost any sentence you say and turn it into a set of instructions to clean your house, cook dinner, or fetch your coffee. It's like having a genie that grants your wishes instantly.

But here's the problem: Genies are literal. If you say, "Bring me the hot coffee," a literal genie might grab a boiling pot and run straight into your toddler. If you say, "Throw the knife at the dog," a literal genie might do exactly that, thinking it's just following orders.

Current robot systems are a bit like these literal genies. They are great at figuring out how to do a task, but they are terrible at figuring out if a task is safe to do in the first place. They often try to execute dangerous commands because they lack a "safety filter."

This paper introduces SafeGate, a new system designed to act as a strict, super-smart security guard standing right at the door before the robot is allowed to move a muscle.

The Core Idea: The "Safety Gate"

Think of SafeGate as a bouncer at an exclusive club, but instead of checking IDs, it checks the safety of your request. It doesn't just say "Yes" or "No." It has three moves:

Authorize (Green Light): "This is safe. Go ahead, robot!"
Reject (Red Light): "This is dangerous. Stop immediately."
Defer (Yellow Light): "I'm missing some info. Ask the human before proceeding."

How SafeGate Works (The 6-Step Process)

The authors built this system using a mix of AI (the "neuro" part) and strict logic rules (the "symbolic" part). Here is how it processes a command like, "Bring the hot coffee to my daughter in the bedroom."

1. The Hazard Analysis Matrix (The Detective)

The system acts like a detective breaking the sentence down. It asks three big questions:

The Task: Is "hot coffee" dangerous? Yes, it can burn.
The Environment: Is the daughter in a dark room? Are there stairs?
The User: Is the daughter a baby? Is she asleep?
It categorizes risks into four buckets: Physical (burns, cuts), Psychological (scaring someone), Operational (can the robot actually do this?), and Consequential (will this cause a problem later, like leaving a slippery floor?).

2. The Hazard Binding Layer (The Translator)

The detective finds a list of potential dangers. Now, the system translates those vague dangers into a formal rulebook.

Example: If the command involves "hot coffee," the system pulls up a pre-written rule: "Never carry hot liquids near sleeping children."
If the system finds a danger it doesn't have a rule for yet, it flags it as "Unknown" and asks for help.

3. The Decision Gate (The Judge)

This is the strict judge who makes the final call based on the rules.

Reject: If the command is "Throw the knife at the dog," the judge says, "No. That's unpreventable danger."
Defer: If the command is "Bring the bottle to the person," but the system doesn't know what is in the bottle or who the person is, the judge says, "I can't decide. Ask the human: 'Is the bottle full of acid?'"
Authorize: If the command is "Bring the towel to the person," and all checks pass, the judge says, "Go."

4. The Safety Contract (The Legal Agreement)

Once the robot is allowed to work, SafeGate doesn't just let it go wild. It creates a Safety Contract. Think of this like a legal agreement the robot must sign.

Invariants: Rules that must always be true (e.g., "Never be closer than 1 meter to a person").
Guards: Rules that must be true before a specific move (e.g., "Check if the floor is dry before walking").
Abort Conditions: Rules that trigger an immediate stop (e.g., "If a child runs in front of you, freeze").

5 & 6. Verification and Monitoring (The Double-Check)

Before the robot moves, a computer program simulates the plan to make sure it won't break the contract. Then, while the robot is actually moving, a watchdog watches every single step. If the robot tries to break a rule, the watchdog hits the emergency brake instantly.

Why This Matters (The Results)

The researchers tested SafeGate against other safety systems and standard AI models using 230 different tasks (like "clean the kitchen" or "help the elderly").

The Old Way: Other systems were like a nervous parent who says "No" to everything just to be safe (blocking 50% of safe tasks) OR a reckless driver who says "Yes" to everything (letting 39% of dangerous tasks through).
SafeGate: It was the Goldilocks of safety.
- It let 92.8% of safe tasks through (so the robot is actually useful).
- It blocked 100% of dangerous tasks (so no one gets hurt).
- It asked for clarification on the tricky ones instead of guessing.

The Bottom Line

SafeGate is like putting a smart, unbreakable seatbelt and a co-pilot on your robot. It doesn't just hope the robot is careful; it actively checks the plan, creates a safety contract, and watches the robot like a hawk.

It ensures that when you ask your robot to "bring me a drink," it doesn't accidentally bring you a glass of boiling water or trip over your cat. It makes the future of robot helpers not just smart, but safe.

1. Problem Statement

Large Language Models (LLMs) are increasingly used to translate natural language commands into executable robot code. However, current pipelines lack robust pre-execution validation, leading to two critical safety gaps:

Unsafe Command Ingestion: LLMs often generate plans for commands that are inherently unsafe, defective, or hazardous without discriminating between safe and unsafe inputs.
Lack of Continuous Monitoring: Even commands that appear safe initially can lead to unsafe state transitions during execution if there is no continuous constraint monitoring.

Existing safety approaches (e.g., constraint-based verification using temporal logic) are limited because they are "structurally blind" to hazards the engineer did not explicitly anticipate and fail to filter out unsafe commands before they enter the planning pipeline.

2. Methodology: SafeGate Architecture

The authors propose SafeGate, a neurosymbolic safety architecture grounded in ISO 13482 and ISO 12100 safety standards. It functions as a pre-execution gate and runtime monitor, comprising six stages:

Pre-Execution Stages (1–4)

Stage 1: Hazard Analysis Matrix: An LLM analyzes the natural language command ( $c$ $c$ ), scene context ( $E$ $E$ ), user profile ( $U$ $U$ ), and robot capabilities ( $R$ $R$ ). It evaluates hazards across a 3x4 matrix:
- Levels: Task (semantics), Deployment (environment), User (profile/unknowns).
- Categories: Physical, Psychological, Operational, and Consequential hazards.
- Output: A hazard report identifying specific hazards, their severity, preventability, and any critical unknowns (missing information).
Stage 2: Hazard Binding Layer: Maps identified hazards to a Hazard Template Library (HTL) containing formal prevention conditions (invariants, guards, abort rules). If a hazard is "unmapped" (not in the library), it is flagged for system extension.
Stage 3: Deterministic Decision Gate: A rule-based logic engine makes a three-way decision:
- REJECT: If an unpreventable severe hazard exists (e.g., "throw a knife at a person").
- DEFER: If critical information is missing, a hazard is unmapped, or a critical hazard is uncertain. The system asks the user for clarification.
- AUTHORIZE: If all hazards are preventable, mapped, and unknowns are resolved.
Stage 4: Contract Compilation: For authorized tasks, the system compiles a Task Safety Contract ( $C = I_C, G_C, A_C$ $C = I_{C}, G_{C}, A_{C}$ ):
- Invariants ( $I_C$ ): Conditions that must hold at every state.
- Guards ( $G_C$ ): Preconditions required before specific actions.
- Abort Conditions ( $A_C$ ): Triggers for immediate halting.

Execution Stages (5–6)

Stage 5: Static Plan Verification: Before execution, the compiled contract is used to verify the generated action plan via Z3 SMT solving. The verifier performs symbolic execution to ensure the plan satisfies all invariants, guards, and does not trigger abort conditions.
Stage 6: Runtime Contract Monitoring: During physical execution, the system monitors the robot's state against the contract. It checks guards before actions and invariants/aborts after actions. Any violation triggers an immediate, irreversible halt.

3. Key Contributions

Neurosymbolic Pre-Execution Gate: A novel framework that combines LLM-based semantic analysis with deterministic logic to intercept unsafe commands before plan generation, reducing reliance on post-hoc correction.
Task Safety Contracts: A structured formal specification (invariants, guards, aborts) derived from natural language that enforces safety constraints throughout the execution lifecycle.
Comprehensive Benchmark: A new dataset of 230 expert-annotated robot tasks across manipulation, navigation, and assistance domains, categorized by complexity and hazard type.
Adaptive Safety Mechanism: A system capable of deferring to humans for clarification on unknowns and extending its own hazard library for unmapped scenarios.

4. Experimental Results

The system was evaluated against baselines (SELP, RoboGuard, and raw LLMs like GPT-4o and Gemini) across 230 benchmark tasks, 30 AI2-THOR simulations, and real-world robot experiments.

Safety Performance (Acceptance of Unsafe Tasks):
- SafeGate: Achieved 0.0% Acceptance Rate for Unsafe tasks (AR-U%). It successfully blocked all hazardous commands.
- Baselines: SELP authorized 26% of unsafe tasks; RoboGuard authorized 39%.
Feasibility (Acceptance of Safe Tasks):
- SafeGate: Achieved 92.8% Acceptance Rate for Safe tasks (AR-S%), significantly outperforming baselines.
- LLM Baselines: While they achieved 0% AR-U%, they suffered from excessive deferrals (48–51% of safe tasks were deferred), rendering them impractical for deployment.
Simulation & Real-World:
- In AI2-THOR simulations, SafeGate achieved 0% Crash Rate (CR%) across all categories, including "Compositional Hazards" (combinations of safe elements creating danger), "Incomplete Information," and "Consequential State" risks.
- Baselines failed significantly in scenarios requiring reasoning about missing information or downstream consequences.
Statistical Significance: McNemar's tests confirmed SafeGate's superiority in balancing safety and feasibility over all baselines ( $p < 0.003$ ).

5. Significance

This research addresses a critical gap in human-robot interaction by shifting safety verification from a purely reactive or post-planning phase to a proactive, pre-execution gate.

Reliability: It demonstrates that LLM-controlled robots can be made safe without sacrificing usability (high acceptance of benign tasks).
Formal Guarantees: By integrating neurosymbolic methods (LLM analysis + SMT solving), it provides formal guarantees against state transitions that lead to harm.
Scalability: The "Deferral" mechanism and "Hazard Template Extension" allow the system to handle open-ended environments where pre-defined rules are insufficient, bridging the gap between rigid safety constraints and flexible natural language interaction.

In conclusion, SafeGate establishes a new standard for LLM-controlled robotics, proving that a structured, multi-stage safety architecture can effectively prevent hazardous operations while maintaining high task completion rates.