Technical Summary: LabGuard

Problem Statement

While embodied agents are increasingly capable of executing complex laboratory procedures using Vision-Language-Action (VLA) models and imitation learning, ensuring safety in dynamic laboratory environments remains a significant challenge. Current safety research often assumes that safety information is already formalized. However, in real-world laboratories, safety knowledge exists primarily as unstructured natural language (e.g., Standard Operating Procedures, manuals, protocols, and chemical handling rules).

The core problem addressed by this paper is the "missing link" between these natural-language safety rules and the machine-checkable runtime constraints required by robot controllers. Existing approaches often overlook the intermediate step of transforming textual safety rules into executable specifications that can be actively monitored and enforced at the controller boundary during execution. The authors identify three key challenges:

1. Representation: How to represent laboratory safety language so controllers can activate and check rules.
2. Grounding: How to map diverse natural-language sources (SOPs, manuals, etc.) into a typed, executable representation.
3. Compilation: How to compile these grounded representations into runtime monitors that can check live states and intervene (pass, modify, or block) actions before execution.

Methodology: The LabGuard Suite

The authors propose LabGuard, a language-to-execution safety suite designed to ground natural-language laboratory rules into executable runtime guards. The system operates through a three-stage pipeline:

1. LabGuard-IR (Intermediate Representation)

The authors define LabGuard-IR, a typed executable representation that serves as the target for grounding and the input for compilation. Unlike simple text descriptions, LabGuard-IR captures the execution semantics needed for controller-side checking. It consists of six fields:

• $\hat{t}$ (Constraint Type): Selects the safety logic (e.g., chemical incompatibility, temperature limits, sequence checking).
• $\hat{S}$ (Substance/Entity Set): Links textual mentions to concrete, trackable laboratory entities.
• $\hat{p}$ (State Predicate): Defines the condition evaluated against the live state or proposed action.
• $\hat{T}$ (Action Scope): Activates the rule only for relevant controller actions.
• $\hat{v}$ (Severity): Determines the priority of the violation.
• $\hat{i}$ (Intervention Mode): Specifies the response (e.g., log, modify, block).

2. LabGuard-Bench (Supervised Benchmark)

To train and evaluate the grounding process, the authors constructed LabGuard-Bench.

• Data: It contains 812 supervised annotations expanded from 203 seed laboratory rules.
• Sources: Seeds are drawn from four distinct families: laboratory safety manuals, PubChem GHS statements, expert-authored controller-facing rules, and chemical safety references.
• Annotation: Five annotators with chemistry/robotics backgrounds reviewed the data, ensuring semantic equivalence between paraphrases and canonical rules. The dataset covers 15 constraint types grouped into four families (material compatibility, physical/quantitative handling, process/equipment checks, and environment/PPE/waste).

3. LabGuard-Grounder and Pipeline

• LabGuard-Grounder: A trainable component (implemented as a LoRA fine-tuned language model) that maps natural-language rules to the LabGuard-IR schema. It is trained via supervised sequence generation to recover the structured fields required for downstream compilation.
• LabGuard Pipeline:
  • Compile-Time: The grounded IR instance is validated against the schema and compiled into an executable runtime monitor ($\rho$). The compiler instantiates check functions ($f_{check}$), scope predicates ($\phi_{scope}$), and parameters ($\theta$) based on the IR fields.
  • Runtime: At each control step, the base controller proposes a raw action. The runtime executor activates relevant monitors based on the action scope. The safety filter ($SafeFilter$) evaluates the action against the live laboratory state (including substance identities, volumes, temperatures, and kinematics). It then returns a verdict: pass, modify, or block.

Key Contributions

1. LabGuard-IR: A typed representation that makes laboratory safety language controller-usable by explicitly capturing constraint types, entity bindings, action scopes, and intervention modes.
2. LabGuard-Bench & Grounder: A supervised benchmark with 812 annotations and a corresponding grounding model that maps diverse natural-language rules into the executable IR schema.
3. LabGuard Pipeline: A full system that compiles grounded IR instances into runtime monitors and integrates them with embodied agents (specifically ACT policies in LabUtopia) to test hazard reduction and task preservation.

Experimental Results

The authors evaluated LabGuard across three stages: grounding, offline replay, and live runtime execution.

• Laboratory-Language Grounding: Under source-held-out settings (testing generalization to unseen rule sources), LabGuard-Hybrid (combining learned grounding with rule-based normalization) achieved a 79.4 task-scope F1. In chemistry-held-out settings (unseen chemical categories), it reached 91.8 F1.
• Compiled-Monitor Replay: In offline stress tests involving 70 unsafe scenarios, the monitors compiled from LabGuard-Hybrid grounded rules achieved a 58.6% catch rate and 69.5 F1, approaching the oracle bound of gold labels (77.1% catch, 84.4 F1). The results indicate that accurate task-scope routing and substance binding are critical for downstream safety, even if exact predicate recovery is challenging.
• Runtime Execution (LabUtopia): Integrated with ACT policies in LabUtopia (Levels 1–4):
  • Safety: Unsafe events were reduced from 39.5% to 23.8% after monitor compilation.
  • Task Success: The safety layer preserved or improved task success rates. For example, in the "HeatLiquid" task, success rate increased from 33.3% (Base) to 46.7% (Full LabGuard).
  • Overhead: Intervention rates remained low (≤0.5%), indicating the system does not introduce spurious blocks for safe actions.
  • Ablation: The "Action Filter" module contributed the most to success rate improvements, while semantic checks and risk scoring provided complementary benefits.

Significance and Claims

The paper claims that LabGuard provides an auditable path from human-readable laboratory safety knowledge to executable monitors. By separating grounding, compilation, and execution, the system bridges the gap between natural language and robot control loops without requiring the base policy to be retrained.

The authors emphasize that their work demonstrates the feasibility of connecting laboratory natural language to runtime monitors in simulation. They explicitly state that the results do not yet establish real-world laboratory safety, as physical deployment would require validation against real robot dynamics, sensing noise, and institutional procedures. The work is presented as a foundational step toward "representable, learnable, and executable" safety for embodied scientific agents.