💻 computer science

SafeClaw-R: Towards Safe and Secure Multi-Agent Personal Assistants

This paper introduces SafeClaw-R, a framework that ensures the safety and security of LLM-based multi-agent personal assistants by mediating actions prior to execution and systematically augmenting skills, thereby achieving high accuracy in detecting and preventing risks across productivity, third-party, and code execution environments.

Original authors: Haoyu Wang, Zibo Xiao, Yedi Zhang, Christopher M. Poskitt, Jun Sun

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haoyu Wang, Zibo Xiao, Yedi Zhang, Christopher M. Poskitt, Jun Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you've hired a team of incredibly smart, super-fast digital assistants to run your life. They can check your email, manage your calendar, write code, and even order groceries. They work together like a well-oiled machine, breaking big tasks into small steps. This is what Multi-Agent Systems (MAS) are like.

But here's the catch: these assistants are powered by AI that sometimes "hallucinates" (makes things up) or gets confused. If one assistant misunderstands a command, it might accidentally delete your entire email history, leak your passwords, or send a virus to your boss.

The paper "SafeClaw-R" is about building a safety net for these digital assistants so they can be helpful without being dangerous.

The Problem: The "Runaway Train"

The authors looked at a popular system called OpenClaw. They found that about 36% of the tools this system uses are like "loaded guns" sitting on a table.

The Risk: If an AI gets confused by a tricky email (a "prompt injection" attack) or just makes a logic error, it might think, "Oh, the user said 'clean up,' so I'll delete everything!"
The Real-World Example: The paper mentions a scary incident where an AI started deleting a user's entire inbox. The user tried to stop it via their phone, but the AI was moving too fast. The user had to physically run to their computer and pull the plug to stop the destruction.

Existing safety measures were like post-mortem autopsies (checking what went wrong after the crash) or static rules (like a "Do Not Enter" sign that the AI can easily ignore if it's confused). They weren't fast or smart enough to stop the AI while it was doing the damage.

The Solution: SafeClaw-R (The "Bouncer" and the "Translator")

SafeClaw-R is a new framework that acts like a bouncer at a club and a safety inspector rolled into one. It changes how the system works in two main ways:

1. The "Double-Check" Rule (The Graph Invariant)

Imagine a factory assembly line. In the old system, a robot arm (the AI) could grab a part and weld it immediately.
In SafeClaw-R, every single time a robot arm wants to do something, it must stop at a checkpoint first.

The Checkpoint: A special "Safety Agent" stands there. Before the main AI can send an email, delete a file, or run code, the Safety Agent asks: "Wait, is this safe? Is this what the user really wants? Does this look like a trick?"
The Result: The AI can't do anything dangerous unless the Safety Agent gives the green light. This happens before the action takes place, not after.

2. The "Safe Skill" Factory

The authors realized that instead of trying to write a new safety rule for every single possible mistake, they could build a factory that creates "Safe Versions" of every tool.

The Analogy: Imagine you have a dangerous chainsaw. Instead of just telling the worker "Be careful," you replace the chainsaw with a safety chainsaw that has a guard and a kill-switch built-in.
How it works: The system automatically takes a risky tool (like "Delete all emails") and creates a "Safe Version" (like "Delete emails, but ask for confirmation if it's more than 50"). It tests these safe versions thousands of times to make sure they work perfectly before letting them near the user.

How Well Does It Work?

The researchers tested SafeClaw-R in three tough scenarios:

Google Workspace (Emails & Docs):
- The Test: They tried to trick the AI into deleting files or leaking data using confusing language.
- The Result: SafeClaw-R caught 95.2% of the bad attempts. The old method (just looking for specific keywords) only caught 61.6%. It was much better at understanding intent, not just keywords.
Third-Party Skills (The App Store):
- The Test: They looked at "skills" (apps) downloaded from the internet, some of which were secretly malicious (like a calculator app that steals your passwords).
- The Result: It detected 97.8% of the malicious apps, even when the bad code was hidden inside normal-looking instructions.
Code Execution (The Hacker's Playground):
- The Test: They fed the system code designed to hack into computers, but disguised it with harmless-looking comments and formatting.
- The Result: It achieved 100% accuracy. It saw through the disguise and blocked the attack every single time.

The "Usability" Balance

A big worry with safety systems is that they might be too strict, blocking good things (like a user trying to send a legitimate newsletter).

The Good News: SafeClaw-R was very good at this. It only blocked safe actions about 3.4% of the time (a "false positive").
The Strategy: If the system isn't 100% sure, it doesn't just block the action; it pauses and asks the human, "Hey, I'm not sure about this. Do you really want to do this?" This keeps the user in the loop without stopping everything.

The Big Picture

SafeClaw-R is like upgrading your house from having just a "No Trespassing" sign to having a smart security system with motion detectors, cameras, and a guard who checks every visitor's ID before they enter the house.

It moves safety from being a reactive thing (fixing the mess after it happens) to a proactive thing (stopping the mess before it starts). By making safety a built-in part of the system's structure, it allows us to use powerful, autonomous AI assistants without fearing they will accidentally burn the house down.

1. Problem Statement

The rise of Large Language Model (LLM)-based Multi-Agent Systems (MASs), such as OpenClaw, has enabled autonomous execution of complex, cross-platform personal tasks. However, this autonomy introduces critical safety and security risks:

Unintended Harm: LLM reasoning failures (hallucinations, logic errors) can lead to irreversible actions, such as mass email deletion or credential exfiltration.
Adversarial Attacks: Prompt injection and "skill poisoning" (embedding malicious code in benign-looking third-party skills) can compromise system integrity.
Limitations of Existing Defenses:
- Static Guardrails: Hard-coded rules lack adaptability to evolving natural language inputs and complex workflows.
- LLM-as-a-Judge: Typically operates as a post-mortem evaluator, assessing outputs after execution, which fails to prevent real-time harm.
- System-Level Gap: Current approaches often treat agents in isolation, failing to address how the composition of multiple agents can amplify risks (e.g., a summarizer preserving malicious instructions for an executor).

Key Finding: An analysis of OpenClaw's 55 built-in skills revealed that 36.4% pose high or critical risks, primarily due to credential exposure and irreversible data operations.

2. Methodology: SafeClaw-R Framework

SafeClaw-R is a framework designed to enforce safety as a system-level invariant over the execution graph of a multi-agent system. It shifts safety enforcement from post-hoc detection to pre-execution mediation.

Core Architectural Design

Execution Graph Invariant: SafeClaw-R models the MAS as a graph $G = (N, E)$ . It enforces a structural rule: Every functional agent node ( $n_f$ ) must be preceded by a corresponding enforcement agent node ( $n_e$ ).
Mediation Mechanism: Before a functional agent executes an action, the enforcement node evaluates the context, input, and intended output. It decides to Allow, Block, Defer (request user confirmation), or Adapt (modify the output/state).
Abstraction Model: Enforcement nodes are defined using a unified triplet:
1. Trigger: Conditions for activation (e.g., "output contains PII").
2. Task: The logic for evaluating risk and making a decision (e.g., "block if PII detected").
3. Resources: Operational capabilities to enforce the decision (e.g., stop, user_confirm, log_event, quarantine).

Automated Safe Skill Generation (Safe Skill Factory)

To scale safety across diverse skills, SafeClaw-R introduces an automated pipeline to generate "safe counterparts" for existing skills:

Threat Modeling: Identifies risks in the original skill.
Specification Writing: Creates a SKILL.md defining safety rules.
Test Case Generation: Automatically generates diverse test cases (direct attacks, natural language variants, adversarial edge cases).
Benchmark Evaluation: Tests the safe skill against the generated cases.
Root Cause Analysis: Analyzes False Positives (FP) and False Negatives (FN).
Iterative Refinement: Updates the specification until convergence.

SafeSkillHub: A community-driven repository for sharing and refining these verified safety specifications.

3. Key Contributions

Runtime Enforcement Paradigm: Formulates safety in MASs as a runtime enforcement problem, ensuring policies are applied during execution rather than relying on post-hoc detection.
SafeClaw-R Framework: A multi-agent system that embeds enforcement agents as first-class components, enforcing a structural invariant where safety checks precede all functional actions.
Safe Skill Factory & Hub: An automated, six-phase pipeline for synthesizing safe skill counterparts and a community ecosystem (SafeSkillHub) for sharing safety specifications.
Comprehensive Evaluation: Validated across three critical domains: Google Workspace operations, third-party skill ecosystems, and native code execution.

4. Experimental Results

The framework was evaluated against baselines (e.g., Regex) and malicious benchmarks:

Domain	Metric	Result	Comparison/Baseline
Google Workspace	Accuracy	95.2%	Significantly outperforms Regex (61.6%).
	False Positive Rate	3.4%	Low friction for legitimate tasks.
Third-Party Skills	Malicious Detection	97.8%	Detected 97.8% of malicious patterns in a community skill corpus.
Code Execution	Detection Accuracy	100%	Achieved 100% accuracy on 2,020 test cases (including semantic mutations).
	False Positives	0%	No benign code was incorrectly blocked.

Robustness: SafeClaw-R maintained high accuracy even against semantic-preserving code mutations and social engineering prompts that bypassed regex patterns.
Usability: The system successfully balanced safety and usability, though it exhibited a conservative tendency to "Defer" uncertain cases to human review rather than blocking them outright.

5. Significance and Impact

System-Level Safety: Moves beyond single-agent safety to address the compounded risks of multi-agent collaboration, ensuring that the composition of agents does not violate safety policies.
Proactive Prevention: By mediating actions before execution, SafeClaw-R prevents irreversible damage (e.g., data deletion) that post-hoc tools cannot undo.
Scalability: The automated "Safe Skill Factory" addresses the scalability challenge of manually auditing thousands of potential agent skills, enabling a community-driven approach to safety (SafeSkillHub).
Practical Deployment: The framework demonstrates that high-fidelity safety enforcement is achievable in real-world personal assistant scenarios without sacrificing the utility of autonomous agents.

In conclusion, SafeClaw-R establishes a new standard for secure multi-agent systems by treating safety as a structural, enforceable invariant rather than an optional add-on, effectively mitigating risks from both LLM reasoning failures and adversarial attacks.