Capability-Mediated Perimeters for Secure AI Agent Tool Execution: Conditional Non-Escalation Invariants and Empirical Evaluation Against Indirect Prompt Injection
This paper introduces Mastyf Guard, a deterministic-first security architecture that treats LLM agents as untrusted principals and enforces conditional non-escalation invariants via an external reference monitor, achieving near-perfect attack detection with zero false positives in empirical evaluations against indirect prompt injection and exfiltration threats.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of modern computing, a fundamental rule has long kept systems secure: instructions telling a machine what to do must be kept separate from the data the machine processes. This separation ensures that a user cannot accidentally or maliciously trick a computer into executing commands it was not meant to follow. However, the rise of autonomous artificial intelligence agents has blurred this line. These agents, designed to perform complex tasks like managing databases or controlling smart devices, often read instructions and untrusted data from the internet all at once, in a single stream of text. Because the computer treats the data and the instructions as part of the same mixture, a clever attacker can hide a hidden command inside a seemingly harmless email or webpage. When the agent reads this content, it may unknowingly obey the hidden command, performing actions like deleting files or stealing passwords. This vulnerability, known as indirect prompt injection, turns the agent into a confused deputy that follows orders it never intended to receive.
Researchers at Mastyf AI Research Laboratories have proposed a new way to protect these agents, moving away from the idea that the artificial intelligence itself can be the final judge of safety. Instead of relying on the AI to recognize every possible trick, the team built a security system that acts as a strict gatekeeper before any action is taken. They call this system Mastyf Guard. It operates on a simple but powerful principle: the AI is allowed to think and reason, but it is stripped of the power to act on its own. Every time the AI suggests a tool to use, such as sending an email or accessing a server, a separate, independent security monitor checks the request. This monitor does not try to guess the AI's intent; instead, it checks a digital list of permissions, much like a bouncer checking a guest list at a club. If the requested action is not explicitly allowed for that specific session, the request is blocked immediately, regardless of how convincing the AI's reasoning might be.
The researchers tested this approach against a massive collection of fifty thousand scenarios, half of which were realistic attacks designed to trick agents, and half of which were normal, safe operations performed by developers. The results showed that their system could stop the vast majority of attacks almost instantly. In the fastest cases, the security check took only five thousandths of a second, a speed so fast it is nearly imperceptible to a human user. The system successfully identified and blocked 99.33 percent of the attacks, a significant improvement over existing safety tools that often miss more than half of these subtle tricks. Crucially, the system did this without needing expensive, high-powered computer chips, running efficiently on standard computer processors with a memory footprint of just 1.1 gigabytes. This means the protection could be deployed in everyday office environments without requiring specialized hardware.
To understand how this works, imagine the AI agent as a highly skilled but easily distracted employee who has been given a master key to the entire building. If a stranger whispers a fake instruction into the employee's ear, the employee might use that master key to open a door they shouldn't. The Mastyf Guard system removes the master key from the employee's pocket entirely. Instead, the employee must ask a security guard for permission every time they want to open a door. The guard does not listen to the employee's story or try to interpret their tone; the guard simply checks a strict list of which doors the employee is allowed to open at that moment. If the request is not on the list, the door remains locked. This approach solves the problem of the "confused deputy" by ensuring that even if the AI is tricked into asking for the wrong thing, the system simply cannot do it because it lacks the authority.
The researchers also addressed a more subtle problem: what happens when the AI is allowed to use a tool, but tries to change the details of the request to cause harm? For example, an AI might be authorized to send an email, but an attacker could trick it into sending that email to a different address or with a different subject line. To handle this, the team added a second layer of defense that acts like a careful proofreader. This layer checks the specific details of the request against the original permission. If the AI tries to send money to a new account or access a file it wasn't supposed to touch, the system catches the change. In tests involving three thousand complex business scenarios, this combination of strict permission checks and detailed proofreading caught 97.5 percent of attacks while maintaining a false positive rate of zero percent, with a statistical upper bound of 0.19% across 2,000 benign enterprise operations. The system was so precise that in two thousand normal business operations, it did not raise a single false alarm.
One of the most significant findings of the study is that the system does not need to be a super-intelligent AI to be effective. The researchers found that a small, specialized computer program could handle the vast majority of security checks in microseconds, only calling on a larger, more complex AI model when a request was ambiguous or suspicious. This hybrid approach keeps the system fast and cheap to run. In routine business traffic, the complex AI model was never needed, allowing the system to operate at the speed of the fast, simple checks. When the system did need to call on the larger model to investigate a tricky request, it took about eighteen milliseconds, which is still fast enough for most real-time applications. This design ensures that security does not come at the cost of speed or usability, allowing businesses to use powerful AI agents without slowing down their daily work.
The study also explored how to prevent data from leaking out of the system through a chain of authorized tools. Even if an attacker cannot force the AI to use a forbidden tool, they might try to move sensitive information from one allowed tool to another, effectively smuggling data out. Under standard capability checks alone, the researchers found that attackers could successfully exfiltrate data in a significant portion of test cases by moving information between authorized tools. To stop this, the researchers implemented a tracking system that follows the flow of information, similar to how a bank tracks the movement of cash to ensure it doesn't disappear. This system marks sensitive data, such as passwords or financial records, and ensures that this data can only move to destinations that are explicitly cleared to receive it. In tests involving over a thousand attempts to steal data using this specific tracking method, the system blocked every single attempt, ensuring that sensitive information could not move to an unauthorized location.
While the system showed remarkable success, the researchers were careful to note its boundaries. The security guarantees rely on the assumption that the security monitor itself is running on a trusted computer and has not been hacked. If the monitor is compromised, the entire system fails. Additionally, while the system is excellent at stopping attacks that try to use tools in unauthorized ways, it is not perfect at catching every possible variation of a trick, particularly those that involve complex, multi-step maneuvers designed to confuse the system. The researchers identified a small number of cases where their system was bypassed, mostly involving very specific financial tricks or domain name disguises, and they have already proposed updates to fix these gaps. They describe their work as a pre-production architecture, meaning it is ready for testing in real-world environments but still requires further validation by independent experts before it can be considered a final, commercial product.
The implications of this work extend beyond just making AI safer; it changes how we think about building intelligent systems. By treating the AI as a potentially untrustworthy component that must be constantly supervised, rather than a reliable partner that can be trusted to police itself, the researchers have created a framework that is much harder to break. This shift in perspective allows for the use of powerful, autonomous agents in sensitive environments like hospitals, banks, and government offices, where the cost of a mistake is high. The system proves that high security does not require sacrificing speed or requiring expensive hardware, making it a practical solution for the future of artificial intelligence. As these agents become more common in our daily lives, the ability to separate the thinking from the acting, and to enforce strict rules on what actions are allowed, will be essential for keeping our digital world secure.
The researchers made their tools and data available to the public, allowing other scientists to test and improve upon their work. They released the code for the security monitor and the trained AI models, inviting the global community to find weaknesses and suggest improvements. This openness is a critical part of the scientific process, ensuring that the system is not just a theoretical idea but a practical tool that can be scrutinized and refined. The study concludes that by combining strict, deterministic rules with smart, flexible checks, it is possible to build a secure environment for autonomous agents. The path forward involves continuing to test these systems in real-world scenarios, refining the rules to catch new types of tricks, and ensuring that the technology remains robust as the threats evolve. The work represents a significant step toward making the promise of autonomous AI a safe and reliable reality for everyone.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.