Fail-Closed Authorization for Tool-Using AI Agents: A Capability Contract and Case Study
This paper introduces the Agent Security Contract (ASC), a deny-by-default authorization framework for tool-using LLM agents that enforces pre-execution gates via policy-constrained tool allowlists and integrity-checked evidence, demonstrating through static trace evaluations that updated policy versions effectively block adversarial attacks while minimizing false positives compared to earlier iterations.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a digital assistant that can do more than just answer questions; it can actually open files, send emails, and run commands on your computer. This is the promise of modern AI agents, but it brings a dangerous new reality: if an attacker tricks the AI into thinking a harmful command is a helpful suggestion, the AI might obey without hesitation. The core problem is not that the AI is confused, but that it lacks a reliable gatekeeper to check every single action before it happens. In the world of computer security, the safest approach is to assume nothing is allowed until it is explicitly permitted, a principle known as "fail-closed." If the gatekeeper cannot verify a request, it must simply say no, rather than risking a mistake that could let a hacker steal data or delete important systems.
A researcher named Andrey Lesnikov has built a system called the Agent Security Contract to act as this gatekeeper. Instead of trying to guess what the AI is thinking or analyzing the tone of a conversation, this system acts as a strict, pre-execution checkpoint. Before the AI is allowed to touch a tool, the system checks a digital contract that lists exactly which tools are permitted, what arguments they can accept, and which actions require a human to sign off first. The system works by comparing the AI's proposed action against a pinned, unchangeable list of approved tools. If the tool is not on the list, or if the arguments look suspicious, the action is blocked immediately. If the action involves something dangerous, like deleting a file or sending money, the system pauses and demands an out-of-band approval from a human, ensuring that high-stakes decisions are never made by the machine alone.
To test if this approach actually works, Lesnikov did not run a live AI in a real environment, which would be too risky. Instead, he created a collection of recorded scenarios, including both harmless tasks and sophisticated attacks designed to bypass security. He ran these recorded scenarios through his system, version by version, watching how the gatekeeper reacted. In the initial version, the system blocked most obvious attacks but missed some clever tricks, such as hiding malicious commands inside file paths or using special characters to escape security rules. After finding these gaps, the researcher tightened the rules. The updated system learned to recognize and block these specific bypasses, such as preventing an AI from reading sensitive password files or using shell commands to navigate outside its allowed workspace.
The results showed that the system is highly effective at stopping unauthorized actions, but it comes with a trade-off. In a test involving over a hundred recorded tasks, the strict new rules blocked nearly every attack attempt. However, the system also blocked a significant number of harmless tasks because they used arguments that the strict rules did not expect. For example, if a task tried to send an email without attaching a file, and the system was configured to require a file, the task was blocked even though it was safe. The researcher found that simply listing the names of allowed tools was not enough to stop attacks; the system needed to check the details of every argument. Yet, when the rules were adjusted to understand which arguments were optional, the system could distinguish between safe and unsafe actions without blocking the harmless ones.
One of the most important findings was that the system's ability to stop attacks depends entirely on how carefully the rules are written. When the researcher tested the system against a set of real-world attack goals from a different project, the system successfully held back almost all of them, but only because the rules required human approval for any action that changed data or sent messages. The system did not magically understand the difference between a legitimate email and a phishing attempt; it simply enforced the rule that any such action needed a human to say "yes." This highlights a crucial limitation: the system is only as strong as the contract it enforces. If the contract is too loose, the AI can still cause harm. If it is too strict, it stops the AI from doing its job.
The study also measured how much time this security check adds to the process. The system is incredibly fast, taking less than a hundredth of a second to decide whether to allow or block an action. This speed means that the security check does not slow down the AI's work, making it practical for real-world use. However, the researcher noted that the system is not a complete solution. It does not stop an AI from using a permitted tool in a harmful way, such as sending a message to a safe address that contains a virus. It also cannot protect against a hacker who has already taken over the computer running the AI. The system is a specific tool for a specific job: ensuring that every tool call matches a pre-approved plan.
Ultimately, the paper demonstrates that a simple, rigid set of rules can effectively prevent an AI from executing unauthorized commands. The system works by refusing to act when it is unsure, a strategy that prioritizes safety over convenience. While the current version requires careful setup and can sometimes be too cautious, the research shows that with better rules and human oversight, it is possible to build a reliable barrier between an AI and the systems it controls. The work suggests that the future of safe AI agents lies not in smarter AI, but in stricter, simpler, and more transparent gatekeeping.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.