Specifying the Delegated-Autonomy Boundary: Requirements Engineering for Agentic AI
This paper addresses the unique requirements engineering challenges of Agentic AI by introducing the concept of a "delegated-autonomy boundary" and proposing two complementary artifacts—the Agency Justification Record (AJR) and the tiered Agentic Delegation Policy (ADP)—to systematically define, justify, and govern the scope of decisions delegated to autonomous systems.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are the captain of a spaceship. For most of human history, building a ship meant writing a strict instruction manual: "If you see a red light, turn left. If you see a blue light, turn right." The ship was a robot that followed orders perfectly, but it couldn't think for itself. Then, we invented "smart" computers that could learn from data, like a student who reads a library of books and guesses the answer. But even those smart computers were mostly just guessers; they waited for you to ask a question and then gave an answer. They didn't really do anything on their own.
Now, we are entering a new era with "Agentic AI." Think of this not as a guesser, but as a new kind of crew member. This crew member doesn't just wait for orders; it can look at a map, decide to grab a wrench, open a door, talk to another crew member, and fix a leak—all without you telling it every single step. It has a bit of a mind of its own. This is exciting, but it's also a little scary. If you give a crew member the keys to the engine room, how do you know they won't accidentally crash the ship? How do you decide what they are allowed to touch, and when you need to step in and take the wheel? This is the big question: How do we set the rules for a robot that can act on its own?
This paper tackles that exact problem. The authors, Chetan Arora, Andreas Vogelsang, and Abbi Sharma, argue that we are currently building these "acting" robots without writing down the rules of their freedom. Instead of clear rules, we are hiding the instructions inside messy computer code and vague prompts. They propose a new way to design these systems using two special tools: a "Justification Record" and a "Delegation Policy."
First, the Agency Justification Record (AJR). Imagine you are hiring a new intern. Before you give them the keys to the office, you have to ask: "Do we really need an intern, or can we just use a vending machine?" The AJR is a checklist that forces teams to answer this question. It asks: Is the task too messy for a simple script? Does it need to talk to different systems? Can we stop the intern if they make a mistake? The paper suggests that if you can't check all the boxes on this list, you shouldn't build an "agent" at all. You should just use a simpler, safer tool. For example, the authors tested this on a "code review" idea. They found that reviewing code is actually a very structured job with clear rules, so a fancy, thinking robot wasn't needed; a simple script would do the job better and safer. The AJR says, "Don't build a robot if a calculator will do."
But what if you do need a robot? That's where the second tool comes in: the Agentic Delegation Policy (ADP). This is like a "Rulebook of Freedom" for your robot crew member. Instead of just saying "Go fix the ship," the ADP breaks down exactly what the robot can and cannot do, using a tiered system of authority.
Think of the ADP as a traffic light system for the robot's actions:
- Green Light (Autonomous Tier): The robot can do these things alone, like reading a map or checking the temperature. No human needs to watch.
- Yellow Light (Advisory Tier): The robot can suggest things, like "I think we should change the engine oil," but it must wait for a human to say "Yes, go ahead" before actually doing it.
- Red Light (Prohibited Tier): These are the "Do Not Touch" zones. The robot is strictly forbidden from doing things like crashing the ship or deleting important files, no matter what.
The ADP also covers other important rules, like:
- Memory: What can the robot remember? Can it keep secrets about patients or code, or must it forget them after the job is done?
- Teamwork: If the robot gets confused or two robots disagree, who do they call? The ADP says, "If you can't solve it in two tries, stop and ask a human."
- Safety: How do we know the robot is telling the truth? The policy requires the robot to keep a log of everything it does so humans can check later.
The authors tested these ideas with two very different examples. The first was a "Hospital Discharge Agent," a robot designed to help send patients home from the hospital. This job is messy, involves talking to insurance companies and doctors, and carries a high risk if things go wrong. The AJR said, "Yes, this is a perfect job for a robot," and the ADP gave it strict rules: it could read patient charts (Green), but it could never sign a discharge order without a doctor's approval (Red). The second example was the "Code Review Agent" mentioned earlier. The AJR looked at it and said, "Nope, this is too simple and structured. Don't use a robot here; use a script."
The paper doesn't claim to have solved everything. It admits that we still need to figure out how to turn these written rules into actual computer code that the robots can understand, and how to test them properly. But the main idea is clear: we can't just let these powerful robots run wild. We need to stop hiding the rules in the code and start writing them down clearly, deciding exactly how much freedom to give, and when to take it back. By using the AJR to decide if we need a robot, and the ADP to decide how to let it act, we can build a future where these digital crew members are helpful helpers, not dangerous wildcards.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.