When to Delegate: A Measurement Study of Bounded Agent Teams for Personal Super Agents
This study introduces the bounded-delegation protocol and Eva-Bench to demonstrate that while delegating tasks to agent teams can marginally improve success rates in specific scenarios, it generally incurs significant latency and token costs without providing a consistent advantage over monolithic approaches, suggesting delegation should be selectively earned based on request structure rather than applied universally.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a personal assistant that does more than just answer questions. It is a persistent software organization that acts on your behalf, reading your car's status, planning a trip, booking a service, remembering your preferences, and answering follow-up questions minutes later. To make this work, engineers often build a team of specialized digital workers. One worker acts as a manager, or router, who listens to your request and delegates parts of the job to other workers who handle specific tasks like navigation or booking a restaurant. The idea is that a team of specialists should be better than a single generalist. However, just as a human team requires meetings and handoffs that take time, a team of software agents must communicate, pass information, and verify each other's work. This coordination costs time and computing power. The critical question for the future of personal assistants is whether the extra accuracy gained by using a team is worth the cost of that coordination, or if a single, powerful agent is often the smarter choice.
A researcher at the City University of Hong Kong and Geely Automobile Group set out to answer this question with a rigorous measurement study. The team built a real-world personal agent named Eva, designed for use inside a vehicle. Eva has access to twenty-two different tools, allowing it to control the car's climate, plan routes, book maintenance, manage reminders, and access personal profiles. To test how well different organizational structures work, the researchers created a benchmark called Eva-Bench, consisting of eighty distinct tasks. These tasks ranged from simple requests like setting the temperature to complex scenarios involving multiple steps, references to previous conversations, and requests that the system must safely refuse. They then ran these same eighty tasks through three different versions of the agent: a single, monolithic agent that handled everything alone; a bounded team where a manager assigned work to specialists with strict limits on how long they could work; and an unbounded team where the specialists had more freedom and time.
The results revealed a clear and measurable cost to using a team. On simple, single-step tasks, the team version was significantly slower and used far more computing resources than the single agent. Specifically, the team took nearly three times as long to complete a task and used more than two and a half times the amount of digital tokens, which are the units of computing work. In one full run of the eighty tasks, the team succeeded slightly more often, completing 92.5 percent of the tasks compared to 91.2 percent for the single agent. However, when the researchers repeated the entire experiment three times to check for consistency, the advantage disappeared. The single agent actually performed slightly better in the repeated trials, completing 93.3 percent of tasks versus 92.9 percent for the team. This suggests that the small advantage seen in the first run was likely due to random chance rather than a fundamental superiority of the team structure.
The study found that the team only offered a genuine benefit in specific situations. The team outperformed the single agent on complex service workflows, such as booking a driver and a restaurant, and on tasks that required remembering details from earlier in the conversation. In these cases, the team's structure helped prevent the agent from dropping parts of the request or losing track of the context. However, for other types of tasks, such as retrieving knowledge or managing safety refusals, the single agent was often just as good or even better. The researchers also discovered that giving the team members more freedom did not help. The unbounded team, which allowed specialists to work longer, did not succeed more often than the strictly limited team; it simply took longer and used more resources. This indicates that tight limits on how long a worker can operate are actually beneficial for keeping the system fast and reliable.
The researchers also tested specific safety mechanisms designed to prevent the team from making false claims, such as a ledger that tracks actions to ensure they are not repeated, and guards that check if a claimed action actually happened. In this specific study, these mechanisms did not improve the final success rate of the tasks. While they did not cause harm, they also did not provide a measurable boost in performance for the types of tasks tested. The most promising finding was a hybrid approach. The researchers created a policy that could decide, based on the nature of the request, whether to use the single agent or the team. By sending complex service requests to the team and simple requests to the single agent, they achieved a success rate of 95 percent while keeping the response time much closer to the fast single agent than to the slow team.
The study concludes that delegation is not a free upgrade. Building a team of agents introduces a coordination tax that slows down the system and increases costs. For a personal assistant, this tax is only worth paying when the task is complex enough to benefit from specialized coordination, such as handling multi-step service transactions or maintaining context across a long conversation. For most other requests, a single, well-designed agent is faster, cheaper, and just as effective. The research suggests that the future of personal super agents lies not in adding more agents, but in carefully choosing when to use them, ensuring that the structure of the request justifies the cost of the organization.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.