Governance at the Boundary: How Agent Decomposition Degrades Policy Compliance
This paper introduces Fiducia-bench to demonstrate that decomposing AI agents into multiple components significantly degrades policy compliance by attenuating critical facts at handoff boundaries, leading to both under- and over-escalation risks that vary by model capability and architecture.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern financial world, banks and institutions rely on sophisticated computer programs to check the identities of new customers and flag suspicious activity. These programs, often called "agents," are designed to follow strict rules: if a customer looks like a known criminal, the agent must freeze the account and report it; if a customer is clearly innocent, the agent must let them proceed without delay. For a long time, engineers built these agents as single, self-contained units that handled every step of a task from start to finish. However, as these systems have grown more complex, developers have begun to break them apart into smaller, specialized pieces. One piece might look up a name, another might check a database, and a third might make the final decision. This approach, known as decomposition, is meant to make the system more organized and efficient, much like a large factory where different workers handle specific parts of an assembly line rather than one person building the entire product alone.
The critical question that researchers recently asked is whether this division of labor helps or hurts the system's ability to follow the rules. When a task is split between different components, information must be passed from one to the next. If a piece of crucial evidence is lost or forgotten during this handoff, the agent might fail to stop a criminal or might wrongly accuse an innocent person. A team of researchers set out to measure exactly how much this splitting of tasks damages the agent's ability to stay within the law. They did not just ask if the computer finished the job; they asked if it finished the job correctly, according to the strict policies that govern financial safety.
To find the answer, the researchers created a testing ground called Fiducia-bench, a collection of one hundred different scenarios based on real-world banking rules. These scenarios involved checking customer identities and looking for signs of money laundering. They designed the tests to see what happened when an agent had to cross a "boundary"—a point where one part of the system handed a task over to another. In some tests, the agent worked alone in a single loop. In others, the work was split into a fixed pipeline of three stages, or managed by a central coordinator who assigned tasks to smaller sub-agents. The researchers used two different computer models to run these tests: a powerful open-source model and a stronger, proprietary model. They watched closely to see if the agents discovered important facts, such as a mismatch in a customer's date of birth or a link to a politically exposed person, and whether those facts survived the journey to the component that had to make the final decision.
The results revealed a clear and troubling pattern. When the agent worked as a single unit, it never lost a single piece of discovered information. However, as soon as the work was split into separate components, the system began to drop critical facts. In the tests using the open-source model, when the work was divided into a three-stage pipeline, the system lost more than half of the important facts it had found. When the work was managed by a central coordinator assigning tasks to sub-agents, the system lost even more, dropping eighty-five percent of the facts it had discovered. This means that in the vast majority of cases where the agent found a clue, that clue vanished before it could reach the part of the system responsible for acting on it. The stronger computer model performed much better, losing only a tiny fraction of the facts, but it still lost more when the work was divided than when it worked alone.
The most dangerous aspect of this failure is that it works in both directions, causing two opposite types of mistakes. If the lost fact was a warning sign, the agent would fail to stop a risk, letting a dangerous customer proceed. If the lost fact was proof of innocence, the agent would wrongly flag a safe customer as a threat. The researchers found that the same mechanism—simply forgetting a detail during the handoff—could cause the system to be too careless or too paranoid. For example, in one scenario, the system dropped a fact that proved a customer was innocent, leading it to freeze the account unnecessarily. In another, it dropped a fact that proved a customer was risky, leading it to let them walk away. The component that made the final mistake was not the one that failed to find the clue; the failure happened in the silence between the two parts.
This study suggests that the way we build these systems matters as much as the intelligence of the computer model itself. The act of breaking a task into smaller pieces introduces a specific point of failure where information can leak away. While using a more capable computer model can reduce the amount of information lost, it does not eliminate the problem entirely. The researchers concluded that the current trend of building complex, multi-part agent systems carries a hidden cost: it makes the system less reliable at following the rules that keep it safe. The solution, they imply, lies in designing better ways for these different parts to talk to each other, ensuring that no vital detail is left behind when the baton is passed. Until then, the more we divide the work, the more likely we are to lose the truth.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.