💻 computer science

Effective Segregation of Duties in Agentic Enterprise Authorization: Evaluating Shared-Dependence Risk in Maker–Checker Controls

This paper introduces the concept of Effective Segregation of Duties (SoD) to distinguish between nominal identity separation and actual independent assurance in AI-driven authorization, demonstrating through a large-scale P2P benchmark that effective risk mitigation relies on evidence mediation and heterogeneous checker models rather than mere principal distinctness.

Original authors: Mohamed Abbas Elmasry

Published 2026-09-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohamed Abbas Elmasry

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of large organizations, from banks to government agencies, there is a fundamental rule for keeping things safe: no single person should have the power to start a transaction and finish it alone. This principle, known as the separation of duties, ensures that if one person makes a mistake or attempts to act improperly, a second person will catch it before anything harmful happens. For decades, this system relied on two different humans checking each other's work. Today, however, many of these tasks are being handed over to artificial intelligence agents. These are software programs that can read documents, make decisions, and even execute financial transfers. The question facing security experts is no longer just about having two different AI programs, but whether those two programs are truly independent. If both agents rely on the same underlying technology, the same sources of information, or the same way of thinking, they might make the exact same mistake at the same time, rendering the safety check useless.

A recent study set out to test this exact scenario. Researchers created a simulated environment based on real-world procurement rules, where an AI agent acting as a "maker" would propose a purchase, and a second AI agent acting as a "checker" would decide whether to approve it. The goal was to see how often the checker would fail to stop a bad proposal. The team tested different ways of setting up these two agents. In some cases, both agents used the same type of AI model. In others, they used different models. They also tested whether the checker was allowed to see the maker's reasoning or if it had to look at the raw facts directly, and whether the checker had to rely on the information the maker chose to highlight or if it could access the original, unfiltered data.

The results were stark. When the system was set up with the highest level of shared dependence—where both agents used the same AI model, the checker could see the maker's reasoning, and the checker only saw the information the maker had selected—the safety system failed more than sixty percent of the time. In these situations, the second agent simply agreed with the first agent's errors, allowing unsafe transactions to proceed. However, when the researchers changed the setup to reduce these shared dependencies, the results improved dramatically. By using a different AI model for the checker, hiding the maker's reasoning, and giving the checker direct access to the original source data, the failure rate dropped to just over thirteen percent. This means that simply having two different AI agents is not enough; the way they are connected and what information they see matters far more than their identities.

The study also looked at how often the system would get in the way of good work. A safety check that rejects every single proposal, even the safe ones, is useless because it stops business from happening. The researchers found that the improved setups did not cause this problem. The systems that caught more bad proposals did not reject significantly more good ones. This suggests that it is possible to build AI safety checks that are both strict and efficient, provided they are designed to avoid the specific traps that cause them to fail together.

One of the most significant findings was about how information flows between the two agents. The biggest improvement came when the checker was given direct access to the original evidence rather than relying on a summary provided by the maker. When the maker acted as a middleman for the data, the failure rate was much higher. This indicates that the first agent can unintentionally, or intentionally, filter out the very details needed to spot a mistake. By cutting out the middleman and letting the second agent see the raw facts, the system became much more reliable. Similarly, hiding the first agent's written explanation from the second agent helped reduce errors, suggesting that the second agent was sometimes being swayed by the first agent's confidence or logic rather than sticking to the facts.

The researchers also tested what happens when two different types of AI models are paired together. They found that using a different model for the checker significantly reduced the chance that both would fail on the same bad proposal. However, they were careful to note that this result was specific to the pair of models they used. It does not guarantee that any two different AI models will always work well together, but it does prove that mixing models is a powerful strategy for breaking the chain of shared errors. The study also highlighted a practical challenge: one of the different model combinations was more prone to technical glitches that made its answers unreadable. This reminded the researchers that safety is not just about intelligence, but also about reliability. If a safety check cannot produce a clear answer, it cannot do its job.

Ultimately, this work changes how we should think about safety in automated systems. It shows that the old rule of "two different people" does not automatically translate to "two different AI agents." If the two agents are too similar in how they think, what they see, or how they are built, they will likely fail together. True safety requires a deliberate design that forces the second agent to look at the problem from a different angle, using different tools and different information. The study concludes that organizations cannot simply assume their AI safety checks are working because they have two agents. They must measure the actual performance of those checks, ensuring that the second agent is truly independent and capable of catching the mistakes of the first. Only by treating safety as a measurable outcome rather than a structural label can we trust these powerful new tools with our most sensitive decisions.

Technical Summary: Effective Segregation of Duties in Agentic Enterprise Authorization

1. Problem Statement

The paper addresses a critical gap in the application of Segregation of Duties (SoD) to agentic AI systems. While classical SoD relies on structural separation (distinct principals or roles) to prevent unauthorized actions, the author argues that nominal SoD (identity separation) does not guarantee Effective SoD (independent assurance) in AI-driven workflows.

In agentic enterprise systems, two AI agents may occupy distinct roles (Maker and Checker) yet share common sources of error, such as:

Using the same model family (common failure modes).
The Checker being exposed to the Maker's rationale (framing bias).
The Checker relying on evidence selected or summarized by the Maker (information fidelity loss).

The core security question is not whether the second model is accurate on average, but whether it measurably intercepts unsafe proposals that passed the first stage. The paper posits that without empirical evaluation, structural separation may mask common-mode failures, leading to "unsafe-approval escape" where policy-violating actions proceed unchecked.

2. Methodology

The study evaluates Effective SoD using a controlled, policy-grounded benchmark in a Procure-to-Pay (P2P) domain.

Benchmark and Gold Standard:

Domain: Enterprise P2P authorization based on the Mohawk College Procurement Policy CS-1202-2011 (revised 2026).
Gold Decisions: Deterministic rule-engine outputs (PERMIT, DENY, REQUIRE_APPROVAL), not LLM-generated judgments.
Dataset: A frozen corpus of 160 proposals mapped to unique cases, including 80 controlled counterfactual "unsafe" proposals and 80 "safe" proposals. A complementary natural maker-failure substudy utilized 240 cases.

Experimental Design:
A balanced 2×2×2 within-proposal factorial design was employed with three replicates per cell (totaling 3,840 checker executions in the factorial stage). The design manipulated three operational contrasts associated with shared dependence:

Checker Composition: Same-Qwen (Qwen Maker + Qwen Checker) vs. Qwen–Mistral (Qwen Maker + Mistral Checker).
Maker-Rationale Exposure: Visible (Checker sees Maker's reasoning) vs. Blinded.
Evidence Mediation: Maker-mediated (Checker sees Maker's selected/summarized evidence) vs. Direct authoritative access (Checker queries the source directly).

Metrics:

Unsafe Approval Escape Rate (UAER): The probability a checker approves a proposal that the gold standard dictates should be denied or escalated.
Safe Workflow Disruption Rate (SDR): The probability a checker rejects or escalates a proposal that the gold standard dictates should be permitted.
Conditional Co-Failure: In the natural substudy, the rate at which a checker fails to intercept a naturally occurring unsafe Maker submission.

Execution:
The study involved 5,520 model executions using Qwen/Qwen3-8B and Mistral-7B-Instruct-v0.3. Parse failures were retained and analyzed via sensitivity scenarios (fail-closed vs. adversarial) to ensure robustness.

3. Key Contributions

Operationalization of Effective SoD: The paper distinguishes between structural identity separation and measurable assurance, defining Effective SoD as a graded profile $(UAER, SDR)$ rather than a binary architectural label.
Empirical Evidence on Shared Dependence: It provides controlled evidence that nominal separation does not prevent correlated failures. Specifically, it quantifies how evidence mediation, rationale exposure, and model homogeneity increase the risk of unsafe approvals.
Reproducible Policy-Grounded Artifact: The study introduces a benchmark with deterministic gold decisions, frozen maker proposals, and explicit handling of parse errors, allowing for counterfactual analysis of control effectiveness.

4. Key Results

Configuration Performance:

UAER Reduction: The highest-dependence configuration (C1: Same-Qwen, Visible Rationale, Mediated Evidence) resulted in a 63.87% UAER. The lowest-dependence configuration (C7: Qwen–Mistral, Blinded Rationale, Direct Evidence) reduced this to 13.22%, a 50.65 percentage point absolute reduction.
SDR Stability: SDR remained low and stable across configurations (6.81%–11.76%), indicating that lower UAER was not achieved by indiscriminately rejecting all proposals.

Marginal Contrasts (Parsed-Only Primary Estimand):

Evidence Mediation: The largest observed effect. Mediated evidence increased UAER by 34.86 percentage points compared to direct access.
Checker Composition: Same-Qwen checking increased UAER by 8.09 percentage points compared to Qwen–Mistral checking.
Rationale Exposure: Visible rationale increased UAER by 7.90 percentage points compared to blinded rationale.

Natural Maker-Failure Substudy:

Conditional co-failure rates were 52.48% for Same-Qwen checking versus 17.82% for Qwen–Mistral checking on naturally occurring unsafe submissions. This confirms that model diversity (in this specific pairing) significantly reduces correlated failures even on uncontrolled data.

Robustness:

Results remained robust under "fail-closed" and "adversarial" parse-error handling, though the magnitude of the checker-composition contrast attenuated slightly under adversarial assumptions due to higher parse-failure rates in the Qwen–Mistral cells.

5. Significance and Claims

The paper claims that identity separation is necessary but insufficient for secure agentic authorization. The author argues that:

Nominal SoD is not Effective SoD: Structural separation of principals does not guarantee that the second control is outside the failure domain of the first.
Failure-Domain Separation is a Design Principle: To achieve Effective SoD, architects must actively manage failure domains by:
- Providing checkers with direct authoritative access to evidence rather than relying on maker-mediated summaries.
- Blinding checkers to maker rationales to prevent framing bias.
- Validating checker heterogeneity empirically rather than assuming model diversity guarantees independence.
Measurable Assurance: Security in agentic systems must be evaluated through measured interception rates (UAER) and workflow disruption (SDR) against specific policy constraints, rather than inferred from architectural labels.

The study concludes that while distinct principals remain vital for accountability, the "second control" in an AI workflow must be engineered and evaluated as a distinct security mechanism with its own failure profile, rather than treated as a passive verification step.