🤖 AI

A Policy Algebra for Trust-Preserving Agentic AI Execution

This paper introduces a policy algebra that defines a trust-preserving reliability envelope for agentic AI by formally composing security constraints and runtime obligations, enabling the system to intervene on nearly all policy violations while maintaining high task completion rates and ensuring audit completeness.

Original authors: Bhaskar Tripathi, Anurag Kumar, Ramendra Kumar, Bhavesh Gadhe

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bhaskar Tripathi, Anurag Kumar, Ramendra Kumar, Bhavesh Gadhe

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of artificial intelligence, a significant shift is occurring. For years, the focus was on building systems that could generate text, answer questions, or summarize documents. These systems were like very knowledgeable librarians who could read anything but were forbidden from touching the books or moving the furniture. Today, a new generation of software, often called "agents," has been given the keys to the building. These agents do not just talk; they act. They can search through private company files, send emails, update databases, call external services, and even hand off parts of a job to other software programs. This ability to move from thinking to doing makes them incredibly powerful for businesses, but it also introduces a new kind of danger. A mistake is no longer just a wrong sentence; it is a deleted file, a leaked secret, or a budget spent on a task that never finished.

The core problem is that an agent might succeed in its goal while breaking the rules to get there. Imagine a worker who successfully delivers a package but does so by breaking into a locked room they were not allowed to enter. In the past, security systems mostly checked if a person had a badge to enter a building. But for these intelligent agents, the security check needs to happen at every single step of the journey, not just at the front door. If an agent is allowed to make a decision, that decision must be checked against who is asking, what tools they are using, how much money they have left to spend, and whether a human has approved the action. Without this constant supervision, a powerful tool can easily cause harm, even if the person who started the process had good intentions.

Researchers at Volkswagen Digital Services and Scania CV AB have developed a new way to manage this risk, which they call a policy algebra. Think of this not as a list of rules, but as a mathematical system for combining different security requirements into a single, unbreakable decision. The researchers realized that to trust an agent, you cannot just look at the final answer it produces. You must look at the entire path it took to get there. Their system treats every action an agent wants to take as a transition that must pass a series of gates. These gates check the agent's identity, its role, the sensitivity of the data it is touching, the tools it is using, its remaining budget, and whether a human has signed off on the move. If any single gate fails, the action is stopped.

The brilliance of this approach lies in how it handles complexity, especially when one agent asks another agent to do a task. In many current systems, when a task is passed down, the security rules can accidentally get weaker, allowing a sub-agent to do things the original agent was not allowed to do. The new system prevents this by ensuring that authority never expands as it moves down the chain. Instead, the security profile becomes stricter or stays the same. If a high-level manager asks a junior employee to help, the junior employee cannot suddenly gain access to the manager's private files. The system also manages money carefully. It ensures that if an agent spends a lot of computing power or money, it must produce a tangible result, like a saved document or a completed record. If an agent runs out of budget without leaving a useful output, the system stops it, preventing the company from wasting resources on a task that went nowhere.

To test how well this works, the researchers ran thousands of simulated scenarios involving agents trying to complete various tasks. They compared their new system against older methods that relied on simple lists of allowed tools and user roles. The results showed a clear difference in safety. The new system stopped 94.8% of the unsafe actions that the older system missed. This included preventing agents from accessing data they shouldn't see, using tools they weren't allowed to touch, or passing tasks to other agents in a way that bypassed security rules. The system also eliminated a specific type of failure where an agent would spend all its money but leave no useful result behind. In terms of tracking, the new system recorded 98.6% of the necessary details for an audit, compared to 71.9% for the older method, making it much easier to understand what happened if something went wrong.

There is a cost to this extra safety. The new system is slightly slower to make decisions, taking about 71 milliseconds on average compared to 38 milliseconds for the older method. It also occasionally stops a safe action by mistake about 4.2% of the time, compared to 2.1% for the older system. Because of these extra checks, the overall rate at which agents successfully finished their tasks dropped slightly, from 90.7% to 86.9%. However, the researchers argue that this trade-off is necessary. The goal is not to build the fastest or most capable agent, but the most reliable one. A task that is completed quickly but involves unauthorized data access or wasted money is not a success.

The study confirms that security for intelligent agents must be a property of the entire journey, not just the destination. By using this new method of combining rules, companies can allow their software to act with more freedom while ensuring that every step remains authorized, traceable, and economically sound. The system does not try to guess what an agent might do wrong; instead, it creates a framework where an agent can only do what is explicitly permitted at every moment. This approach turns the raw power of artificial intelligence into a tool that enterprises can actually trust to operate in the real world, where mistakes have real consequences.

Technical Summary: A Policy Algebra for Trust-Preserving Agentic AI Execution

1. Problem Formulation

The paper addresses a critical gap in enterprise Agentic AI: the distinction between capability (the ability to reach a goal) and reliable capability (reaching a goal through an admissible, governed path). While Large Language Model (LLM) agents excel at reasoning, tool use, and delegation, their transition from reasoning to action introduces new failure modes. An agent may produce a correct final answer via unauthorized data access, widened delegated authority, unapproved side effects, unrecoverable budget consumption, or incomplete audit trails.

Existing frameworks (e.g., OWASP AISVS, NIST AI RMF) and testbeds (e.g., AgentDojo) identify risks and provide threat models but lack a compositional, executable runtime semantics to enforce these controls dynamically. Current benchmarks often prioritize task success over enterprise authorization, auditability, or profile preservation. The core problem is that agent security is a path property: a final answer is only reliable if the entire sequence of reasoning-to-action transitions satisfies governing invariants regarding identity, profile, data, memory, tools, budget, artifacts, approval, and audit.

2. Methodology: The Policy Algebra

The authors propose a policy algebra that defines a "reliability envelope" within which agent capability is exercised. This algebra treats security profiles as ordered policy objects and composes them to ensure trust preservation and least-restrictive soundness.

Core Concepts

Reliable Capability: An agent is reliably capable of a task $\zeta$ only if there exists a trace $\xi$ where the goal is reached and every action event $\eta_t$ satisfies a conjunction of predicates: Identity, Role, Profile, Data, Memory, Tool, Budget, Artifact, Human-in-the-Loop (HITL), and Audit.
Policy State ( $\varpi$ ): A tuple comprising the security profile, executable tool set, memory/data scope, remaining budget, artifact-materialization policy, approval state, and evidence obligations.
Composition ( $\otimes$ ): The algebra defines how policy states combine. When two states interact (e.g., during delegation or tool invocation), the composition operator $\otimes$ $\otimes$ ensures:
- Profiles: The join ( $\sqcup$ ) of profiles, selecting the more restrictive (higher assurance) profile.
- Authorities: The intersection ( $\cap$ ) of tool sets and memory scopes, narrowing authority.
- Budgets: The minimum ( $\min$ ) of remaining budgets.
- Obligations: The union ( $\cup$ ) of evidence requirements and logical OR ( $\lor$ ) of approval requirements.
- Theorems: The composition is proven to be trust-preserving (never reducing assurance or expanding authority) and least-restrictive (satisfying all inputs without imposing unnecessary restrictions).

Key Mechanisms

Profile Cascade: The effective profile for any action is the join of platform, team, user, agent, and publication profiles. No lower layer can loosen constraints inherited from higher layers.
Trust-Preserving Delegation: When an agent delegates to a sub-agent, the child context inherits the parent's restrictions. The delegated profile is the join of caller and callee profiles; tool sets and memory scopes are intersected; budgets are narrowed. This prevents "profile laundering" where a high-security call is delegated to a low-security agent to bypass constraints.
Cost-Aware Artifact Materialization: To prevent "zero-outcome budget exhaustion" (spending the full budget without a recoverable result), the runtime enforces a materialization predicate. As cost exposure grows, the admissible action set is redirected toward producing a durable artifact (draft, checkpoint, file) before the budget is fully consumed.
Conditional Correctness: The paper distinguishes between the policy decision (conditional on correct context) and the accuracy of the context itself (e.g., misclassified documents). The algebra guarantees correctness only if the input metadata is accurate.

System Architecture

The runtime implements the algebra through a layered architecture:

Framework Layer: Reusable agent blueprints.
Runtime Execution Layer: The system of record enforcing the policy gate.
Application Layer: User interfaces and service consumers.
Enforcement Flow: Before any action (tool call, memory write, delegation), the runtime compiles heterogeneous inputs (RBAC, threat models, metadata) into a monotone policy state. It evaluates a conjunction of predicates. If any fail, the action is denied, escalated to HITL, or redirected to materialization.

3. Key Contributions

Definition of Reliable Capability: A formal distinction between reaching a goal and reaching it via an admissible path, framing agent autonomy as utility maximization within a reliability envelope.
Policy Algebra: A compositional framework for enterprise profiles, RBAC, tools, memory, publication, budget, approval, and evidence. It guarantees that composition is trust-preserving and the least restrictive state satisfying all inputs.
Runtime Semantics:
- Non-amplifying Delegation: Mechanisms (profile joins, authority intersections) that prevent authority widening across call chains.
- Cost-Aware Materialization: Logic that redirects execution toward recoverable outputs as budget exposure increases.
Design-Science Path: A methodology for mapping security artifacts (threat models, standards) to executable predicates, a reference runtime, and a conformance evaluation protocol.

4. Evaluation and Results

The evaluation focuses on conformance to the proposed control taxonomy rather than benchmark ranking. The runtime was tested against an RBAC-and-allowlist comparator using a workload of 612 traces (1,936 events) covering prompt injection, tool misuse, RAG governance, multi-agent delegation, and budget scenarios.

Key Metrics:

Unsafe Intervention Rate: The policy algebra intercepted 94.8% of policy-violating events, compared to 67.1% for the RBAC comparator.
Trust Violations: The algebra eliminated 100% of observed profile-monotonicity violations (31 cases in the comparator) and 100% of zero-artifact budget exhaustion cases (18 cases in the comparator).
Audit Completeness: Increased from 71.9% to 98.6%, ensuring traces contain sufficient data for review and repair.
Task Completion: The governed runtime retained 86.9% of task completion, a 3.8 percentage point reduction from the comparator's 90.7%.
Overhead: Median authorization latency increased from 38 ms to 71 ms; false intervention rate increased from 2.1% to 4.2%.

Residual Failures: The remaining misses (e.g., 79.1% intervention rate for sandbox escapes) were attributed to context construction errors (e.g., ambiguous tool intent, misclassified documents) rather than failures of the algebraic logic itself.

5. Significance and Claims

The paper claims to provide a formal, executable method for converting security standards and threat models into runtime behavior without sacrificing useful agent capability.

Shift in Perspective: It moves the security focus from "output safety" (is the final answer safe?) to "action-path reliability" (is every transition safe?).
Compositional Trust: It demonstrates that multi-agent trust can be preserved through algebraic operators (joins and intersections) that prevent authority amplification during delegation.
Economic Reliability: It introduces a mechanism to treat budget exhaustion without a durable artifact as a policy failure, ensuring economic recoverability in unattended workflows.
Repair Loop: The framework supports a closed-loop process where failed evaluation cases are converted into policy updates and regression tests, allowing the system to evolve.

The authors modestly state that this is an implementation-oriented formalization that connects standards aggregation, enterprise governance, and multi-agent trust propagation. It does not claim to replace existing standards or defenses but to make them executable. The results are presented as measured trade-offs on a specific workload, demonstrating that a "reliability envelope" can be enforced with a quantifiable, manageable cost to raw capability. The paper concludes that the path to "agents that can act reliably" lies in expanding the reliability envelope through better context perception and policy calibration, rather than weakening the invariants of the policy algebra itself.