🤖 AI

Observability for Delegated Execution in Agentic AI Systems

This paper addresses the inability of standard audit logs to uniquely identify delegation scopes in dynamic LLM-based agentic systems by proposing an agent-aware observability substrate that binds delegation context at execution time to enable reliable, heuristic-free forensic reconstruction.

Original authors: Abhinav Mishra, Kumar Sharad

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abhinav Mishra, Kumar Sharad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a team of highly skilled, autonomous assistants (AI agents) to handle a complex project for you. You give them a broad goal: "Organize our sales data and share the summary with the team."

In the past, if you hired a human employee, you could easily track their day: they opened a file, sent an email, and printed a report. But these AI agents are different. They don't just follow a straight line. They might:

Split the job into smaller tasks and send them to other AI sub-assistants.
Try a tool, fail, and try a different one.
Run several tasks at the exact same time on different computers.
Get confused by a tricky document and accidentally do something they weren't supposed to, all while still acting under your original permission.

The Problem: The "Ghost" of Delegation
The authors of this paper argue that our current way of tracking these agents is broken. We have "audit logs" (like a security camera) and "traces" (like a map of the path taken).

However, the paper claims that you cannot figure out which actions belonged to your specific delegation just by looking at these logs.

Here is an analogy: Imagine two different delivery drivers (Delegation A and Delegation B) are working in the same city. They both use the same roads (tools) and sometimes pass each other.

The Old Way: We only have a log of every car that passed a specific intersection and the time it happened.
The Problem: If Driver A and Driver B both pass the intersection at 2:00 PM, and their paths look similar, we can't tell which driver did what just by looking at the traffic log. We might guess they were together because they were close in time, but that's just a guess. If they overlap, retry, or split up, the guess becomes impossible.

The paper proves mathematically that without a specific label attached to the permission itself, it is impossible to reconstruct exactly what your delegated authority did. The "authority" (who gave the order) is completely separate from the "causality" (the order in which things happened).

The Solution: The "Smart Gateway" and the "Universal Badge"
To fix this, the authors propose a new system called CIM (Common Information Model) and a Gateway.

The Gateway (The Security Guard): Imagine a security guard standing at the door of every tool the AI wants to use (like a file server, a chat app, or a database).
The Universal Badge (The Delegation ID): When the AI starts a job, the system issues a unique, unchangeable "Badge ID" for that specific delegation.
The Rule: Every time the AI touches anything, the Gateway checks for the Badge. If the AI is acting under your permission, the Gateway stamps that action with your Badge ID before it happens.

This is different from just adding a "Session ID" (like a ticket number for a single trip). A Session ID breaks if the AI splits the job into sub-teams or retries a task. The Badge ID stays with the authority, even if the AI spawns a sub-agent, runs tasks in parallel, or goes back and forth.

What This Allows Us to Do
With this new system, asking questions becomes easy and accurate, like looking up a specific customer in a database rather than guessing based on who was standing near the door.

The "Blast Radius" Question: Instead of guessing, "What files did the AI touch between 2:00 and 2:30?" (which might include other people's work), you can ask: "Show me every single file touched by Badge #123, regardless of which tool was used or when."
The "Sub-Agent" Question: If the main AI hired a sub-agent to do the dirty work, the Badge ensures that the sub-agent's actions are still clearly linked to your original delegation.
The "Drift" Question: If the AI started reading safe files but then suddenly started reading secret files, the system can spot this "drift" in the Badge's activity log, even if the AI didn't ask for permission to change.

Important Limitations (What the Paper Does NOT Claim)

It doesn't read minds: The system doesn't know if the AI intended to do something or if it was tricked (like by a "prompt injection"). It only records that the action happened under your delegation. If the AI was tricked into stealing a file, the system says, "Your delegation did this," not "The AI was hacked."
It needs a door: The system only works if the AI goes through the "Gateway." If the AI finds a backdoor or uses a tool the Gateway isn't watching, those actions are invisible.
It's not a prevention tool: It doesn't stop the AI from doing bad things; it just makes sure we can accurately see what happened afterward.

In Summary
The paper argues that as AI agents get more chaotic and complex, our old way of tracking them (looking at time and paths) fails. We need a new way of tracking that follows the permission itself, not just the path. By attaching a permanent "Badge" to every action at the moment it happens, we can finally reconstruct exactly what our delegated AI did, even when it's running in circles, splitting into teams, or working across many different systems.

Technical Summary: Observability for Delegated Execution in Agentic AI Systems

Problem Statement

The paper identifies a critical gap in the observability of Large Language Model (LLM) based agentic systems: delegation-scoped execution is not identifiable from standard observables (audit logs and execution traces).

In traditional systems, execution follows a prescribed workflow. In agentic systems, execution is delegated: a principal issues a durable authorization context to an agent, which then dynamically selects tools, varies execution sequences, backtracks, and spawns sub-agents. This creates a structural mismatch:

Orthogonality of Authority and Causality: The unit of accountability (the delegation) is structurally orthogonal to the unit of execution (the causal trace/span). A single delegation may span causally disconnected tool calls (parallel retries, asynchronous fanout), while causally connected actions may occur under distinct delegations (re-delegation).
Non-Identifiability: The paper proves that delegation membership cannot be reconstructed from execution traces and event-local attributes alone. Multiple incompatible delegation assignments can produce identical observable telemetry (audit logs and causal graphs).
Limitations of Current Tools: Existing mechanisms (SIEM, OpenTelemetry, LangSmith) record individual actions or causal chains but lack the semantics to reconstruct "what actions occurred under a given delegation" across heterogeneous systems. They rely on heuristic time-window correlations or session IDs, which fail under multi-agent composition, retries, and overlapping delegations.

Methodology: The Agent-Aware Common Information Model (CIM)

To address this, the authors propose CIM, a lightweight, agent-aware observability substrate. CIM is not an intent inference engine but a structural model that binds delegation context at execution time.

Core Components

CIM introduces a dual-structure model for delegated execution:

Execution Graph ( $G_{exec}$ ): Represents causal relationships (traces, spans, parent-child links).
Authority Graph ( $G_{auth}$ ): Represents delegation relationships (principals, agents, re-delegation, lineage).
Authorization Relation ( $R_{auth}$ ): Explicitly binds events to delegations.

Semantic Requirements

CIM enforces four semantic commitments to make delegation-scoped reconstruction well-defined:

R1 (Persistent Delegation Context): Every event is bound to a durable delegation_id.
R2 (Authority–Causality Separation): Authority partitions need not coincide with traces; the model explicitly supports divergence.
R3 (Delegation Closure): Re-delegation and agent spawning are represented with lineage closure, allowing queries over a delegation and all its descendants.
R4 (Cross-System Normalization): Actions and resources are normalized into a stable vocabulary (e.g., read, write, share, invoke) to enable cross-tool queries without payload inspection.

Implementation: The Gateway

The authors propose a lightweight gateway as the capture boundary. This component (proxy, sidecar, or library) intercepts agent tool invocations to:

Mint and Bind: Associate events with authenticated principals and issue delegation_ids.
Propagate: Ensure delegation context travels across multi-agent chains.
Normalize: Map tool-specific operations to the canonical action vocabulary.
Enforce Invariants: Ensure delegation_id is not a free-form string but a validated binding, and explicitly mark events as "unbound" if delegation context is missing.

Key Contributions

The paper makes three primary contributions:

Formalization of Delegation-Observability: Defining a system property where the set of actions under a delegation can be reconstructed without heuristic time-window correlation, even under concurrency and multi-agent composition.
Proof of Non-Identifiability: Demonstrating that audit logs and causal tracing alone are insufficient to recover delegation-scoped execution because delegated authority is orthogonal to causal control flow.
Introduction of CIM: A Common Information Model that models delegated execution as dual structures (authority and execution graphs), enabling reliable post-hoc reconstruction and direct forensic queries.

Experimental Results

The authors evaluated CIM using two environments: a high-volume synthetic generator (100k agents, 10k users) and a micro-deployment with real tool invocations.

Reconstruction Accuracy

Ambiguity: Under baseline methods (trace-only, time-window, or hybrid correlation), ambiguity (the number of valid delegation groupings) increased drastically with concurrency. For example, in multi-agent workflows, baseline ambiguity reached a median of 10,856. CIM achieved an ambiguity of 1 for covered events by construction.
Recall: Baseline methods suffered from low recall (p05 as low as 0.03) when trace boundaries did not align with delegation boundaries. CIM achieved 100% recall for covered events.
Divergence: The experiments confirmed that authority-causality divergence is prevalent (e.g., single delegations spanning 35+ causal components in some workflows), validating the need for a model that treats authority and causality as independent.

Operational Overhead

Event Size: CIM adds approximately 247 bytes per event (delegation context, lineage, normalized predicates).
Latency: Gateway processing adds negligible latency (median 0.01 ms, p95 0.29 ms).
Query Effort: The most significant benefit is the reduction in query construction burden. Baseline queries require 6–14 operations (joins, windowing, normalization) per class. CIM reduces this to 1–3 direct predicates over stable fields. Query runtime performance was comparable between CIM and baselines at the tested scale.

Significance and Claims

The paper claims that agentic systems require delegation-aware observability because delegated authority cannot be reconstructed reliably after the fact from traces alone.

Structural Accountability: CIM enables the reconstruction of workflows as structured objects rather than flat event streams, allowing for direct answers to forensic questions like "What is the blast radius of delegation $d$ ?" or "Which sub-agents acted under this authority?"
Agnostic to Intent: The system explicitly separates attribution from intent. CIM attributes actions to the delegation under which they executed, regardless of whether the action was intended, a result of prompt injection, or an error. This makes the "access/share footprint" auditable without needing to infer the agent's internal reasoning or detect prompt injection.
Complementarity: CIM is presented as infrastructure that complements existing security and auditing efforts. It does not prevent misuse or infer intent but provides the necessary structural data to investigate composite outcomes, emergent behaviors, and execution drift in multi-agent systems.

The authors conclude that without such a substrate, the "delegation-scoped reconstruction" problem remains structurally underdetermined in agentic AI, leading to unreliable forensic analysis as these systems scale.