Technical Summary: Agent Operating Systems (AOS)

Problem Statement

Traditional operating systems (OS) are architected around deterministic programs with explicit control flow, bounded execution, and human-initiated workflows. Their core abstractions—processes, threads, system calls, and file permissions—assume predictable interaction patterns. However, Agentic AI systems introduce a fundamentally different execution model: long-lived, goal-directed entities that reason probabilistically, invoke tools dynamically, and adapt behavior based on feedback.

Treating agents merely as user-space applications creates critical gaps in production environments:

• Scheduling Misalignment: Agent progress depends on external events, tool latencies, and budget constraints, not just CPU instruction progress.
• Memory Characterization: Agent state extends beyond heap memory to include contextual windows, retrieved knowledge, and provenance metadata with distinct lifetimes and trust requirements.
• Security Incompleteness: Agents require scoped permissions for dynamic tool calls with side effects, which static permission models cannot adequately govern.
• Observability Deficits: Operators lack decision lineage (the "why" and "how" of actions) and policy compliance evidence, relying instead on resource metrics.
• Governance Gaps: Safety and policy enforcement must remain deterministic even when the underlying reasoning is probabilistic.

The central systems question posed is: If agents become first-class computational entities, what should the operating system boundary look like to ensure controllability, accountability, and security at scale?

Methodology and Architecture

The paper proposes the Agent Operating System (AOS) not as a replacement for existing kernels, but as a semantic elevation of control that integrates an agentic control plane into or alongside traditional OSs. The methodology involves defining precise boundaries, decomposing responsibilities, and mapping these concepts to existing Linux and Windows primitives.

Core Definition and Boundaries

An AOS is defined as a systems software layer that manages the lifecycle, execution, coordination, and governance of goal-directed agents. It enforces a strict separation between:

1. Reasoning Plane: Probabilistic inference, planning, and strategy selection (treated as untrusted).
2. Execution Plane: Deterministic tool invocation and side effects (enforced in least-privilege environments).
3. Policy Plane: Authorization, risk checks, and budget controls (deterministic enforcement).

The architecture relies on a Trusted Computing Base (TCB) comprising the policy engine, capability validation, mediation layers, and audit subsystems. The TCB ensures that nondeterministic outputs from reasoning systems cannot produce side effects without traversing deterministic enforcement paths.

Key Architectural Components

• Agent Lifecycle Manager: Manages states beyond process states (e.g., awaiting events, suspended) and defines deterministic termination semantics, including token revocation and audit closure.
• Agent Scheduler: Optimizes for goal progress subject to policy, risk, and budget constraints, rather than just CPU fairness. It separates reasoning resources from execution resources to prevent unbounded inference consumption.
• Context and Memory Management: Treats memory as a managed resource with explicit semantics (ephemeral context, durable memory, retrieved knowledge, execution records). It mandates deterministic context construction and provenance tracking for auditability.
• Tool and Capability Registry: Functions as a privileged admission mechanism binding dynamically discovered tools to capability scopes. It enforces a mediation pipeline: schema validation, policy/risk/budget checks, execution in sandboxes, and append-only audit logging.
• Policy and Trust: Implements layered policies (static authorization, capability constraints, context/risk/governance policies) and deterministic trust states (normal, restricted, quarantined).

Integration Models and OS Mapping

The paper evaluates three integration models and maps AOS concepts to existing OS primitives:

1. User-Space Runtime: AOS runs as services/libraries. It offers low-risk adoption and portability but relies on strong network/OS controls to prevent bypass.
2. OS Extension: Introduces OS-level primitives (e.g., agent identity as a principal). This offers stronger enforcement but increases kernel complexity and attack surface.
3. Distributed Control Plane: A cluster-level plane managing identity and policy, delegating execution to node-level sandboxes.

Mapping to Linux:

• Isolation: Uses namespaces, cgroups, seccomp, and Mandatory Access Control (SELinux/AppArmor).
• Scheduling: Leverages cgroups for CPU/memory quotas and separate resource pools for reasoning vs. execution.
• Observability: Utilizes eBPF and auditd to correlate OS events with AOS action IDs.

Mapping to Windows:

• Isolation: Uses Job Objects, Windows Containers, Hyper-V, and AppContainer.
• Security: Leverages restricted access tokens, ACLs, and Security Descriptors for scoped delegation.
• Observability: Correlates Event Tracing for Windows (ETW) with agent decision lineage.

Key Contributions

The paper provides the following specific contributions:

1. Precise Definition: A formal definition of AOS with explicit boundaries separating probabilistic reasoning from deterministic enforcement.
2. Responsibility Decomposition: A structured breakdown of AOS duties into lifecycle, scheduling, memory, tool mediation, policy, and observability.
3. Integration Models: A comparative analysis of user-space, kernel-extension, and distributed control plane models.
4. Pragmatic Mapping: Concrete implementation paths for Linux and Windows using existing primitives (cgroups, namespaces, tokens, eBPF, ETW).
5. Security Framework: Agent-specific threat models (prompt injection, tool misuse, privilege escalation) and deterministic mitigation principles (deny-by-default, capability scoping, mediation).
6. Evaluation Criteria: Metrics focused on deterministic enforcement correctness, audit completeness, containment, and operational comprehensibility.

Results and Claims

The paper does not present experimental benchmarks or a deployed system. Instead, it presents a systems architecture proposal and a theoretical framework. Its primary "result" is the demonstration that:

• Classical OS abstractions are incomplete for agent workloads, specifically regarding goal progress, decision lineage, and dynamic capability management.
• AOS can be realized without rewriting kernels by elevating control to a deterministic plane that mediates between agents and OS primitives.
• Deterministic enforcement is achievable even with probabilistic reasoning by strictly separating the reasoning plane from the execution plane and enforcing "deny-by-default" policies with revocable capabilities.

Significance and Scope

The paper claims significance as a foundational step toward making autonomy deterministic at the boundary. It argues that the research challenge is not making agents autonomous, but ensuring their autonomy is controllable, accountable, and secure.

• Modesty of Claims: The authors explicitly state that AOS does not replace operating systems wholesale, nor does it claim to solve Artificial General Intelligence (AGI). It does not require the kernel to run model inference.
• Scope: The work focuses on establishing a rigorous systems foundation for agentic computation. It acknowledges open research problems, including agent scheduling theory for goal progress, formal verification of enforcement pipelines, and multi-agent delegation chains.
• Context: This work was conducted as part of the Unified Intelligent Infrastructure workstream at the Open Compute Project (OCP), aiming to provide a standardized approach for running agents reliably in production environments.