AgenTEE: Confidential LLM Agent Execution on Edge Devices

AgenTEE is a system built on Arm Confidential Compute Architecture (CCA) that enables secure, confidential execution of Large Language Model agents on edge devices by isolating the runtime, inference engine, and third-party applications within independently attested confidential virtual machines, achieving near-native performance with less than 5.15% overhead.

The Gist

Imagine you have a very smart, personal assistant living inside your phone. This isn't just a simple calendar app; it's an AI Agent. It can read your emails, plan your vacation, check your bank account, and even write code for you. To do this, it needs to talk to your phone's operating system, the internet, and various third-party apps.

The problem? This assistant is a bit of a "glass house." Because it's so connected, it has a huge "attack surface." If a hacker gets in, they could:

1. Steal your secrets: Like your bank passwords or private emails.
2. Steal the assistant's brain: The company that built the assistant spent millions training it. They don't want their "secret sauce" (the code and the model weights) stolen.
3. Trick the assistant: A hacker could whisper a secret command to the AI, making it do things it wasn't supposed to do, like deleting your files.

Currently, most of these assistants live in the cloud (on big servers far away). But people are starting to want them to live right on their phones (the "edge") for better privacy and speed. The catch? Running them on a phone is risky because phones are vulnerable to software attacks, and the phone's own operating system (like Android) might be untrustworthy or compromised.

The Solution: AgenTEE (The "Secure Glass Box")

The authors of this paper, AgenTEE, propose a new way to run these assistants safely on your phone. They use a special hardware feature in modern phones called Arm Confidential Compute Architecture (CCA).

Here is how they explain it using a simple analogy:

The Analogy: The Secure Bank Vault vs. The Open Office

Imagine a traditional office building (your phone).

• The Old Way: The AI assistant, the model (its brain), and the third-party apps (like a travel booking tool) all sit in the same open room. The building manager (the Operating System) can walk in, read everyone's mail, and change their notes. If a thief breaks into the building, they can steal everything.
• The AgenTEE Way: The authors build three separate, high-security glass vaults inside the building.
  1. Vault A: Holds the Agent (the manager).
  2. Vault B: Holds the Model (the brain).
  3. Vault C: Holds the Third-Party Apps (the tools).

These vaults are made of a special material (hardware) that the building manager cannot see through or touch. Even if the building manager is a spy or the building is on fire, the contents of the vaults remain safe and secret.

How They Talk to Each Other

You might wonder: "If they are in separate vaults, how do they work together?"
The authors created a secure, invisible tunnel between the vaults.

• The Agent in Vault A can send a request to the Model in Vault B.
• The Model processes it and sends the answer back.
• The Agent then asks the App in Vault C to book a flight.
• Crucially: The building manager (the phone's OS) cannot peek into these tunnels. It can only see that something is being passed, but it cannot read the message or change it.

The "Notary" System

To make sure no one is pretending to be the real Agent or the real Model, the system uses a digital notary (called "Remote Attestation").

• Before the Agent lets the Model into the vault, the hardware checks the Model's ID card.
• It verifies: "Is this really the official model from the company? Has anyone tampered with it?"
• Only if the ID is 100% valid does the door open. This ensures that even if a hacker tries to swap the Model with a fake one, the system will reject it.

What They Found (The Results)

The researchers built a prototype of this system using a special development board (since the final commercial phones with this specific hardware aren't out yet). They tested it with two types of AI agents:

1. A simple Chatbot (just answering questions).
2. A complex Travel Planner (doing math, checking schedules, and booking trips).

The Verdict:
They found that putting everything in these secure vaults was almost as fast as running it normally.

• The "secure" version was only about 5% slower than the standard, insecure version.
• This is a huge win because it means you get top-tier security without your phone feeling sluggish or the battery draining instantly.

Summary

AgenTEE is like building a fortress inside your phone. It allows different parts of a smart AI (the brain, the manager, and the tools) to work together securely, even if they don't trust each other, and even if the phone's own operating system is compromised. It proves that we can have powerful, private AI on our devices without sacrificing speed or security.

Technical Summary

Technical Summary: AgenTEE – Confidential LLM Agent Execution on Edge Devices

Problem Statement
Large Language Model (LLM) agents introduce a significantly broader attack surface than traditional applications due to their tight integration with non-deterministic models and third-party services. While edge-device deployments offer improved privacy by keeping user data local, they face critical security challenges. Current edge deployments rely on software-only isolation mechanisms (e.g., multi-processing, seccomp), which are insufficient for protecting proprietary assets (such as system prompts, agent logic, and model weights) and sensitive runtime state (such as Key-Value caches) from privileged software, including a compromised host OS or hypervisor. Furthermore, scenarios involving mutually distrustful stakeholders (e.g., separate providers for the model, the agent code, and third-party tools) require an execution environment that can host these components while preventing them from inspecting or tampering with each other's data.

Methodology: The AgenTEE Framework
To address these challenges, the authors present AgenTEE, a system designed to deploy confidential agent pipelines on edge devices using Arm Confidential Compute Architecture (CCA).

• Architecture: AgenTEE partitions the agent pipeline into three distinct components: the agent runtime, the LLM inference engine, and third-party applications. Each component is deployed into an independently attested Confidential Virtual Machine (cVM), referred to as a "realm."
• Hardware Isolation: Leveraging Arm CCA, these realms execute in hardware-isolated memory (the "realm-world"), which is inaccessible to the normal-world OS, hypervisor, and other privileged software. The Realm Management Monitor (RMM) mediates resource access, significantly reducing the Trusted Computing Base (TCB) compared to traditional TEEs like TrustZone.
• Secure Communication: To enable interaction between these isolated components without exposing data to the host, AgenTEE utilizes Confidential Shared Memory (CSM) via the CAEC design. This allows peer realms to exchange data through memory regions inaccessible to the normal world.
• Attestation and Provisioning: Upon launch, each realm generates a cryptographically signed attestation token (signed by the RMM). Stakeholders (model providers, agent providers, app providers) verify the identity and integrity of the realm before transmitting proprietary assets (e.g., model weights, agent code, API keys) into the protected memory.
• Implementation: The system was implemented using OpenCCA, an open-source prototype of Arm CCA running on Radxa Rock 5B hardware. The software stack includes Trusted Firmware-A, a reference RMM, Linux-CCA as the host/guest OS, and kvmtool-cca as the VMM. A lightweight Python module was developed to abstract CSM usage for user-space components, partitioning shared memory into logical half-duplex channels.

Key Contributions

1. Requirement Analysis: The paper identifies the specific security requirements for edge-device LLM agents, emphasizing the need to protect system prompts, agent logic, model weights, and the KV cache from both the host platform and co-resident applications.
2. System Design: The authors designed AgenTEE, a framework that executes agent runtimes and third-party applications within mutually attested, hardware-isolated cVMs, enabling secure composition of distrustful components on a single device.
3. Implementation and Evaluation: The system was implemented and evaluated on Arm CCA hardware prototypes. The evaluation demonstrates that confidential edge-device LLM agents are practical and scalable, incurring minimal performance overhead.

Results
The authors evaluated AgenTEE using two representative agents (a Chatbot and an Itinerary Planner) and two models (GPT2-Medium and Llama-3.2-1B) across three isolation scenarios:

1. AgenTEE: cVMs communicating via CSM (untrusted OS/Hypervisor).
2. NW VM: VMs communicating via shared memory (trusted Hypervisor).
3. NW Process: Processes communicating via shared memory (trusted OS/Hypervisor).

The results indicate that AgenTEE achieves near-native performance:

• Compared to standard OS multi-process deployments (the weakest isolation baseline), AgenTEE incurred a runtime overhead of less than 5.15%.
• Compared to NW VM deployments, the overhead was less than 2.53% across all tested configurations.
• The overhead remained consistent regardless of the model size or agent complexity, suggesting the isolation mechanism does not significantly bottleneck the inference or communication pipeline.

Significance
The paper argues that secure agentic systems on edge devices handling proprietary assets fundamentally require hardware-backed confidential execution. By confining sensitive assets and runtime states to hardware-protected memory, AgenTEE enables the secure composition of mutually distrustful components on a single device. The authors conclude that this approach provides strong isolation guarantees with negligible performance penalties, making it a viable solution for deploying complex, privacy-preserving LLM agents on modern edge hardware. The work highlights that Arm CCA is a suitable foundation for general-purpose confidential computing, moving beyond the limitations of fixed-function TEEs like TrustZone.