virtCCA: Virtualized Arm Confidential Compute Architecture with TrustZone

This paper presents virtCCA, a virtualized implementation of Arm's Confidential Compute Architecture using TrustZone on existing hardware that enables the deployment of confidential virtual machines with strong API compatibility and acceptable performance overhead, bridging the gap until dedicated CCA hardware becomes widely available.

The Gist

Imagine you are running a massive, high-security bank in the cloud. You want to store your customers' most sensitive data (like passwords and bank balances) in a "vault" that not even the bank manager (the cloud provider) can peek into.

In the tech world, this is called Confidential Computing. Arm, the company behind the chips in most smartphones, recently invented a new, super-secure vault system called CCA (Confidential Compute Architecture). It's like a brand-new, state-of-the-art bank vault that is incredibly secure.

The Problem:
The problem is that this new "CCA vault" hardware won't be available in real servers for a few years. But companies need this security now. They have thousands of older servers (like AWS Graviton or Google's Tau machines) that are powerful but lack this new vault technology. They can't just throw away their old servers and wait years for the new ones.

The Solution: virtCCA
The authors of this paper built virtCCA. Think of it as a high-tech "force field" or a "magic suit" that you can put on your old servers to make them act like they have the new CCA vaults.

Here is how it works, using simple analogies:

1. The Two Worlds: The Living Room vs. The Panic Room

Arm chips already have a feature called TrustZone. Imagine the chip is a house with two rooms:

• The Normal World (Living Room): This is where the regular operating system (like Linux) lives. It's where the "bank manager" (the hypervisor) works. It's powerful but not perfectly secure.
• The Secure World (Panic Room): This is a tiny, locked room inside the chip. Only very trusted code can enter.

The Challenge:
The new CCA system wants to run entire "Virtual Machines" (mini-computers inside the computer) inside this Panic Room. But the Panic Room was originally designed only for tiny, simple apps, not full-blown operating systems. Also, the "bank manager" in the Living Room is considered untrusted; we don't want him to be able to peek into the Panic Room.

2. The "Gatekeeper" (TMM)

To solve this, the authors created a tiny, super-trusted software layer called the TrustZone Management Monitor (TMM).

• Analogy: Imagine the TMM is a super-strict bouncer standing at the door between the Living Room and the Panic Room.
• How it works: The "bank manager" (the hypervisor) in the Living Room still does the heavy lifting: he decides when to turn a machine on or off, and he allocates memory. But he cannot touch the data inside the Panic Room.
• If the manager needs to do something for a machine in the Panic Room, he has to ask the Bouncer (TMM). The Bouncer checks the ID, verifies the request, and then lets it happen. This ensures the manager can never steal the data.

3. The "Magic Suit" for Old Servers (S-EL2 vs. No S-EL2)

The paper is special because it works on two types of old servers:

• Type A (Newer Old Servers): These have a feature called S-EL2. This is like having a special key that lets the Bouncer work inside the Panic Room efficiently. The Bouncer can manage the room directly.
• Type B (Very Old Servers): These don't have the special key. The Bouncer has to stand outside the room (at a higher security level called EL3) and use a walkie-talkie to talk to the inside. It's a bit more work, but it still works!
• Why this matters: Most other solutions only work on Type A. virtCCA works on both, meaning almost any existing Arm server can become a secure vault.

4. The "Translation" Problem (Interrupts and Devices)

When a machine in the Panic Room needs to talk to a printer or a network card, it usually just "shouts" (interrupts). But in the old servers, the hardware doesn't know how to shout from inside the Panic Room.

• The Fix: The Bouncer (TMM) acts as a translator. When the machine inside tries to shout, the Bouncer catches it, translates the message into a format the outside world understands, and then relays it. To the machine inside, it feels like it's talking directly to the hardware, even though it's actually talking through the Bouncer.

The Results: Is it fast?

You might think adding a Bouncer and a translation layer would make things slow.

• The Test: The authors tested this on real servers with heavy workloads like databases (Redis, MongoDB) and web servers (Apache).
• The Verdict: It's surprisingly fast!
  • On the newer servers (with the special key), the slowdown was less than 30% even for heavy tasks.
  • On the very old servers, it was actually faster than the standard setup in many cases because the "Bouncer" was so efficient at managing the memory.

Summary

virtCCA is like a time-traveling security upgrade. It takes old, powerful servers that lack the newest security hardware and wraps them in a "magic suit" that makes them just as secure as the future hardware. It allows companies to protect their data today without waiting years for new chips to arrive, all while keeping the software compatible with the future standards.

Technical Summary

Here is a detailed technical summary of the paper "virtCCA: Virtualizing Arm CCA with TrustZone".

1. Problem Statement

Arm's Confidential Compute Architecture (CCA), introduced in the Armv9-A architecture, enables Confidential Virtual Machines (CVMs) by isolating them in a "Realm world." While CCA promises a robust future for confidential computing, widespread commercial hardware support is years away. Consequently, a significant number of existing Arm servers (e.g., AWS Graviton, Google Tau T2A) and future non-CCA hardware lack the necessary features to run CVMs.

Existing solutions face two main limitations:

1. Hardware Dependency: Most CVM solutions (like Twinvisor) rely on Secure EL2 (S-EL2), a feature introduced in Armv8.4 that is not yet widely available in commercial hardware.
2. Trust Model & Compatibility: Existing hypervisor-based TEEs (e.g., SeKVM, pKVM) often require a trusted core within the hypervisor or lack full compatibility with the CCA software stack, making it difficult to run unmodified guest OSes and applications.

There is an urgent need for a solution that brings CCA capabilities to legacy hardware (both with and without S-EL2) while maintaining strong isolation, high performance, and API-level compatibility with the official CCA specifications.

2. Methodology: The virtCCA Architecture

The authors propose virtCCA, a virtualized CCA design that leverages the mature TrustZone hardware feature (present in most Arm platforms) to host CVMs in the Secure World, treating the normal-world hypervisor as untrusted.

Core Design Principles

• TrustZone as the Hosting Environment: Unlike hypervisor-based TEEs that run CVMs in the normal world, virtCCA runs CVMs in the Secure World. The hypervisor (KVM) in the normal world manages the lifecycle of CVMs but is denied access to their memory and state.
• The TrustZone Management Monitor (TMM): A small, trusted software layer (analogous to the Realm Management Monitor in CCA) acts as a gatekeeper between the untrusted normal world and the CVMs. It enforces isolation and manages state transitions.
• Dual Implementation Strategy:
  • virtCCA-SEL2 (for Armv8.4+): The TMM runs at S-EL2. It utilizes hardware support for stage-2 page tables to isolate CVMs and manage memory virtualization.
  • virtCCA-EL3 (for Legacy Hardware): The TMM runs at EL3 (part of the Arm Trusted Firmware). Since EL3 lacks native stage-2 virtualization for the secure world, the TMM fully emulates virtualization features (interrupts, MMIO, page tables) in software. It restricts CVMs from modifying page tables by marking them read-only and intercepts all memory access attempts.

Key Technical Mechanisms

1. Memory Isolation:

   • Cross-World: Uses the TZASC (TrustZone Address Space Controller) to physically separate Secure and Normal memory regions.
   • Cross-CVM:
     • In SEL2, the TMM manages stage-2 page tables to prevent CVMs from accessing each other's memory.
     • In EL3, the TMM enforces isolation by making CVM page tables read-only and intercepting any modification attempts via SMC (Secure Monitor Call).
   • Management: The TMM implements a custom Buddy and Slab memory allocator within the secure world. It reserves a contiguous block of memory at boot, avoiding the need for dynamic memory migration (unlike Twinvisor), which reduces TLB flushes and context switches. It also optimizes for NUMA affinity.

2. Interrupt and Device Virtualization:

   • Interrupts: The TMM translates interrupts between worlds. In EL3, where hardware list registers are unavailable, the TMM emulates interrupt injection by saving/restoring vCPU states and manually injecting virtual IRQs (vIRQs) via software.
   • MMIO & Devices: MMIO accesses are trapped. In EL3, the guest OS is modified to replace MMIO instructions with SMCs that trap to the TMM. The TMM then forwards requests to the hypervisor for emulation (e.g., virtio devices).

3. Compatibility:

   • API Level: virtCCA implements TrustZone Management Interfaces (TMIs) and TrustZone Service Interfaces (TSIs) that are functionally identical to the CCA's Realm Management Interface (RMI) and Realm Service Interface (RSI).
   • Software Stack: The entire stack (ATF, RMM, KVM, Guest OS) is built to be compatible. Existing CCA software can run on virtCCA with minimal changes.

3. Key Contributions

1. First S-EL2 Independent CVM Solution: virtCCA is the first system capable of running secure KVM guests in the TrustZone secure world on platforms without S-EL2 support, utilizing EL3 and software emulation.
2. Full Software Stack Implementation: The authors developed a complete stack including the TMM, modified KVM (KVM-virtCCA), QEMU, and a modified Guest OS (Linux) supporting remote attestation and secure memory management.
3. Strong CCA Compatibility: The design ensures that the software stack developed for future CCA hardware can be reused on virtCCA today, as the interfaces are API-compatible.
4. Efficient Memory Management: By managing secure memory entirely within the TMM using a buddy/slab allocator and avoiding dynamic memory migration, the system minimizes overhead compared to previous designs like Twinvisor.

4. Evaluation Results

The system was evaluated on real Arm servers:

• Hardware:
  • virtCCA-SEL2: Arm V8.4 server (64 cores, S-EL2 support).
  • virtCCA-EL3: Kunpeng 920 server (No S-EL2 support).
• Micro-benchmarks:
  • Context Switches: virtCCA introduces overhead due to world switches. IPI (Inter-Processor Interrupt) latency is roughly 2x that of vanilla KVM.
  • I/O: Overhead is higher due to MMIO trapping and emulation.
• Macro-benchmarks (Real-world Workloads):
  • CPU-bound: VirtCCA-SEL2 showed ~1.8% overhead; VirtCCA-EL3 showed ~0.8% overhead.
  • I/O-bound (e.g., Apache, Redis):
    • virtCCA-SEL2: Overhead is acceptable, reaching a maximum of 29.7% for I/O-intensive workloads due to frequent world switches.
    • virtCCA-EL3: Surprisingly, outperformed the baseline in many cases (e.g., Hackbench +64.3%, Redis +16.3%). This is attributed to CVMs running in "bare-metal mode" (EL3) without the overhead of CPU/memory virtualization layers present in the normal-world hypervisor.
  • NUMA Optimization: Enabling NUMA affinity in the TMM allocator improved memory bandwidth by 66.7%.

5. Significance

• Bridging the Hardware Gap: virtCCA allows cloud providers to deploy confidential computing workloads on existing legacy Arm infrastructure immediately, rather than waiting for Armv9-A CCA hardware.
• Software Ecosystem Continuity: By maintaining strict API compatibility with CCA, virtCCA ensures that the massive investment in CCA software development (guest kernels, RMM, toolchains) is not wasted and can be deployed today.
• Security Model: It shifts the trust boundary from the hypervisor to the hardware-enforced Secure World (TrustZone), providing a robust isolation model even when the host OS is compromised.
• Performance Viability: The results demonstrate that virtualizing CCA on legacy hardware is not just theoretically possible but practically viable, with overheads that are acceptable for most production workloads and, in some cases, superior to standard virtualization.