Technical Summary: EnclaveX – End-to-End Confidential AI with CPU/GPU TEEs

Problem Statement

The rapid proliferation of Large Language Models (LLMs) has driven widespread adoption of AI applications, yet most deployments rely on centralized cloud infrastructures (e.g., Azure, AWS, Google Cloud). This reliance necessitates that users share sensitive data, training code, and fine-tuning parameters with cloud providers, creating significant security and privacy risks. Traditional encryption paradigms protect data at rest and in transit but leave "data in use" exposed to privileged insiders, malicious administrators, and supply-chain vulnerabilities.

While Trusted Execution Environments (TEEs) such as Intel SGX/TDX, AMD SEV-SNP, and ARM CCA have been introduced to mitigate these risks, and NVIDIA has recently developed GPU TEEs (e.g., H100/H200), comprehensive evaluations of end-to-end workflows integrating both CPU and GPU TEEs remain limited. Critical gaps include:

1. Incomplete Trust Boundaries: State-of-the-art approaches often fail to remove the entire Cloud Service Provider (CSP) management stack from the Trusted Computing Base (TCB). Specifically, Kubernetes (K8s) administrators retain full API privileges and can execute commands (e.g., kubectl exec) to enter a Confidential Virtual Machine (CVM), effectively acting as root users within the secure enclave.
2. Lack of Application-Level Protection: Existing solutions often rely on VM-level isolation but lack mechanisms to protect application-level secrets (e.g., encryption keys) from administrators who have compromised the VM OS.
3. Insufficient Performance Data: There is a lack of empirical data quantifying the performance overhead of running LLM inference on confidential GPUs (cGPUs) within CVMs, particularly regarding the integration of process-based TEEs inside CVMs.

Methodology

The paper presents EnclaveX, a comprehensive end-to-end Cloud Confidential Computing platform designed to secure AI/ML workloads by integrating CPU and GPU TEEs with application-level protection.

System Architecture

EnclaveX combines three primary components:

1. Hardware TEEs: Utilizes Intel TDX for CPU-based memory isolation and NVIDIA H200 GPUs for confidential AI acceleration. These hardware extensions establish a root of trust in the vendor hardware rather than the CSP software.
2. SCONE Runtime: A shielded execution framework that enables unmodified applications to run inside TEE enclaves. SCONE provides process-level isolation, encrypting data and code in use, and shielding them from the host OS and hypervisor.
3. Custom Kernel Module: A module deployed within the Guest OS of the CVM that performs dynamic measurements of running ML workloads. This module disables memory-dumping features in the Guest kernel to prevent administrators from extracting application memory.

Workflow and Attestation

The system employs a multi-layered remote attestation strategy based on the Remote Attestation Procedures Architecture (RATS):

1. CVM Attestation: The CVM (Intel TDX/AMD SEV-SNP) attests its boot state and hardware integrity to a Configuration and Attestation Service (CAS).
2. GPU Attestation: The confidential GPU (cGPU) generates an attestation report signed by an Attestation Key (AK) derived from hardware measurements. The CAS verifies the GPU's state, driver, and VBIOS against Reference Integrity Manifests (RIM).
3. Application Attestation: Once the CVM and GPU are verified, the CAS provisions a signing key to the SCONE kernel module. The module measures the running ML application, generates a cryptographic hash, and signs it to produce a quote.
4. Secret Provisioning: Only after successful verification of the CVM, GPU, and application integrity does the CAS release configuration data and secrets (e.g., decryption keys) to the application.

This design ensures that even if a K8s administrator gains access to the CVM, they cannot access the application's encryption keys or dump its memory, as the keys are only released post-attestation and memory dumping is disabled.

Key Contributions

1. End-to-End Confidential AI Workflow: EnclaveX is presented as the first system to integrate process-based TEEs (via SCONE) inside CVMs with confidential GPU support, extending security guarantees across CPU-GPU boundaries.
2. Application-Level Protection Against Admins: The paper proposes and implements mechanisms to protect application secrets from K8s administrators who have access to the CVM, addressing a critical gap in current CVM-only solutions.
3. Comprehensive Performance Evaluation: The authors provide the first confidential LLM-inference benchmarks on an NVIDIA H200 cGPU that include full remote-attestation measurements.
4. Open-Source Kernel Module: The implementation includes a custom kernel module for attestation (the SCONE kernel module), which is designed to be open-sourced to support diverse architectures (Intel TDX, AMD SEV-SNP, ARM CCA) without modifying application code.

Results

The authors evaluated EnclaveX using industry benchmarks on a Supermicro SYS-322GA-NR server equipped with dual Intel Xeon 6900-series processors and an NVIDIA H200 GPU.

LLM Inference Performance

• EnclaveX vs. Native CVM: The integration of the SCONE runtime into the CVM imposes negligible performance overhead. Metrics for Tokens Per Second (TPS), Inter-Token Latency (TBT), and Time-to-First-Token (TTFT) showed no significant difference compared to a native CVM deployment. This is achieved by running the SCONE runtime in "SIM" mode, which relies on CVM memory protection without additional encryption of memory.
• Confidential Compute (CVM + cGPU) vs. Native: Running in confidential mode (CVM + cGPU) introduces overhead compared to non-confidential native execution.
  • TPS Overhead: Ranged from 62.8% to 35.0%, decreasing as batch size increased.
  • TBT Overhead: Ranged from 73.9% to 35.2%, decreasing as batch size increased.
  • TTFT Overhead: Ranged from 38.3% to 11.2%, decreasing as input token size increased.
  • Analysis: The overhead is primarily attributed to the encryption of I/O between the CVM and cGPU via bounce buffers. As compute time increases (larger batches/tokens), the relative impact of I/O overhead diminishes.

Attestation Overhead

• Native Attestation: The baseline latency for TDX and cGPU attestation was approximately 1.16 seconds.
• EnclaveX Attestation: Adding the application-level attestation (SCONE kernel module and runtime) increased the total latency to 1.1758 seconds.
• Overhead: The application-level attestation adds a negligible ~1.27% overhead to the total attestation process.

Significance and Claims

The paper claims that EnclaveX successfully reimagines secure AI/ML deployment by unifying HPC-grade compute with robust security guarantees. Its significance lies in:

• Data Sovereignty: Enabling organizations to run high-value workloads in public clouds while retaining fine-grained control over access and compliance, effectively mitigating risks from privileged insiders.
• Scalability: Providing a framework that supports unmodified ML applications, facilitating the transition from traditional on-premise silos to secure, scalable cloud environments.
• Security Guarantees: By enforcing strong data sovereignty through multi-layered attestation (VM, GPU, and Application), EnclaveX ensures the integrity of code, data, and execution environments before releasing sensitive material.

The authors conclude that while side-channel attacks and denial-of-service attacks are outside the scope of this work, the system effectively addresses the primary threats of data exfiltration and model poisoning by privileged administrators in multi-tenant cloud environments.