💻 computer science

Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computing

This paper provides a comprehensive security analysis of NVIDIA's GPU Confidential Computing by reconstructing its proprietary architecture, examining its specialized engines and bootstrap process, and experimentally validating data protection across CPU-GPU transfers, with all findings responsibly disclosed to NVIDIA.

Original authors: Zhongshu Gu, Enriquillo Valdez, Salman Ahmed, Julian James Stephen, Michael Le, Hani Jamjoom, Shixuan Zhao, Zhiqiang Lin

Published 2026-04-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhongshu Gu, Enriquillo Valdez, Salman Ahmed, Julian James Stephen, Michael Le, Hani Jamjoom, Shixuan Zhao, Zhiqiang Lin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Glass House" Problem

Imagine you have a very valuable, secret recipe (your AI data) that you want to bake in a professional kitchen (a cloud server). The problem is, the kitchen belongs to someone else. You don't trust the chef, the owner, or even the security guards. You worry they might peek at your recipe, steal it, or swap your ingredients while you aren't looking.

Confidential Computing is like building a magical, invisible, soundproof glass box inside that kitchen. You put your recipe inside, lock the box, and bake your cake. Even the kitchen owner can't see inside the box or touch the ingredients while they are being used.

NVIDIA GPU-CC is the specific technology that builds this glass box for AI tasks running on powerful graphics cards (GPUs). This paper is like a team of security detectives who were hired to peek behind the scenes of this "magic box" to see how it really works, because NVIDIA keeps the blueprints secret.

The Three-Step Investigation

The researchers broke their investigation down into three parts, which they named Blueprint, Bootstrap, and Bridge.

1. The Blueprint (The Map of the Fortress)

Before you can break into a fortress, you need to know where the walls, guards, and secret passages are.

The Challenge: NVIDIA doesn't give out the official map. The "guards" (specialized computer chips inside the GPU) are hidden behind closed doors.
The Detective Work: The researchers acted like spies. They watched how the software talked to the hardware to figure out who was doing what.
The Findings: They discovered the "Guard Dogs" of the system:
- FSP (Foundation Security Processor): The first guard that wakes up and checks the ID of everyone else.
- GSP (GPU System Processor): The manager that handles the keys and the daily operations.
- SEC2 (Secure Processor): The specialized vault keeper that handles the most sensitive data and verifies identities.
- CE (Copy Engine): The delivery trucks that move data around.

2. The Bootstrap (Waking Up the System)

This is the process of turning the system on and making sure it's trustworthy before you let any data in.

The Chain of Trust: Imagine a relay race where the baton is a "trust certificate."
1. A hardware chip (CEC) checks the first guard (FSP).
2. FSP checks the manager (GSP).
3. GSP checks the vault keeper (SEC2).
- If anyone in this chain is lying or tampered with, the whole system shuts down.
The Firewall: Once the system is awake, it locks the doors. In the past, the kitchen owner could walk right up to the oven and change the temperature. Now, a "Firewall" (called a BAR0 Decoupler) blocks the owner from touching the oven controls. They can only see a blank wall (zeros) instead of the controls.
The ID Check (Attestation): Before you start baking, the system takes a photo of its own "fingerprint" (a digital certificate) and sends it to a remote verifier. This proves, "I am a genuine NVIDIA GPU, and I haven't been hacked."

3. The Bridge (The Dangerous Crossing)

This is the most critical part. Your secret recipe (data) has to travel from your private room (the CPU) to the glass box (the GPU). The hallway between them (the PCIe cable) is untrusted. The kitchen owner walks down this hallway all the time.

The Goal: The data must be encrypted (scrambled) while it travels down the hallway so the owner can't read it, even if they are standing right next to it.
The Findings (The Leaks): The researchers found that while the main data is well-protected, there are some small cracks in the armor:
- The Metadata Leak: Imagine sending a package. The contents are locked in a safe, but the shipping label (which says "Package #45: 500lbs of Gold") is left in plain text. Attackers can't see the gold, but they know how much gold is moving and when. This is called a "timing side-channel."
- The "Put" Pointer: In the hallway, there's a sign that says "Next Package Here." The researchers found this sign is visible to the owner. It doesn't reveal the secret, but it reveals the pattern of activity.
- The Cleanup Crew: When the GPU is done baking, it needs to scrub the counter clean so no one can find crumbs of the secret recipe. The researchers found the "scrubbing instructions" were sent in plain text. An attacker couldn't stop the cleaning, but they could see when it was happening.

The Verdict: Is it Safe?

Yes, mostly.
The system is incredibly strong. The actual secret data (the recipe) is encrypted and safe. The "glass box" works.

But, it's not perfect.
Because the system is so complex and built on top of old technology, some "metadata" (information about the data, like size and timing) leaks out.

Analogy: It's like a bank vault where the money is perfectly safe, but the security guard's logbook is left on the counter. A thief can't steal the money, but they can see when the bank is busiest and how much cash is being moved. A smart thief could use that info to guess what the bank is doing.

The Conclusion

The researchers didn't find a way to steal the data, but they found ways to "peek" at the system's behavior. They responsibly told NVIDIA about these small leaks.

The takeaway for the future:
NVIDIA is building a fortress, but they are building it over an old castle. To make it truly secure, they need to encrypt not just the contents of the packages, but also the shipping labels and the timing of the deliveries. They also need to fix the "scrubbing" process so the cleanup crew doesn't leave a trail.

This paper is a vital step in making sure that when we trust our AI secrets to the cloud, they stay truly secret.

1. Problem Statement

NVIDIA introduced GPU Confidential Computing (GPU-CC) starting with the Hopper architecture (e.g., H100) to extend Trusted Execution Environments (TEEs) from CPUs to GPUs. This allows sensitive AI workloads (like Large Language Models) to run securely in untrusted cloud environments without modifying application code.

However, the adoption of GPU-CC faces a critical "black box" problem:

Lack of Transparency: NVIDIA provides only high-level overviews; technical details regarding the specialized hardware engines (FSP, GSP, SEC2) and their interactions are undocumented.
Proprietary Ecosystem: Most components (firmware, CUDA runtime, user-mode drivers) are closed-source or binary-only, preventing independent security verification.
Complexity: GPU-CC overlays new security mechanisms on heterogeneous, legacy components, creating a complex attack surface involving multiple data paths (RPC, DMA, UVM, etc.) across the untrusted PCIe interface.

The Core Question: Does the current implementation of NVIDIA GPU-CC actually guarantee the confidentiality and integrity of data as it traverses the untrusted bridge between the trusted CPU (CVM) and the GPU?

2. Methodology

The authors reverse-engineered the GPU-CC system on an NVIDIA H100 SXM5 platform (AMD EPYC host with SEV-SNP enabled) using a three-stage analytical framework: Blueprint, Bootstrap, and Bridge.

Instrumentation & Interception:
- Open Source Components: Instrumented the open-source NVIDIA kernel-mode driver and Unified Virtual Memory (UVM) driver to trace control flows and intercept function calls.
- Closed Source Components: Preloaded modified libraries to intercept the CUDA runtime and user-mode driver.
- Hardware Interaction: Monitored communications between software stacks and hardware engines (FSP, GSP, SEC2, Copy Engines).
Data Correlation: Cross-referenced experimental observations with fragmented public information (white papers, patents, developer forums) to reconstruct the system's static blueprint.
Quantitative Analysis:
- Firewall Testing: Scanned the GPU's Base Address Register (BAR0) space (16MB) to measure register accessibility before and after enabling GPU-CC.
- Timing Analysis: Logged execution times for memory transfers to detect timing side channels.
- Attestation Verification: Extracted and validated certificate chains and attestation reports using nvTrust.
Targeted Experiments: Triggered specific events (memory faults, scrubbing, RPC calls) to observe how data is protected (or leaked) across different paths.

3. Key Contributions & Findings

A. Blueprint: Architectural Engines

The paper reconstructs the roles of four key hardware engines:

Foundation Security Processor (FSP): A RISC-V microcontroller acting as the hardware Root of Trust. It performs secure boot and initializes the GSP.
GPU System Processor (GSP): The GPU's control plane (RISC-V). It handles key negotiation (SPDM), resource management, and acts as a firewall. It possesses AES hardware for encryption/decryption.
Secure Processor (SEC2): A RISC-V microcontroller dedicated to GPU-CC. It creates the Compute Protected Region (CPR), handles device attestation, and manages secure data transmission (decrypting/verifying data). Note: SEC2 can decrypt but cannot encrypt.
Copy Engine (CE): Manages memory transfers. It has AES hardware to encrypt/decrypt data moving between the CPR and unprotected memory.

B. Bootstrap: Security Initialization

The authors analyzed the initialization sequence:

Chain of Trust: CEC EROT $\rightarrow$ FSP $\rightarrow$ GSP $\rightarrow$ SEC2.
Key Negotiation: The NVIDIA kernel driver and GSP establish an SPDM session to derive a master secret. From this, 32 distinct session keys are derived for different channels (RPC, DMA, Faults, Scrubbing, UVM, CUDA).
Firewall (BAR0 Decoupler): Enabling GPU-CC blocks ~99.78% of BAR0 register access, hiding control registers. However, 0.02% of registers remain accessible, raising concerns about potential information leakage or manipulation.
Attestation: The system uses a certificate chain anchored by NVIDIA's CA. The GPU generates an attestation report signed by an Attestation Key (AK) derived from a fused Device Identity Key (DIK).

C. Bridge: Data Transmission Security Analysis

The authors evaluated six data paths for Confidentiality (C) and Integrity (I). Key findings include:

Data Path	Status	Key Findings & Vulnerabilities
CPU-GSP RPC	Vulnerable	Confidentiality/Integrity Leak: Only the command payload is encrypted. Metadata (headers, pointers, queue status) remains in plaintext in shared staging buffers. Attackers can infer invocation status or manipulate execution order by modifying `readPtr`/`writePtr`.
CPU-GSP Memory Transfers	Side-Channel	Timing Leak: Large transfers (4KB) take significantly longer than small ones (8-256 bytes). This creates a timing side channel allowing attackers to infer data transfer sizes.
GPU Memory Faults	Low Risk	Minor Leak: Shadow buffer PUT pointers are exposed via BAR0. Since faults are rare, the risk is low, but the pointer is observable.
Unified Virtual Memory (UVM)	Partial Risk	Integrity Risk: SEC2 command queues (GPFIFO, GPPUT) in staging buffers are unsigned/unencrypted. While SEC2 verifies method signatures (preventing execution of tampered methods), the metadata leakage could allow redirection of execution.
Memory Scrubbing	Vulnerable	Integrity/Confidentiality Leak: Scrub pushbuffers are signed but not encrypted. Tracking semaphores are unprotected. Attackers can observe scrub activity or tamper with completion signals.
CUDA	Unknown	Opaque: Due to closed-source CUDA runtime, exact mechanisms are speculative. However, the authors infer that critical metadata (QMD, arguments) likely lacks full encryption in staging buffers, relying on SEC2/CE for protection.

4. Results & Implications

Partial Security: GPU-CC successfully encrypts bulk data and code transfers, protecting against direct snooping of payloads.
Residual Leaks: The system fails to protect metadata (queue headers, pointers, status) and timing behavior.
- Confidentiality: Attackers can infer the type of operation, data size, and execution state by observing unencrypted metadata and timing variations.
- Integrity: Attackers can manipulate metadata (e.g., queue pointers) to alter the order or repetition of operations, potentially causing denial of service or logic errors, even if they cannot read the payload.
Attack Surface Expansion: The complexity of managing multiple engines and keys introduces new vectors, particularly in the interaction between the host driver and the GPU's internal processors.

5. Significance

First Deep Dive: This is the first comprehensive security analysis of NVIDIA's commercial GPU-CC implementation, moving beyond high-level marketing claims to a granular, evidence-based assessment.
Responsible Disclosure: All findings were responsibly disclosed to NVIDIA PSIRT.
Guidance for Future Design: The paper highlights that "encrypting the payload" is insufficient for a TEE. Future secure systems must also encrypt metadata, implement constant-time execution to prevent timing attacks, and ensure end-to-end integrity for all control signals.
Impact on AI Security: As AI models become more valuable and sensitive, understanding the true security boundaries of GPU-CC is critical for deploying federated learning and collaborative AI in untrusted clouds. The findings suggest that while GPU-CC is a significant step forward, it is not yet a "silver bullet" and requires further hardening of metadata protection.

6. Conclusion

The paper concludes that while NVIDIA GPU-CC establishes a unified trust boundary for AI pipelines, its current implementation relies on legacy software stacks that do not fully meet the rigorous security requirements of a TEE. The "Bridge" between CPU and GPU remains vulnerable to metadata leakage and timing side channels. The authors advocate for a shift toward encrypting all control plane data and standardizing Trusted I/O (TIO) to eliminate the need for untrusted staging buffers.