Technical Summary: GPUBreach

Problem Statement

While Rowhammer attacks on CPU DRAM (DDR) have been extensively studied and shown to enable privilege escalation (e.g., corrupting sudo or page tables to gain kernel access), Rowhammer exploits on GPUs have historically been limited. Previous work, such as GPUHammer, demonstrated that GDDR6 memories on NVIDIA GPUs are susceptible to bit-flips, but these were primarily used to inject untargeted errors into machine learning model weights to degrade accuracy.

The core problem addressed by this paper is whether Rowhammer attacks on NVIDIA GPUs can be elevated from simple data corruption to privilege escalation. Specifically, the authors investigate if an unprivileged user-level CUDA kernel can manipulate GPU page tables to gain arbitrary read/write access to GPU memory (device-side escalation) and subsequently compromise the host CPU kernel (system-wide escalation), even in the presence of protections like IOMMU.

Methodology

The authors developed GPUBreach, a framework that combines Rowhammer bit-flips with novel techniques for "massaging" GPU page tables into vulnerable locations. The attack relies on three primary challenges and their corresponding solutions:

1. Uncovering Page Table Allocation (Challenge C1)

NVIDIA GPUs store page tables entirely in device memory, but their allocation strategy is undocumented.

• Technique: The authors patched the open-source NVIDIA GPU driver to dump memory locations and analyzed the allocation strategy. They discovered that page tables are allocated in contiguous 2MB regions (PT regions). Initially, these are placed far from user data, but once the initial region fills, subsequent regions are allocated from the same physical pool as user pages, allowing for the interspersed allocation required for Rowhammer attacks.

2. Efficient Page Table Filling (Challenge C2)

Filling a 2MB PT region with Page Table Entries (PTEs) using standard 2MB data pages would require ~256GB of memory, exceeding GPU capacity.

• Technique: The authors developed new allocation primitives using Unified Virtual Memory (UVM). By exploiting the driver's behavior with specific allocation sizes (e.g., 2MB + 4KB), they coerced the driver to allocate smaller 4KB and 64KB page frames. This reduced the memory required to fill a PT region from 256GB to as little as 1GB (for sparse filling) or 16GB (for dense filling), making the attack feasible on consumer GPUs.

3. Detecting and Steering Allocations (Challenge C3)

The attacker must know exactly when a new PT region is allocated to place it at a vulnerable memory location (a row susceptible to bit-flips).

• Technique: The authors identified a timing side-channel in UVM allocations. When GPU memory is near capacity, new allocations trigger evictions of pages to CPU memory, causing a measurable spike in access latency. By monitoring these latency spikes, the attacker can detect the creation of a new PT region. By carefully managing memory (evicting specific pages and freeing target slots), the attacker can "massage" the allocator into placing a new PT region precisely at a known vulnerable physical address.

The Attack Workflow

1. Profiling: Identify vulnerable DRAM rows susceptible to bit-flips.
2. Massaging: Fill GPU memory and use the timing side-channel to detect PT region allocations. Evict specific pages to force the next PT region to be allocated at a vulnerable row.
3. Tampering: Use Rowhammer to flip bits in the Page Frame Number (PFN) field of a PTE within the vulnerable region.
4. Escalation:
   • Step 1: Flip a bit so the corrupted PFN points to a page controlled by the attacker.
   • Step 2: Free that attacker-controlled page and massage a second PT region into that location.
   • Result: The attacker now controls the PTEs of the second PT region, granting arbitrary read/write access to the entire GPU physical memory.

Key Contributions

1. First GPU Privilege Escalation via Rowhammer: The paper demonstrates the first Rowhammer-based privilege escalation on NVIDIA GPUs, granting arbitrary read/write access to GPU memory by corrupting page tables.
2. New GPU Page Table Mechanisms: The authors uncover previously unknown GPU page table allocation behaviors and develop efficient techniques to fill and position page tables using UVM primitives.
3. System-Wide Escalation: The paper demonstrates that compromising the GPU can lead to CPU-side privilege escalation. By corrupting trusted, GPU-writable data structures in the privileged GPU driver (specifically the GSP message queue), the attacker can trigger out-of-bounds (OOB) writes in the Linux kernel, bypassing IOMMU protections to gain a root shell.
4. Practical Exploits: The authors validate the attack with three real-world scenarios:
   • Data Leakage: Stealing cryptographic keys from the cuPQC library (post-quantum cryptography) and LLM weights.
   • Stealthy Code Tampering: Modifying a single branch in the cuBLAS assembly code to universally degrade ML model accuracy (from ~80% to 0.1%) without crashing the model or significantly impacting latency, making it harder to detect than weight tampering.
   • Root Access: Escalating to root privileges on the host system.

Results

The authors evaluated GPUBreach on an NVIDIA RTX A6000 with 48GB GDDR6 memory (ECC disabled).

• Bit-Flip Success: In a 6-bank profiling campaign, they identified 9 bit-flips suitable for PTE corruption out of 34 total flips.
• Attack Speed: The end-to-end privilege escalation (Steps 1–4) takes approximately 17 seconds on average, including the time to find a valid PFN corruption.
• Data Leakage: In experiments with cuPQC, the attacker successfully recovered private keys in 4.4% of key-exchange runs (44 out of 1000), demonstrating reliable leakage of short-lived secrets.
• ML Degradation: Tampering a single branch in cuBLAS reduced the accuracy of multiple models (AlexNet, ResNet50, etc.) to 0.1% while maintaining inference latency within ±10% of the original.
• CPU Escalation: The attack successfully bypassed IOMMU protections by exploiting unvalidated data structures in the NVIDIA kernel driver, achieving system-wide root access.

Significance and Claims

The paper claims that GPUBreach fundamentally changes the threat model for GPU security. Unlike previous GPU Rowhammer attacks which were limited to degrading model accuracy, this work proves that GPUs are as potent as CPUs regarding Rowhammer-based privilege escalation.

The authors emphasize that:

• Trust Boundaries are Broken: The assumption that a compromised GPU can only affect its own memory is false; it can compromise the host OS kernel.
• IOMMU is Insufficient: Standard defenses like IOMMU do not prevent this attack because the vulnerability lies in the driver's failure to sanitize data structures written by the GPU (which the attacker controls via Rowhammer).
• Broad Applicability: While demonstrated on the A6000, the exploitation techniques (page table massaging and driver vulnerabilities) are applicable to other NVIDIA GPU generations (Ampere, Ada, Hopper, Blackwell) and driver versions, provided the GPU is vulnerable to Rowhammer and ECC is disabled.

The authors responsibly disclosed the findings to NVIDIA, Google, AWS, and Microsoft. NVIDIA indicated they may update their Rowhammer security notice, and Google awarded a bug bounty. The authors conclude that effective Rowhammer mitigations for GPUs and a re-evaluation of the trust placed in GPU-writable driver data are critical.