💻 computer science

FHECore: Rethinking GPU Microarchitecture for Fully Homomorphic Encryption

This paper proposes FHECore, a specialized functional unit integrated into GPU Streaming Multiprocessors that natively supports wide-precision modulo arithmetic to significantly accelerate Fully Homomorphic Encryption workloads, achieving up to 2.12× speedup with minimal area overhead by addressing the mismatch between modern GPU architectures and FHE requirements.

Original authors: Lohit Daksha, Seyda Guzelhan, Kaustubh Shivdikar, Carlos Agulló Domingo, Óscar Vera Lopez, Gilbert Jonatan, Hubert Dymarkowski, Aymane El Jerari, José Cano, José L. Abellán, John Kim, David Kaeli, Aja

Published 2026-07-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lohit Daksha, Seyda Guzelhan, Kaustubh Shivdikar, Carlos Agulló Domingo, Óscar Vera Lopez, Gilbert Jonatan, Hubert Dymarkowski, Aymane El Jerari, José Cano, José L. Abellán, John Kim, David Kaeli, Ajay Joshi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a secret message to a friend, but you are terrified that the mail carrier might peek at it. In the world of computers, this is a huge problem: usually, to do math on a secret, you have to open the envelope, look at the numbers, do the calculation, and then re-seal it. But what if you could do the math while the envelope was still sealed? That is the magic of Fully Homomorphic Encryption (FHE). It lets computers crunch numbers on data that remains locked up tight the entire time, keeping your medical records or bank details safe from prying eyes.

However, there's a catch. Doing math on locked envelopes is incredibly hard work. It's like trying to solve a giant puzzle where every piece is made of glass and you have to wear thick, clumsy gloves. The process is so slow and energy-hungry that it can take thousands of times longer than doing the same math on regular, unlocked data. Because of this, FHE has mostly stayed in the realm of theory, too slow for real-world use. To make it work, scientists have been trying to build special, super-fast computers just for this job, but those are expensive and take years to design. Meanwhile, the world's most powerful computers for other tasks—like the graphics cards (GPUs) in our gaming PCs and data centers—are getting faster every day, but they are built for a different kind of math that doesn't quite fit the "locked envelope" style.

This paper, titled "FHECore: Rethinking GPU Microarchitecture for Fully Homomorphic Encryption," asks a simple question: Can we tweak these powerful, everyday graphics cards to handle the "locked envelope" math without building a brand-new, expensive machine from scratch? The authors, a team of researchers from universities and companies like AMD, found that the current graphics cards are actually missing a specific tool they need. They propose a clever, small addition called FHECore—a specialized "helper unit" built directly into the graphics card's brain.

Think of a modern graphics card (GPU) like a massive, high-speed factory assembly line. It's designed to be incredibly efficient at building low-precision, tiny Lego bricks (like the numbers used in video games and AI). But FHE requires building with massive, heavy, wide-precision steel beams. Trying to use the Lego factory to build with steel beams is a disaster: the workers have to chop the steel beams into tiny Lego-sized pieces, build the structure, and then glue them all back together. This "chop-and-glue" process is what makes FHE so slow on current computers.

The researchers discovered that the two biggest bottlenecks in this process are like two specific assembly steps that happen over and over again. Instead of forcing the factory to chop the steel, they designed a new tool called FHECore. Imagine adding a specialized, heavy-duty press right onto the assembly line. This press can take those massive steel beams, smash them together, and cut them to the perfect size in a single, smooth motion. It doesn't need to chop them into tiny pieces first.

The paper shows that by adding this FHECore tool to the graphics card, they can skip the messy "chop-and-glue" steps entirely. In their simulations, this new tool reduced the number of tiny, repetitive instructions the computer had to follow by about 2.4 times for basic math tasks and nearly 2 times for full, complex applications. This translated to the computer running 1.57 times faster for basic tasks and 2.12 times faster for big jobs like training AI models or decrypting data. Perhaps most excitingly, they managed to cut the time it takes to "bootstrap" (a complex reset step needed to keep the encryption secure) by 50%.

The best part? This new tool is tiny. The researchers calculated that adding it to the graphics card would only increase the size of the chip's "factory floor" by 2.4%. This is a very small price to pay for such a huge speed boost. The paper argues that instead of waiting years to build a custom, expensive machine that might be obsolete by the time it's finished, we can simply upgrade the graphics cards we already have. By fitting this specialized FHECore unit into the existing design, we can finally make secret, encrypted computing fast enough to be useful in our daily lives, all while keeping the rest of the computer's factory running exactly as it did before.

Technical Summary: FHECore: Rethinking GPU Microarchitecture for Fully Homomorphic Encryption

1. Problem Statement

Fully Homomorphic Encryption (FHE) enables computation on encrypted data but suffers from massive computational and memory overheads, often exceeding plaintext execution by several orders of magnitude. While GPUs offer the necessary parallelism and programmability for FHE, modern architectures are increasingly specialized for low-precision machine learning (ML) workloads (e.g., INT8, FP8). This creates a fundamental mismatch: FHE schemes like CKKS rely on wide-precision integer modulo arithmetic (INT32/INT64), which current GPU Tensor Cores do not natively support.

Existing attempts to map FHE to Tensor Cores involve decomposing wide integers into low-precision chunks (e.g., INT8), performing matrix multiplications, and then reassembling the results with intermediate modulo reductions. This approach incurs substantial overhead:

Instruction Overhead: The decomposition and reassembly process generates long chains of fine-grained instructions, increasing front-end pressure and dynamic instruction counts.
Latency: Intermediate reassembly and modulo reduction account for nearly 40% of the total Number Theoretic Transform (NTT) latency.
Inefficiency: Current GPUs lack native opcodes for wide-precision modulo operations, forcing software to implement complex reduction sequences.

2. Methodology and Design

The authors propose FHECore, a specialized functional unit integrated directly into the GPU's Streaming Multiprocessor (SM) to natively accelerate modulo-linear transformations central to FHE.

Architectural Integration

Placement: FHECore is instantiated alongside existing CUDA cores and Tensor Cores within each SM. The design mirrors the Tensor Core count (e.g., 4 FHECores per SM in an A100 configuration) to maintain symmetry with the warp scheduler and register file access.
Microarchitecture: FHECore utilizes a 16×8 systolic array of Processing Elements (PEs). Each PE performs a modulo multiply-accumulate operation ( $R \leftarrow (R + a \cdot b) \mod q$ ) on 32-bit operands.
Modulo Reduction: To eliminate long instruction sequences for reduction, each PE incorporates a dedicated Barrett reduction pipeline. This hardware implementation handles the modulo operation natively, removing the need for software-level predicate and multiplication chains.
Dataflow: The design adopts an output-stationary dataflow. Unlike operand-stationary flows where one operand stalls while traversing the pipeline, output-stationary flow forwards both operands every cycle, allowing the 6-stage PE pipeline to operate continuously without bubbles. This reduces the latency of a 16×8×16 matrix multiplication to 44 cycles.
Memory Interface: Consistent with Tensor Cores, FHECore operates exclusively on values in the register file. It does not directly interact with shared memory or caches, preserving the existing GPU memory hierarchy and allowing asynchronous data transfers.

Instruction Set Architecture (ISA) Extension

New Intrinsic: The authors introduce a new CUDA intrinsic, fhe_sync, which maps to a custom SASS instruction (FHEC).
Functionality: Similar to mma_sync for Tensor Cores, fhe_sync performs a modulo matrix multiplication. It accepts the modulus $q$ and a precomputed Barrett constant $\mu$ as arguments, programming the PEs directly.
Compilation: The fhe_sync intrinsic compiles down to PTX and subsequently to the custom FHEC instruction, consolidating arithmetic and data-movement operations into a single coarse-grained primitive.

3. Key Contributions

FHECore Functional Unit: A specialized systolic array integrated into the GPU SM that natively supports wide-precision modulo-linear transformations (specifically NTT and Base Conversion) with built-in Barrett reduction.
ISA Extension: A new instruction (FHEC) that reduces dynamic instruction counts by consolidating complex modulo arithmetic sequences into single operations.
Comprehensive Evaluation: A simulation-driven evaluation using Accel-Sim and real A100 traces (via NVBit) across CKKS primitives (NTT, INTT, Base Conversion) and end-to-end workloads (Bootstrapping, Logistic Regression, ResNet20, BERT-Tiny).
Area and Performance Analysis: A full physical design flow (RTL to GDS) using the ASAP7 process node to quantify area overhead and verify that the design does not impact the GPU's critical path.

4. Results

The evaluation demonstrates significant performance gains with minimal hardware cost:

Performance Speedup:
- CKKS Primitives: A geometric mean speedup of 1.57×.
- End-to-End Workloads: A geometric mean speedup of 2.12×.
- Bootstrapping: A 50% reduction in latency (from 314.67 ms to 163.90 ms on the A100).
Instruction Count Reduction:
- CKKS Primitives: A geometric mean reduction of 2.41× in dynamic instruction count.
- End-to-End Workloads: A geometric mean reduction of 1.96×.
Area Overhead:
- Integrating FHECore into the A100 increases the die area by only 2.4% (from 826 mm² to 845.91 mm²), remaining well within standard reticle limits.
- This contrasts sharply with prior work (e.g., GME on AMD MI100), which incurred a 26.6% area overhead.
Efficiency: The design achieves these gains without compromising the performance of conventional GPU workloads, as FHE and plaintext ML workloads are mutually exclusive in practice.

5. Significance and Claims

The paper claims that FHECore addresses the fundamental architectural mismatch between modern GPUs and FHE requirements. By recognizing that the dominant bottlenecks in FHE (NTT and Base Conversion) can be formulated as modulo-linear transformations, the authors demonstrate that a specialized, coarse-grained functional unit can drastically reduce instruction overhead and latency.

The significance of this work lies in its practicality and scalability:

Minimal Overhead: Unlike ASICs or FPGAs which have high time-to-market and cost, or previous GPU modifications that exceeded reticle limits, FHECore offers a low-area (2.4%) solution that fits within existing manufacturing constraints.
Ecosystem Compatibility: By integrating into the SM and reusing the existing register file and programming model (via fhe_sync), FHECore leverages the mature CUDA ecosystem, avoiding the need for entirely new software stacks.
Future-Proofing: The design is not tied to the A100 specifically and can be extended to newer architectures (e.g., Hopper, Blackwell) and other cryptographic workloads relying on modulo arithmetic, such as Zero-Knowledge Proofs and Post-Quantum Cryptography.

The authors conclude that FHECore provides a viable, high-performance path for deploying privacy-preserving applications at scale on existing GPU infrastructure.