💻 computer science

FP8 is All You Need (Part 2): Efficient Ozaki-Bailey Style FFT Through Tensor-core Garner Reformulation and Kulisch Escape Route

This paper proposes "Ozaki-Bailey FFT," a method that enables full FP64-accurate 3-D FFTs on NVIDIA Blackwell Ultra GPUs by reformulating the computation through FP8 tensor cores and Kulisch fixed-point arithmetic, thereby overcoming the hardware's reduced native FP64 throughput to achieve memory-bound performance parity.

Original authors: Satoshi Matsuoka

Published 2026-06-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Satoshi Matsuoka

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Heavy Lifter" Lost Its Strength

Imagine a high-performance computer chip (like the new NVIDIA B300) as a massive construction site. In the past, this site had a giant, super-strong crane (the FP64 vector pipe) capable of lifting heavy, precise steel beams (scientific calculations) with incredible speed.

However, the new chip design decided to focus almost entirely on building AI models. To make room for AI, they replaced the giant crane with a fleet of tiny, super-fast delivery drones (the FP8 tensor cores). These drones are amazing at moving light packages (AI data) but are terrible at lifting the heavy, precise steel beams needed for scientific work like weather forecasting or physics simulations.

The result? The site is so fast at moving light packages that it's actually waiting for the trucks to arrive (memory speed), but the few heavy cranes left are so slow that if you try to use them, the whole project grinds to a halt.

The Goal: Build a Bridge to the "Memory Roof"

The authors want to get scientific calculations running as fast as the memory trucks can deliver data. This speed limit is called the "Memory Roof." Currently, the B300 chip is stuck far below this roof because its heavy crane is too weak.

The paper proposes a clever, three-part construction plan to bypass the broken crane and reach the roof again, using only the tools the chip already has.

The Three-Part Solution

1. The "Ozaki-Bailey" Strategy: Breaking the Beam into Bricks

Instead of trying to lift the heavy steel beam (a complex 3D math problem called a 3D FFT) all at once, the team breaks it down.

The Metaphor: Imagine you need to move a giant, fragile statue. You can't lift it whole. Instead, you break it into thousands of small, manageable Lego bricks.
The Tech: They use a mathematical trick called the Bailey six-step decomposition to break the big math problem into tiny pieces. Then, they use the Ozaki Scheme, which translates these pieces into "bricks" that the tiny, fast delivery drones (FP8 tensor cores) can handle easily.

2. The "Garner" Problem: The Reassembly Bottleneck

Once the drones have moved all the Lego bricks, you have to put them back together to rebuild the statue.

The Problem: In the old way of doing this (called Recursive Garner), reassembling the bricks was slow and clumsy. It was like trying to glue a million tiny bricks together by hand. On the new chip, this reassembly step took 260 milliseconds, which is 20 times slower than the memory trucks could deliver the bricks. It was the new bottleneck.
The Fix (Phase A): The authors realized they could use the fast drones to do the "gluing" for the first part of the reassembly. They split the job:
- Phase A: The fast drones do the heavy lifting of the initial assembly. This is super fast.
- Phase B: The final, tricky part of putting the statue together. This is where the old method failed.

3. The "Kulisch Escape Route": The Secret Weapon

This is the paper's most creative innovation.

The Problem: The final step (Phase B) usually requires a very precise, heavy-duty calculator (the FP64 pipe) to sum up the numbers. But on the B300 chip, that heavy calculator is broken/slow.
The Solution: The authors found a way to use a different tool that the chip didn't cut down: the INT32 pipe (a standard integer calculator).
The Metaphor: Imagine you need to count a massive pile of sand to the exact grain. The "heavy crane" (FP64) is broken. But you have a fleet of counting robots (INT32) that are incredibly fast at adding whole numbers.
- The authors realized that if they treat the sand grains as simple whole numbers and use a "wide bucket" (a Kulisch accumulator) to catch them, the counting robots can do the job perfectly.
- They don't need the broken heavy crane at all. They just use the fast counting robots to do the final sum, then pour the result into the heavy crane once at the very end.
The Result: This "Kulisch" method allows the chip to finish the job in 18 milliseconds, which is almost as fast as the memory trucks can deliver the data (the Memory Roof).

The "Four-Floor" Rule for Future Chips

The authors analyzed this process and created a rulebook for chip designers, called the "Four-Floor Codesign Rule." To ensure a chip can handle these scientific tasks in the future, it must meet one of two conditions:

The Native Floor: Keep the heavy crane (FP64) strong enough to do the job on its own.
The Kulisch Escape Route: If you weaken the heavy crane, you must keep the counting robots (INT32) and the delivery drones (FP8) strong enough to do the work together.

The Verdict on Current Chips:

H100 and B200: They have strong heavy cranes. They don't need the trick; they just do the job normally.
Rubin (Future Chip): It has a slightly weaker crane but is still just strong enough to do the job normally.
B300 (The Problem Child): Its heavy crane is 10 times too weak. However, because its counting robots (INT32) and delivery drones (FP8) are still strong, the authors' "Kulisch Escape Route" saves it. They can still run these scientific tasks at top speed, but only if they use this specific software trick.

Summary

The paper says: "Don't panic if the heavy crane is gone. If you break the job into small pieces, use the fast drones to move them, and use the fast counting robots to do the final math, you can still reach the speed limit of the memory trucks."

This proves that even with a chip designed primarily for AI, we can still do high-precision scientific computing efficiently, provided we use the right software "hacks" to route the work around the missing hardware.

Technical Summary: FP8 is All You Need (Part 2)

1. Problem Statement

The paper addresses the architectural shift in NVIDIA's Blackwell Ultra (B300) and Rubin GPUs, where FP64 vector throughput has been drastically reduced to prioritize AI workloads (FP8/FP16 tensor cores). Specifically, the B300's FP64 vector throughput (~1.3 TFLOPS) is approximately 30× lower than the B200 and insufficient to sustain memory-bound scientific workloads, such as 3D Fast Fourier Transforms (FFT), at the memory bandwidth roof.

While the companion paper [1] demonstrated that the Ozaki Scheme II (Ozaki-II) emulation framework could recover FP64-equivalent throughput for dense matrix operations (GEMM, GEMV, Stencils, SpMV) by routing computations through FP8 tensor cores, a critical gap remained: the 3D FFT. The FFT presents a unique bottleneck where the reconstruction cost (Garner reconstruction) scales differently than in dense GEMM, potentially rendering the standard Ozaki-II approach inefficient or infeasible on these "FP64-collapsed" architectures.

2. Methodology

The authors propose Ozaki-Bailey FFT, an emulation framework that combines the Ozaki-II residue number system with the Bailey six-step decomposition for 3D FFTs. The methodology involves a three-part analysis and reformulation:

A. The $\gamma$ -Roof Analysis

The authors analyze the TME (Tensor-Memory Equilibrium) model for the FFT kernel. Unlike dense GEMM, where the compute ( $\alpha$ ) and memory ( $\beta$ ) terms dominate, the FFT case at Bailey inner dimension $k \approx \sqrt{N} = 32$ (for $N=1024$ ) causes the Garner reconstruction latency ( $\gamma$ ) to become the binding constraint.

Problem: Standard recursive Garner reconstruction requires $O(r^2)$ integer operations per output element. On B300, this results in a wall time of ~260 ms, roughly 20× slower than the memory roof (12.9 ms).
Reformulation: The authors decompose Garner reconstruction into two phases:
- Phase A: Inner products using mantissa-sliced coefficients (forward CRT) mapped to FP8/INT8 tensor cores. This step is highly efficient (~0.5 ms).
- Phase B: A per-output reduction of the Phase A results. In a naive implementation, this runs on the collapsed FP64 vector pipe, creating a new bottleneck (~163 ms).

B. The Kulisch Escape Route (Phase B Reformulation)

To resolve the Phase B bottleneck without relying on the weak FP64 vector pipe, the authors survey classical exact-accumulation schemes. They identify Kulisch's fixed-point complete-arithmetic accumulator [16, 17] as the solution.

Mechanism: Instead of floating-point summation, the reduction is performed using a wide fixed-point accumulator (160-bit, implemented via five INT32 registers) on the INT32 SIMT pipe.
Precision: The accumulation is exact (no intermediate rounding). The only rounding occurs once, at the final conversion from the 160-bit integer to FP64. This maintains full FP64 accuracy (better than naive FP64 summation, which accumulates ~11 ulps of error).
Performance: The B300's INT32 throughput (~75 TOPS) is ~58× higher than its FP64 throughput. Routing Phase B here reduces the projected wall time to ~18 ms.

C. Four-Floor Codesign Rule

The paper derives four "bandwidth-parity floors" (minimum throughput requirements relative to memory bandwidth, $BHBM$) required to achieve memory-roof parity for FFTs:

Native FP64 Floor: $\eta_{FP64} \ge 1.56 \cdot BHBM$ . (B300 fails this by ~10×).
Naive Ozaki Phase B Floor: $\eta_{FP64} \ge 2.06 \cdot BHBM$ . (Informational; strictly harder than native).
Kulisch INT32 Sub-floor: $\eta_{INT32} \ge 8.25 \cdot BHBM$ . (B300 meets this with ~14% margin).
FP8 Tensor-Core Floor: $\eta_{FP8} \ge 170 \cdot BHBM$ . (B300 meets this with ~3.7× margin).

3. Key Contributions

Identification of the $\gamma$ -Roof: The paper establishes that for 3D FFTs, the Garner reconstruction cost ( $\gamma$ ) becomes the dominant bottleneck at specific problem sizes ( $k \approx 32$ ), invalidating the assumption that compute/memory terms alone dictate performance.
Tensor-Core Garner Reformulation: The authors introduce a split of Garner reconstruction into Phase A (Tensor Core) and Phase B (SIMT), enabling the use of high-throughput FP8/INT8 hardware for the heavy lifting.
Kulisch Phase B Rescue: The paper proposes using Kulisch's fixed-point accumulator on the INT32 SIMT pipe to perform the Phase B reduction. This is presented as a "software escape route" that allows FP64-collapsed GPUs to achieve full FP64 accuracy without native FP64 vector hardware.
Four-Floor Codesign Rule: A new architectural design rule is proposed. To support full FP64 FFTs on post-FP64 hardware, a GPU must either satisfy the native FP64 floor OR satisfy both the FP8 tensor-core floor and the Kulisch INT32 sub-floor.
Quantitative Projections:
- B300: Native FFT is ~124 ms; Recursive Garner is ~260 ms; Naive Phase B is ~163 ms. The Kulisch path projects ~18 ms, approaching the 12.9 ms memory roof.
- Rubin: Sits within 4% of the native FP64 floor, making emulation unnecessary for FFT.
- H100/B200: Satisfy the native floor comfortably.

4. Results and Claims

Feasibility: The paper claims that B300, despite its reduced FP64 vector pipe, can achieve near-memory-roof performance for full FP64 3D FFTs (1024³) solely through software engineering (Ozaki-Bailey + Kulisch).
Accuracy: The Kulisch method provides strictly better accuracy than naive FP64 summation (single final rounding vs. accumulated error) while running entirely on the INT32 pipe.
Architectural Implications:
- B300: Is "salvageable" for spectral codes if the Kulisch kernel is implemented efficiently (assuming 50–70% INT32 SIMT efficiency).
- Rubin: Is balanced for spectral codes; emulation offers no speedup.
- Future Design: The paper argues that the native FP64 floor ( $\eta \ge 1.56 \cdot BHBM$ ) remains the safe procurement target. The Kulisch route is a "rescue" for existing sub-floor architectures but should not justify further cuts to FP64 or INT32 silicon in future generations. If both FP64 and INT32 are cut, the escape route closes.

5. Significance

The paper concludes that the combination of Ozaki-II emulation, the Bailey decomposition, and the Kulisch accumulator provides an end-to-end kernel-coverage audit for the post-FP64 stack. It demonstrates that even with a "deeply collapsed" FP64 vector pipe, full FP64 scientific computing (specifically FFTs) is not necessarily dead, provided the architecture retains sufficient INT32 SIMT throughput and FP8 tensor-core capacity.

The work motivates the creation of a libKulisch library to implement these wide-accumulator primitives for FFTs, SpMV, and other low-operational-intensity reduction-bound kernels. The authors emphasize that while the projection is promising (~18 ms), empirical measurement of the Kulisch Phase B kernel is the immediate next step to validate the efficiency assumptions.