💻 computer science

High-Performance FP16 General Matrix Multiplication on NVIDIA Ada Lovelace via CUTLASS and WMMA: A Benchmark and Pipeline-Depth Analysis

This paper benchmarks FP16 GEMM performance on NVIDIA's Ada Lovelace RTX 4060 across cuBLAS, CUTLASS, and custom WMMA implementations, revealing that a pipeline depth of three with specific tile geometry achieves 52.7% of peak throughput while demonstrating that shared-memory pressure, rather than pipeline depth, is the primary bottleneck limiting further optimization.

Original authors: Lucas Lima Freitag

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lucas Lima Freitag

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-fast kitchen (a GPU) with a team of specialized chefs called Tensor Cores. These chefs are incredibly fast at chopping and mixing ingredients (matrix multiplication) for a giant feast (deep learning). The kitchen on the new "Ada Lovelace" stove (an RTX 4060 graphics card) is theoretically capable of serving 60.55 TFLOPS (that's 60.55 trillion math operations per second).

But here's the catch: just because the chefs can chop that fast doesn't mean they are chopping that fast. They often have to wait for ingredients to be delivered from the pantry.

The Big Experiment: How to Keep Chefs Busy

The researchers wanted to figure out the best way to organize the kitchen so the chefs never stop chopping. They tested three different ways to run the kitchen:

The Black Box (cuBLAS): A pre-packaged, closed-source recipe from NVIDIA that usually works great but doesn't let you tweak the details.
The Open Blueprint (CUTLASS): A flexible, open-source toolkit that lets you build your own kitchen layout from scratch.
The DIY Approach (WMMA): A custom-built kitchen where you control every single movement of the chefs, but it's much harder to build.

They set up a massive cooking challenge with ingredients sized 8192 × 8192 and tried different "pipeline depths." Think of pipeline depth as the number of ingredient trays the kitchen keeps ready.

Pipeline Depth 2: Only 2 trays ready.
Pipeline Depth 3: 3 trays ready.
Pipeline Depth 4: 4 trays ready.

The common wisdom (and a simple math model) suggested that more trays = more speed. The logic was: "If we have more trays, the chefs will never run out of ingredients, so they'll work faster."

The Surprise: More Isn't Always Better

The researchers measured the speed, and here is what they found:

Depth 2: The kitchen ran at 25.6 TFLOPS. The chefs were waiting around too much.
Depth 3: The kitchen sped up to 31.9 TFLOPS (which is 52.7% of the theoretical maximum). This was the sweet spot!
Depth 4: The kitchen actually slowed down to 31.1 TFLOPS.

This is the main finding: Adding that fourth tray didn't help; it made things worse. The simple math model predicted the speed should have jumped to 35.4 TFLOPS, but it didn't. The model was wrong by 13.8%.

Why Did Adding a Tray Backfire?

The paper explains that the kitchen has limited space. When they added the 4th tray (increasing the "Shared Memory" usage to 96 KB), it forced the kitchen to reduce the number of cooking teams (threadblocks) that could work at the same time.

With 3 trays, the kitchen could fit 2 teams of chefs on each stove.
With 4 trays, the kitchen could only fit 1 team.

Even though the single team had more ingredients, they had to wait longer because there were fewer teams to swap in when one got stuck. The kitchen became "crowded" with ingredients but "empty" of workers. The researchers measured that the occupancy (how many teams are working) dropped, and the chefs had to use more "registers" (their personal notepads), which also slowed them down.

The Winners and Losers

The Winner: The CUTLASS toolkit with 3 trays and a specific tile size of 256 × 128. It hit 31.9 TFLOPS. This was so good it actually beat the standard "Black Box" (cuBLAS) by a tiny, almost unmeasurable 1.3% (which the authors say is likely just noise, but it proves the open blueprint can match the closed one).
The DIY Runner-Up: The custom WMMA kernel (the fully DIY kitchen) managed 25.1 TFLOPS on a slightly smaller test. It was slower because it didn't use the fancy "double buffering" tricks to overlap ingredient delivery with cooking.
The Loser: The idea that "more pipeline depth is always better." The paper explicitly rules this out for this specific hardware.

How Sure Are They?

The authors are very confident in these numbers because they measured them directly on the hardware, running the test 10 times and averaging the results. They even checked their math against a built-in NVIDIA tool (the profiler) and found their custom code matched it perfectly.

However, they admit a few things they didn't prove:

They only tested one specific size of problem (8192). They don't know if the "3 trays is best" rule holds for smaller or weirdly shaped problems.
They couldn't test pipeline depths higher than 4 because the kitchen simply ran out of space (memory).
They had to run their tests on Windows, where a driver glitch prevented them from comparing their custom code and the standard code in the exact same run (though they did it in separate runs).

The Takeaway

If you are trying to get the most out of a consumer graphics card like the RTX 4060 for math-heavy tasks, don't just keep adding more buffers. There is a "Goldilocks" zone. In this case, 3 stages of preparation was perfect. Going to 4 stages made the kitchen too cramped, slowing everyone down.

The researchers have released all their code as open-source, so anyone can try to build their own kitchen and see if they can beat the 31.9 TFLOPS record. They suggest that future work should look at different problem sizes and use Linux to get even cleaner data, but for now, they have shown that 31.9 TFLOPS is the current peak for this setup, and that 31.1 TFLOPS is what happens when you try to be too greedy with memory.

Technical Summary: High-Performance FP16 GEMM on NVIDIA Ada Lovelace via CUTLASS and WMMA

Problem Statement
General Matrix Multiplication (GEMM) is the computational foundation of deep learning and scientific computing. While NVIDIA's Ada Lovelace architecture (specifically the consumer-grade RTX 4060 with the AD107 chip) offers a theoretical FP16 Tensor Core peak throughput of 60.55 TFLOPS, achieving near-peak performance requires intricate orchestration of the GPU memory hierarchy. The challenge lies in balancing tiling strategies, software pipelining depth, and threadblock occupancy to saturate Tensor Cores without being bottlenecked by shared memory pressure or register constraints. Prior literature has extensively characterized GEMM at the library-profiler level, but there is a gap in systematically evaluating the impact of arbitrary pipeline depths and tile geometries using the CUTLASS device-level API on consumer hardware.

Methodology
The authors conducted a systematic benchmark of FP16→FP32 GEMM operations on an NVIDIA RTX 4060 (Ada Lovelace) running Windows 11 with CUDA 13.1. The study compared three implementation tiers:

cuBLAS: The closed-source, vendor-optimized library used as a reference.
CUTLASS Device-Level API: An open-source C++ template library allowing compile-time customization of tile dimensions, pipeline depth ( $S$ ), and epilogue vectorization.
Custom WMMA Kernel: A hand-tuned kernel using the CUDA Warp Matrix Multiply Accumulate (WMMA) API for pedagogical validation.

The experiments focused on a fixed problem size of $N = 8192$ ( $M=N=K=8192$ ). The methodology involved:

Pipeline Depth Variation: Testing CUTLASS configurations with pipeline depths $S=3$ and $S=4$ using threadblock tiles of $256 \times 128$ and $128 \times 128$ .
Analytical Modeling: Developing a performance model based on pipeline efficiency ( $S/(S+\kappa)$ ) and extending it to include occupancy constraints ( $\gamma(S)$ ) to predict throughput.
Validation: Ensuring numerical correctness of the custom WMMA kernel against cuBLAS with a relative error threshold of $10^{-2}$ .
Measurement Protocol: Using host-side timing with warmup phases to mitigate clock scaling and initialization overhead, while noting specific Windows driver stability issues (TDR) that necessitated separating cuBLAS and CUTLASS execution streams.

Key Results

Optimal Configuration: The CUTLASS configuration with a pipeline depth of $S=3$ , a $256 \times 128$ threadblock tile, and epilogue vectorization of 4 achieved 31.9 TFLOPS (52.7% of the theoretical peak). This matched the performance of the CUTLASS profiler's built-in s16816gemm_f16 kernel and slightly exceeded the cuBLAS reference (31.5 TFLOPS) by 1.3%, a difference attributed to benchmark overhead rather than algorithmic superiority.
Pipeline Depth Anomaly: Counterintuitively, increasing the pipeline depth to $S=4$ resulted in a performance regression to 31.1 TFLOPS (a 2.5% decrease).
Bottleneck Analysis: The degradation at $S=4$ was attributed to occupancy collapse. While $S=3$ (requiring 72 KB shared memory) allowed 2 threadblocks per Streaming Multiprocessor (SM), $S=4$ (requiring 96 KB) forced the occupancy down to 1 block per SM. This halved the number of concurrent warps, reducing the scheduler's ability to hide memory latency. Additionally, register pressure increased from ~64 to ~72 registers per thread.
Model Accuracy: A simple analytical model accurately predicted performance for $S=2$ and $S=3$ but overestimated $S=4$ by 13.8%. An extended model incorporating occupancy ceilings corrected the $S=4$ prediction to within 1% of the measured value.
Custom Kernel Performance: The custom WMMA kernel achieved 25.1 TFLOPS (41.5% of peak) at $N=4096$ . Its lower performance was attributed to the lack of software pipelining (single buffering) and higher synchronization overhead compared to the optimized CUTLASS templates.

Key Contributions

Reproducible Benchmarking Framework: An open-source framework utilizing the CUTLASS device-level API to instantiate GEMM kernels with arbitrary compile-time parameters on Ada Lovelace GPUs.
Pipeline Depth Empiricism: Experimental evidence demonstrating that deeper pipelines ( $S=4$ ) do not always yield higher throughput on consumer GPUs due to shared memory and occupancy constraints, contradicting simple throughput models.
Calibrated Analytical Model: A performance model that successfully predicts throughput for $S=2$ and $S=3$ but requires an occupancy-capped extension to accurately model the $S=4$ regime.
Pedagogical Reference: A fully validated custom WMMA kernel serving as a reference for low-level Tensor Core programming.
Driver Behavior Observation: Documentation of a specific Windows driver instability (TDR-related) when interleaving CUTLASS and cuBLAS kernels, highlighting practical constraints in cross-library benchmarking on Windows.

Significance and Claims
The paper claims that achieving high-performance GEMM on modern consumer GPUs is a multi-bottleneck problem where pipeline depth is not the sole determinant of throughput. The study highlights that shared memory pressure and occupancy constraints can reverse the expected benefits of deeper software pipelines. By demonstrating that a consumer-grade RTX 4060 can achieve ~53% of the theoretical Tensor Core peak (31.9 TFLOPS), the work suggests that near-datacenter performance levels are accessible on affordable hardware for specific workloads. The authors position their open-source tools and findings as a resource for educators and researchers to study GPU optimization techniques, contributing to the broader goals of accessible high-performance computing and energy-efficient machine learning inference. The work does not claim to solve all GEMM optimization challenges but provides a specific, reproducible analysis of the trade-offs between pipeline depth and occupancy on the Ada Lovelace architecture.