How Much Progress Has There Been in NVIDIA Datacenter GPUs?

Problem Statement
The rapid evolution of Artificial Intelligence (AI) has made Graphics Processing Units (GPUs) the de facto standard for training and inference, fundamentally altering the landscape of scientific research and industrial computing. However, the pace of this progress is difficult to quantify accurately due to conflicting metrics, varying data precisions, and the inclusion of non-dominant hardware in previous studies. Furthermore, the critical role of AI chips in global competition has led to U.S. export control regulations, creating a need to understand the technical trajectory of these devices to assess the potential performance gaps between nations. Existing literature, such as "Huang's Law," often relies on narrow datasets or mixes data precisions (e.g., FP4 to FP16), leading to inflated progress estimates. Other studies focus on broad hardware categories, diluting the specific trends of the dominant datacenter class. This paper addresses the need for a rigorous, apples-to-apples analysis of technical progress in NVIDIA datacenter GPUs from the mid-2000s through 2025, specifically examining computing performance, memory, power, and cost, while also evaluating the impact of recent U.S. export controls.

Methodology
The authors constructed a comprehensive dataset comprising 102 NVIDIA datacenter GPUs released from the Tesla microarchitecture (2006) through Blackwell Ultra (2025).

• Data Collection: Metrics were gathered from NVIDIA documentation (whitepapers, datasheets) and secondary sources like TechPowerUp and VideoCardz. The study focuses on theoretical peak performance values (TFLOPS) to ensure consistency across decades, rather than relying on variable real-world benchmarks.
• Metric Selection: Key metrics include Floating Point (FP) performance at 16-bit (FP16), 32-bit (FP32), and 64-bit (FP64) precisions; off-chip memory size and bandwidth; release prices; and Thermal Design Power (TDP). The analysis distinguishes between CUDA Cores and Tensor Cores, and accounts for hardware sparsity support introduced in the Ampere architecture.
• Analytical Approach: The authors employed log-linear exponential regression models to calculate Compound Annual Growth Rates (CAGRs) and Doubling Times (DTs) with 90% confidence intervals.
• Comparative Scope: The study analyzes two subsets: (1) the top-performing GPU per year to identify peak capability trends, and (2) the entire dataset of datacenter GPUs to understand average industry progress.
• Export Control Analysis: The paper evaluates U.S. export regulations (2022–2025) by applying specific criteria (Total Processing Performance, performance density, and I/O bandwidth) to the dataset to estimate the resulting performance gaps between unrestricted and restricted markets.

Key Contributions and Results
The paper identifies ten key insights regarding the trajectory of NVIDIA datacenter GPUs:

1. Surpassing Moore's Law: GPU progress has outpaced the historical doubling time of Moore's Law. FP16 and FP32 performance doubling times are approximately 1.43 and 1.67 years, respectively, driven by AI demand.
2. Chip-Level Enhancements: Performance gains are largely attributed to architectural shifts, including the introduction of FP16 hardware support (Pascal, 2016) and specialized Tensor Cores (Volta, 2017).
3. Rapid FP16/FP32 Growth: The computing power for FP16 and FP32 doubles in less than two years. When accounting for sparsity support, these doubling times decrease further to 1.27 and 1.46 years.
4. Deprioritization of FP64: FP64 compute units have become a lower-priority resource. Growth in FP64 performance is significantly slower (doubling every 2.05 to 3.79 years), and top-tier AI-focused chips like the Blackwell Ultra B300 feature reduced FP64 capabilities.
5. HBM Impact: The transition to High Bandwidth Memory (HBM) significantly boosted off-chip memory size and bandwidth growth rates (CAGRs of 31.8% and 26.8% for HBM-only, respectively) compared to GDDR.
6. Cost and Power Divergence: While average datacenter GPUs show moderate growth in price and power, the top-performing subset sees release prices and TDP growing at nearly twice the rate of the broader lineup (22.7% and 8.6% CAGR, respectively).
7. The Memory Wall: Computing power is growing significantly faster than off-chip memory bandwidth. The ratio of performance to bandwidth is increasing, suggesting a potential bottleneck for memory-intensive workloads.
8. Efficiency vs. Cost: Technical improvement per watt is growing faster than improvement per dollar, as release prices have risen more sharply than power consumption.
9. Competitive Landscape: While AMD and Intel have narrowed the performance gap in specific areas (notably FP16 and FP64), the margin remains insufficient to displace NVIDIA's market dominance, largely due to the maturity of the CUDA software ecosystem.
10. Export Control Impact: The analysis quantifies the performance gap created by U.S. export controls. Under the 2022 regulations, the gap between unrestricted and restricted access was estimated at 23.6×. Recent policy updates allowing the export of the H200 reduced this theoretical gap to 3.54×.

Significance and Claims
The paper claims to provide the most comprehensive and consistent analysis of NVIDIA datacenter GPU progress to date, correcting the inflated estimates found in "Huang's Law" and the diluted trends of broader hardware studies. By isolating the datacenter class and separating data precisions, the authors offer a realistic view of the "scaling laws" driving the AI revolution.

The study highlights that while architectural innovations (like Tensor Cores and multi-die designs) have successfully kept pace with exploding AI demands, they have come at the cost of increased power consumption and price, particularly for top-tier devices. Furthermore, the paper provides a data-driven assessment of geopolitical constraints, suggesting that while export controls create significant short-term performance disparities, they may not permanently halt technological diffusion, as evidenced by the rapid development of alternative domestic solutions and the potential for policy adjustments (such as the H200 export approval) to mitigate these gaps. The authors conclude that while current trends are impressive, the sustainability of such rapid growth rates faces diminishing returns due to physical and architectural limits, necessitating future research into the long-term viability of these trajectories.