LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis

LUMINA is an LLM-driven framework that enhances GPU architecture exploration for AI workloads by automatically extracting design rules and performing bottleneck analysis, achieving significantly higher efficiency and better performance-area trade-offs than existing methods with minimal search cost.

The Gist

Imagine you are an architect trying to design the ultimate supercomputer chip (a GPU) to run massive AI brains (like the ones powering chatbots). You have a massive warehouse of building blocks: different sizes of processors, memory banks, and data highways.

The problem? There are 4.7 million possible ways to stack these blocks.

If you tried to build and test every single one, it would take longer than the age of the universe. Even the smartest computer algorithms we have today are like blindfolded people throwing darts in the dark; they might hit a good design eventually, but they waste a lot of time (and money) testing terrible ones first.

Enter "Lumina."

Lumina is a new system that uses a super-smart AI (a Large Language Model, or LLM) to act as a master architect who doesn't just guess, but actually understands how the chip works. Here is how it works, using some everyday analogies:

1. The "Blindfolded Dart" vs. The "X-Ray Vision"

• Old Methods (The Dart Throwers): Traditional tools try random combinations or follow simple rules made by humans. They are like trying to find the best recipe by tasting 1,000 random soups. It's slow and inefficient.
• Lumina (The X-Ray Vision): Lumina reads the "instruction manual" (the simulator code) of the chip. It doesn't just guess; it learns the physics of the design. It knows that if you make the "data highway" (interconnect) too narrow, the soup gets cold (slow performance), no matter how big the pot (processor) is.

2. The "Traffic Cop" Strategy

Imagine the chip is a busy city. Sometimes, traffic jams happen because of a specific bottleneck (like a tiny bridge causing a gridlock).

• The Old Way: You might try widening the whole city or adding random lanes everywhere, hoping traffic gets better.
• Lumina's Way: Lumina acts like a smart traffic cop. It looks at the city, spots the exact bridge causing the jam, and says, "Okay, let's widen just that bridge and maybe shrink the park next to it to save space." It makes targeted, intelligent changes rather than random guesses.

3. The "Self-Correcting" Student

One of the coolest things about Lumina is that it learns as it goes.

• Imagine a student taking a test. If they get a question wrong, a normal computer just moves to the next question.
• Lumina is like a student who stops, looks at why they got it wrong, updates their mental notes, and says, "Ah, I see! I thought making the engine bigger would help, but actually, it just made the car too heavy. Next time, I'll focus on the tires instead."
• This "reflection loop" allows it to get smarter with every single test it runs.

4. The "Magic Benchmark"

The researchers knew that AI can sometimes "hallucinate" (make things up). So, they created a special exam (the DSE Benchmark) to test if the AI was actually good at chip design or just good at sounding confident.

• They gave the AI questions like: "If the chip is slow at reading memory, what part should we change?"
• Only the AI that truly understood the logic passed the test. This ensured Lumina was using a "smart" brain, not a "lucky" one.

The Result: A Miracle in 20 Steps

The most impressive part of the story is the result.

• The researchers had a design space with 4.7 million possibilities.
• They compared Lumina against the current champion chip (NVIDIA's A100).
• While other methods needed thousands of tries to find one good design, Lumina found 6 designs better than the A100 in just 20 tries.

It's as if you were trying to find the best route across a continent. Everyone else was driving randomly, burning fuel for weeks. Lumina looked at the map, understood the terrain, and found the perfect highway in 20 minutes.

Why Does This Matter?

AI is getting bigger and more expensive. Designing the chips to run them is becoming a bottleneck. Lumina proves that by using AI to help design AI chips, we can:

1. Save massive amounts of money (less simulation time).
2. Find better chips faster (better performance for the same size).
3. Make AI more sustainable (less energy wasted on bad designs).

In short, Lumina is the co-pilot that helps engineers fly their chip designs to new heights without crashing the plane.

Technical Summary

Here is a detailed technical summary of the paper "LUMINA: LLM-Guided GPU Architecture Exploration via Bottleneck Analysis."

1. Problem Statement

Design Space Exploration (DSE) for modern GPUs, particularly for Large Language Model (LLM) inference workloads, faces three critical challenges:

• High-Dimensional Complexity: The design space is vast (e.g., ~4.7 million configurations for a specific node) and multi-modal, involving interdependent parameters like compute units, cache hierarchy, interconnects, and memory bandwidth.
• Prohibitive Evaluation Costs: High-fidelity simulation of a single design point can take hours. Exploring thousands of points is computationally expensive (e.g., 6,000 CPU hours for 1,000 designs).
• Inefficient Existing Methods:
  • Expert-Driven Heuristics: Rely on manual critical-path analysis. While sample-efficient, they lack generalization, require deep domain expertise, and struggle with complex interactions between multiple bottlenecks.
  • Machine Learning (ML) Approaches: Methods like Bayesian Optimization (BO) or Genetic Algorithms (GA) learn from data but suffer from low sample efficiency in high-dimensional spaces and require massive training datasets, making them expensive to deploy.
  • LLM Limitations: While LLMs possess reasoning capabilities, their application to DSE is unproven, prone to hallucinations, and lacks a systematic framework for consistent architectural reasoning.

2. Methodology: The Lumina Framework

Lumina is an LLM-driven framework that bridges the gap between black-box search and white-box expert knowledge. It operates on an iterative loop of Knowledge Acquisition, Strategy Formulation, and Refinement.

A. Architectural Heuristic Knowledge (AHK) Acquisition

Instead of relying on static human rules, Lumina automatically extracts structural and quantitative knowledge from the simulator code:

1. Qualitative Engine (QualE): Uses an LLM to parse simulator code and perform static analysis. It generates an Influence Map that explicitly defines causal dependencies between architectural parameters (e.g., core count) and performance metrics (e.g., throughput). This prunes the search space by identifying irrelevant parameters.
2. Quantitative Engine (QuanE): Executes automated sensitivity analysis (micro-benchmarks) to assign quantitative influence values to the relationships identified by QualE. This provides robust priors for the exploration process.

B. Strategy and Exploration Engines

• Strategy Engine (SE): Analyzes simulation results (PPA and critical-path data) to identify the dominant bottleneck (e.g., interconnect congestion). Using the AHK, it formulates a mitigation strategy, deciding which parameters to adjust and the "aggressiveness" of the change (e.g., increasing interconnect links while reducing core count to balance area).
• Exploration Engine (EE): Translates the SE's directives into simulator inputs, executes the evaluation, and records the results in Trajectory Memory (TM).

C. Refinement Loop

Lumina continuously learns. It reflects on the trajectory history in the TM to identify failed design patterns. It updates the quantitative influence factors in the AHK based on observed data, allowing the system to adapt to non-linear behaviors and correct its own reasoning over time.

D. DSE Benchmark

To ensure reliability, the authors introduced the first benchmark specifically for LLM-driven DSE. It evaluates LLMs on three core skills:

1. Bottleneck Attribution: Identifying which parameters cause performance stalls.
2. Performance/Area Prediction: Forecasting metrics based on design changes.
3. Parameter Tuning: Selecting optimal configurations under constraints.
   The benchmark revealed that raw LLMs often fail due to circular reasoning or ignoring constraints; however, integrating corrective rules (derived from failure analysis) significantly improved accuracy.

3. Key Contributions

1. Lumina Framework: A novel, sample-efficient DSE framework that leverages LLMs to automate the extraction of architectural knowledge and guide exploration. It outperforms ML baselines by 17.5× in sample efficiency and achieves 32.9% higher Pareto Hypervolume (PHV).
2. DSE Benchmark: A principled, reproducible benchmark to evaluate and select LLMs for architecture reasoning, exposing systematic failure modes (e.g., hallucination in parameter tuning) and establishing a baseline for consistent reasoning.
3. Counter-Intuitive Design Discovery: Lumina uncovered a strategy that contradicts traditional intuition: reallocating area from core counts to tensor-compute units and memory bandwidth yields superior performance.
   • Design A: Achieved 1.805× TTFT/Area and 1.770× TPOT/Area compared to NVIDIA A100, with 23% less area.
   • Design B: Prioritized Time-to-First-Token (TTFT), achieving 0.592× TTFT (faster) with reduced area.

4. Experimental Results

• Setup: Evaluated on a 4.7 million-point design space for GPT-3 inference, using the NVIDIA A100 as the reference baseline.
• Efficiency:
  • In a 1,000-sample budget, Lumina found 421 designs superior to A100, whereas the best black-box baseline (Ant Colony Optimization) found only 24.
  • Lumina achieved 32.9% higher PHV than ML baselines.
• Extreme Constraints: Under a strict budget of only 20 samples (simulating high-cost end-to-end inference), black-box methods found zero designs better than A100. Lumina successfully identified 6 superior designs in this same budget.
• Benchmark Performance: With enhanced prompting and corrective rules, the Qwen-3 model achieved 80% accuracy in bottleneck analysis and 82% in performance prediction, demonstrating that LLMs can be reliable architects when guided by the framework.

5. Significance

• Paradigm Shift: Lumina moves DSE from "trial-and-error" or "static heuristics" to dynamic, AI-guided reasoning. It proves that LLMs can effectively replace human experts in the initial knowledge extraction and iterative optimization phases.
• Cost Reduction: By drastically reducing the number of required simulations (from thousands to dozens) to find high-quality designs, Lumina significantly lowers the cost and time of GPU architecture development.
• Scalability: The framework is designed to be generalizable across different architectures and workloads, offering a scalable solution for the rapidly evolving AI hardware landscape.
• Insight Generation: The discovery of the "area reallocation" strategy highlights that LLMs can uncover non-obvious trade-offs that human experts might overlook, leading to more efficient AI infrastructure.