🤖 machine learning

SOLAR: AI-Powered Speed-of-Light Performance Analysis

SOLAR is an automated framework that translates PyTorch and JAX source code into validated Speed-of-Light performance bounds using a hybrid generative-deterministic pipeline, enabling developers to quantify theoretical execution limits, identify optimization opportunities, and guide hardware provisioning across diverse deep learning workloads.

Original authors: Qijing Huang, Sana Damani, Zhifan Ye, Athinagoras Skiadopoulos, Siva Kumar Sastry Hari, Jason Clemons, Sahil Modi, Jingquan Wang, Aditya Kane, Edward C Lin, Humphrey Shi, Christos Kozyrakis

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qijing Huang, Sana Damani, Zhifan Ye, Athinagoras Skiadopoulos, Siva Kumar Sastry Hari, Jason Clemons, Sahil Modi, Jingquan Wang, Aditya Kane, Edward C Lin, Humphrey Shi, Christos Kozyrakis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a race car driver. You have a brand-new, high-performance car (your AI model) and a specific race track (your computer chip). You want to know: What is the absolute fastest time this car could possibly lap this track?

Right now, most tools can only tell you how fast the car actually went in a test run, or they can count the engine's horsepower (how many math problems it solves). But they can't tell you the theoretical speed limit of that specific car on that specific track. Is your car running at 50% of its potential? 90%? Or is it stuck in traffic because of bad road design?

This paper introduces SOLAR, a new tool that calculates that theoretical "Speed of Light" (SOL) limit automatically.

Here is how SOLAR works, broken down into simple analogies:

1. The Problem: Guessing vs. Knowing

Currently, if you want to know the speed limit of a complex AI model, you have to do it by hand. It's like trying to calculate the fastest possible lap time by manually measuring every curve and straightaway with a ruler. It's slow, prone to mistakes, and often wrong.

Old tools either just count the math (ignoring traffic jams) or measure how fast a specific driver is going right now (ignoring the car's potential).
Pure AI guessing tries to predict the time but often gets the math wrong, especially for complex routes.

2. The Solution: The "Translator" and the "Engineer"

SOLAR solves this with a clever two-part team: a Translator and an Engineer.

Step 1: The Translator (The AI Agent)
Imagine you have a messy, handwritten recipe (your AI code written in PyTorch or JAX). A human chef (the AI Large Language Model) reads this messy recipe and rewrites it into a strict, standardized blueprint called Affine Loop IR.
- The Safety Check: Before the blueprint is accepted, SOLAR runs both the original messy recipe and the new blueprint to make sure they produce the exact same dish. If the dishes taste different, the AI tries again. This ensures the translation is 100% accurate.
Step 2: The Engineer (The Deterministic Calculator)
Once the blueprint is verified, a strict, rule-following engineer takes over. This engineer looks at the blueprint and asks: "If we build this exactly as drawn, what is the absolute minimum time it takes?"
- They don't guess. They calculate the math based on the blueprint's structure.
- They create a map of dependencies (an "Einsum Graph"), showing exactly how data flows from one step to the next.

3. The "Speed of Light" Calculation

Once the engineer has the map, they calculate the speed limit using two main factors:

Compute: How fast the engine can do math.
Memory (Traffic): How fast data can move in and out of the engine.

SOLAR looks at the map and says:

"Naïve Speed": If you stop at every single intersection to reload your groceries (data), how long does it take?
"Fused Speed": If you combine two stops into one (fusing operations), you save time.
"Realistic Speed": If you consider that your trunk (cache) is small and you can't carry everything at once, how long does it take?

4. What SOLAR Tells You (The "Headroom")

The most exciting part is what SOLAR reveals: The Gap.

If your current AI model takes 10 seconds to run, and SOLAR says the theoretical limit is 1 second, you have 10x "headroom."

The "Headroom" Analogy: Imagine you are driving at 60 mph in a 100 mph zone. SOLAR tells you, "You are driving at 60, but the car and road can handle 100. Here is exactly where you are slowing down: Is it because you are stopping too often? Is it because your trunk is too small? Is it because you are taking a bad route?"

5. Real-World Examples from the Paper

The authors tested SOLAR on three types of scenarios:

Standard Benchmarks: They found that for complex tasks, current software is often 50 times slower than the theoretical limit. This means there is massive room for improvement just by rearranging how the code runs.
Robotics: They looked at robots that need to react instantly (500 times a second). SOLAR told them: "Your current computer chips are too slow because they can't move data fast enough. You need a chip with 20 times more bandwidth to make this robot work in real-time."
Different Hardware: They showed that you can predict how a model will run on a new, unreleased chip just by doing the math, without needing to buy the chip first.

Summary

SOLAR is like a GPS for AI performance. Instead of just telling you how long your trip took, it calculates the absolute fastest possible trip time for your specific car and road. It then points out exactly where you are wasting time—whether it's bad route planning (algorithms), too many stops (memory traffic), or a small trunk (cache)—so you know exactly how to fix it to reach the "Speed of Light."

Technical Summary: SOLAR – AI-Powered Speed-of-Light Performance Analysis

1. Problem Statement

Modern deep-learning accelerators achieve peak throughputs in petaFLOPS, yet real-world workloads rarely approach these theoretical limits. A critical gap exists in understanding how fast a model could run on target hardware versus how fast it does run. Current tools fail to address this comprehensively:

FLOP counters (e.g., fvcore, ptflops) suffer from narrow operator coverage (75–84% on KernelBench), ignore I/O traffic, and cannot derive theoretical minimum execution times (Speed-of-Light or SOL bounds).
Hardware profilers (e.g., NCU, NSys) measure achieved performance of specific implementations rather than theoretical limits, potentially reporting "headroom" that is physically unachievable due to algorithmic constraints.
Pure LLM estimation is error-prone, particularly on composite workloads, and lacks the systematic search capabilities required for cache-aware analysis.
Existing performance modeling frameworks (e.g., Timeloop, AccelForge) require manual workload specification, which is error-prone and disconnected from rapid model development.

There is no existing tool that automatically derives validated SOL bounds directly from high-level source code (PyTorch/JAX) with broad operator coverage.

2. Methodology

SOLAR (Speed-of-Light Analysis for Runtime) is a source-to-SOL framework that bridges this gap through a three-stage pipeline separating generative translation from deterministic analysis.

Stage 1: Validated Agentic Frontend

The first stage translates arbitrary source programs into an executable Affine Loop IR.

LLM as a Parser: An LLM agent translates source code into the Affine Loop IR, a language-agnostic intermediate representation.
IR Structure: The IR splits programs into:
- Restricted Affine Loop Kernels: Perfectly nested loops over named-dimension tensors with static bounds and affine index expressions, designed for mechanical lifting to einsums.
- Unrestricted Composition Layer: Handles control flow, data movement (reshape, slice), and sparse operations.
Validation Loop: To ensure correctness, the system executes the original program to capture "golden" outputs, generates the IR via the LLM, executes the IR (using Numba), and compares outputs. If a mismatch occurs, diagnostic feedback guides the LLM to correct the translation. This "generate-then-verify" loop ensures the downstream pipeline only receives numerically validated translations.

Stage 2: Deterministic Einsum Generation

Once the IR is validated, a deterministic compiler lifts it into an Einsum Graph.

Graph Construction: The backend traces the IR to build a Directed Acyclic Graph (DAG) where nodes are extended einsums (supporting reductions like max) and edges represent tensor data dependencies.
Mechanical Derivation: From the subscript structure of the einsums, the system mechanically derives exact MAC counts and memory traffic formulas.
Fusibility Analysis: The graph structure identifies adjacent operators that can share on-chip buffers, allowing the system to calculate the cost of fused operations versus independent execution.

Stage 3: SOL Analysis

The analytical backend computes theoretical minimum execution times ( $T_{SOL}$ ) against target hardware specifications (peak throughput $\Pi$ and DRAM bandwidth $\beta$ ).

Roofline Model: $T_{SOL} = \max(\frac{\text{FLOPs}}{\Pi}, \frac{\text{Traffic}}{\beta})$ .
Multi-Fidelity Modes:
1. Unfused: Assumes independent execution with full DRAM round-trips for intermediates.
2. Fused: Eliminates intermediate DRAM traffic by assuming infinite on-chip buffer capacity for fused operators.
3. Cache-Aware Fused (OROJENESIS): Models finite on-chip buffer capacity. It uses integer linear programming to optimize tiling strategies, accounting for re-reads from DRAM when the working set exceeds cache limits.
Inverse Roofline: Given a target latency, the system derives the minimum required hardware specs ( $\Pi_{min}$ and $\beta_{min}$ ).

3. Key Contributions

First Source-to-SOL Framework: SOLAR is the first tool to derive validated SOL bounds directly from PyTorch and JAX source code, achieving 100% analysis coverage on KernelBench.
Hybrid Generative-Deterministic Pipeline: It uniquely combines an LLM for language-agnostic translation (with numerical validation) and a deterministic compiler for reproducible performance analysis. This avoids the errors of pure LLM estimation while bypassing the manual effort of traditional modeling.
Comprehensive Evaluation: The framework provides evidence across kernel, algorithm, and architecture use cases, demonstrating capabilities in headroom analysis, optimization identification, cross-platform exploration, and hardware provisioning.

4. Results

Evaluation was conducted on KernelBench (270 problems), JAX/Flax models, and robotics workloads on Jetson Thor.

Headroom Analysis: SOLAR reveals significant optimization gaps. On KernelBench, the gap between measured runtime and fused SOL ranges from 4.3× (L1) to 54.6× (L3) for PyTorch eager execution. Even torch.compile leaves substantial headroom (e.g., 47.7× on L3), indicating that graph-level fusion is critical for composite workloads.
Multi-Fidelity Insights:
- Fusion: Fused analysis reveals up to 7.8× additional headroom compared to per-operator analysis on KernelBench.
- Caching: Cache-aware bounds tighten SOL by up to 2.25× over naive fused bounds, identifying tiling as a bottleneck in specific scenarios.
- Algorithmic Optimization: Reordering einsum chains (e.g., in DeepSeek MLA) achieved a 2.04× FLOP reduction over naive ordering.
Cross-Platform Projection: SOLAR projects performance across four hardware targets (B200, H100, A6000, Jetson Thor) without physical access. Fusion benefits vary significantly by platform (1.8× on B200 vs. 14.5× on Jetson Thor), highlighting the interaction between bandwidth and fusion.
Hardware Provisioning (Inverse Roofline): For robotics models targeting 500 Hz control loops, SOLAR identified that bandwidth is the primary constraint. For example, the DreamZero WAM model requires a 19.7× improvement in current bandwidth to meet latency targets, suggesting that silicon upgrades alone are insufficient without model compression or algorithmic changes.

5. Significance and Claims

The paper claims that SOLAR fundamentally changes how performance optimization is approached by making theoretical limits accessible and actionable without manual modeling.

Bridging the Gap: It closes the disconnect between rapid model development and hardware performance analysis, providing a "validated" ceiling for optimization.
Actionable Insights: By offering multi-fidelity analysis, it helps engineers distinguish between bottlenecks caused by lack of fusion, poor tiling, or fundamental algorithmic inefficiencies.
Hardware-Agnostic Design: It enables architects and algorithm designers to explore design spaces and provision hardware requirements before silicon exists, preventing the waste of compute and tokens on work with little remaining headroom.

The authors emphasize that while the analysis is based on tensor shapes (not values) and assumes ideal hardware behavior (ignoring thermal throttling), it provides the first automated, validated, and comprehensive view of the theoretical speed-of-light for deep learning workloads.