Technical Summary: EnergyLens

Problem Statement

As Large Language Models (LLMs) evolve to include dense, mixture-of-experts (MoE), and state-space architectures, and as they are deployed on heterogeneous accelerators (NVIDIA, Intel, AMD) for diverse multimodal workloads (text, image, video), optimizing inference energy has become as critical as optimizing latency and throughput. However, existing approaches to energy prediction suffer from three primary limitations:

1. Latency Proxies: Many analytical models treat latency as a proxy for energy. The authors demonstrate that latency and energy optima diverge in over 20% of tested configurations, leading to energy penalties of up to 79.8% when latency-based strategies are used.
2. Black-Box Surrogates: Data-driven machine learning models (e.g., Random Forests, Neural Networks) achieve reasonable accuracy but are "data-hungry," requiring hundreds of profiling samples to generalize. They lack interpretability, offering no physical insight into why a configuration is efficient, and fail to extrapolate reliably to unseen batch sizes or hardware.
3. Lack of Generalizability: Existing analytical models often fail to decouple the distinct physical effects of Tensor Parallelism (TP) and Pipeline Parallelism (PP), or they fail to account for the non-linear scaling of energy with batch size and sequence length across different modalities.

Methodology: The EnergyLens Framework

The authors propose EnergyLens, a framework that uses symbolic regression to discover compact, closed-form mathematical expressions for GPU energy consumption. Unlike black-box models, EnergyLens derives formulas where terms correspond directly to measurable system properties.

1. Unified Energy Model

The core of EnergyLens is a twelve-parameter closed-form equation that decomposes energy ($E$) into three physically grounded components:
$$E = E_{prefill} + E_{decode} + E_{overhead}$$

• Phase Decomposition: The model explicitly separates the compute-bound prefill phase (processing input tokens) from the memory-bound decode phase (generating output tokens).
• Parallelism Decoupling: The model distinguishes between Tensor Parallelism ($d_{TP}$) and Pipeline Parallelism ($d_{PP}$). It recognizes that TP reduces per-GPU computation via efficient tensor-core utilization but introduces synchronization overhead, while PP introduces pipeline bubbles and keeps devices powered during idle stages.
• Non-Linear Scaling: The formula captures the sublinear amortization of fixed overheads (like kernel launches and KV-cache initialization) as batch size increases, and the specific scaling laws for different modalities (e.g., logarithmic compression for video/image workloads).

The general form (Equation 4 in the paper) is:
$$E = \alpha_p \frac{\text{total input tokens}}{(\text{batch size}^{\beta_p} + \epsilon_p)} d_{TP}^{\gamma_{1,p}} d_{PP}^{\gamma_{2,p}} + \alpha_d \frac{\text{max tokens}}{(\text{batch size}^{\beta_d} + \epsilon_d)} d_{TP}^{\gamma_{1,d}} d_{PP}^{\gamma_{2,d}} + \delta_1 d_{TP} + \delta_2 d_{PP}$$

2. Training and Fitting

• Data Collection: A profiling framework systematically sweeps parallelism strategies, quantization schemes, batch sizes, and sequence lengths across NVIDIA, Intel, and AMD GPUs.
• Symbolic Regression: The framework searches the space of mathematical expressions to find the structure that best fits the profiling data.
• Parameter Estimation: Once the formula structure is fixed, the twelve scalar parameters ($\phi$) are fitted using L-BFGS-B optimization on a small representative dataset.
• Sample Efficiency: The model requires as few as 50 profiling measurements to achieve high accuracy, a 10x reduction compared to ensemble ML methods which require ~500 samples.

Key Contributions

1. Unified Closed-Form Model: Development of a single energy model that decouples TP and PP contributions and separates prefill from decode energy, with parameters carrying explicit physical meaning.
2. Symbolic Regression Discovery: Demonstration that symbolic regression can uncover nonlinear formula structures that generalize across diverse architectures (Dense, MoE, SSM), modalities (text, image, video), and hardware vendors without changing the formula template.
3. Systematic Profiling Framework: Implementation of a framework that explores the joint space of parallelism, quantization, and workload parameters to generate the structured data needed for analytical modeling.
4. Direct Energy Modeling: By modeling energy directly rather than using latency as a proxy, the framework identifies more energy-efficient parallelism configurations that latency-based optimizers systematically overlook.

Experimental Results

The authors evaluated EnergyLens across 940 scenarios involving dense transformers, MoE, and state-space models on NVIDIA A100, AMD MI100, and Intel Max 1550 GPUs.

• Configuration Selection Accuracy: EnergyLens achieves 88.2% Top-1 configuration selection accuracy (identifying the single best energy configuration), significantly outperforming the closest analytical baseline, MAVERIQ, which achieved 60.9%.
• Pairwise Ranking: EnergyLens achieves 95.8% pairwise ranking accuracy (correctly predicting which of two configurations is more efficient) compared to 91.5% for MAVERIQ.
• Sample Efficiency: EnergyLens matches the predictive accuracy of ensemble ML methods (Random Forest, Gradient Boosting) while requiring 10x fewer profiling samples (13.0% MAPE at $n=50$ vs. 49.0% MAPE for Random Forest at $n=50$).
• Generalization:
  • Cross-Hardware: The model transfers across GPU vendors (NVIDIA, Intel, AMD) with only scalar coefficients re-fitted.
  • Cross-Modality: It performs robustly on text, image+text, and video+text workloads.
  • Extrapolation: EnergyLens reliably extrapolates to unseen batch sizes (3x to 10x larger than training data) with ≥93% pairwise accuracy on five of seven datasets, whereas linear-batch baselines fail significantly.
• Robustness: Unlike latency-based proxies that require precise per-configuration power measurements, EnergyLens models energy natively and maintains high accuracy even when power telemetry is unavailable.

Significance and Claims

The paper claims that EnergyLens provides a practical, interpretable tool for energy-optimal LLM deployment. Its significance lies in:

• Interpretability: Unlike black-box surrogates, EnergyLens offers "auditable" predictions where operators can inspect terms to understand the physical drivers of energy consumption (e.g., verifying the relative cost of TP vs. PP scaling).
• Actionability: The model enables the identification of Pareto-optimal configurations for deployment without exhaustive measurement.
• Cost-Effectiveness: By reducing the profiling budget from hundreds of samples to ~50, it makes energy-aware configuration tuning feasible for production environments.
• Physical Grounding: The discovery that latency and energy optima diverge, and that pipeline parallelism can reduce energy even when it does not reduce latency, challenges the prevailing assumption that latency is a sufficient proxy for energy efficiency.

The authors conclude that while the current template uses additive terms, future work could explore multiplicative interactions for MoE and long-context workloads, and integration with dynamic schedulers for online reconfiguration. The framework and datasets are intended to be released as open-source.