🤖 AI

Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Task

This paper demonstrates that energy-aware knowledge distillation guided by direct energy-surrogate models, rather than traditional FLOPs metrics, significantly reduces inference energy and memory usage for large language models in software engineering tasks while maintaining performance.

Original authors: Enrique Barba Roque, Luís Cruz, Annibale Panichella

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Enrique Barba Roque, Luís Cruz, Annibale Panichella

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of software, a new kind of intelligence has emerged: large language models. These are vast computer systems trained on enormous libraries of code, capable of understanding programming languages and helping developers write, fix, and explain software. While these tools are incredibly powerful, they come with a heavy price tag. Running them requires massive data centers filled with specialized hardware that guzzles electricity, contributing significantly to energy consumption and environmental strain. Because these systems are so large, they often cannot run on the laptops or personal computers that most people use every day. To make these tools more sustainable and accessible, researchers have turned to a technique called knowledge distillation. Imagine a master teacher passing their knowledge down to a smaller student; in this digital version, a massive, complex model teaches a much smaller, simpler model how to perform the same tasks. The goal is to keep the smaller model smart enough to be useful while making it small enough to run efficiently on everyday hardware.

A team of researchers at Delft University of Technology set out to test how well this process works for software engineering tasks, specifically looking at how to measure the energy cost of these models. For years, the industry has relied on a standard metric called floating-point operations, or FLOPs, to estimate how much energy a model will use. This number counts the theoretical mathematical calculations a model must perform. However, the researchers suspected this number might be misleading, much like counting the number of steps in a journey without accounting for the terrain or the weather. To find the truth, they conducted a controlled experiment using a sophisticated method that automatically searches for the best possible small models. They tested this on two distinct types of problems: finding duplicate code and predicting security flaws in software.

The study began by asking a simple but critical question: does the theoretical number of calculations actually match the real energy used? The researchers ran the models on actual computer hardware and measured the electricity consumed by the processor and graphics card during the work. The results were surprising and inconsistent. For the task of finding duplicate code, the theoretical calculation count had almost no connection to the actual electricity used. A model that performed fewer calculations could sometimes use more energy than a model that performed many more. In contrast, for the task of predicting security flaws, the calculation count did show a relationship to energy use, but even there, it was not a perfect guide. Some models with high calculation counts used significantly less power than others with lower counts. This proved that the standard industry metric is not a reliable way to judge energy efficiency across different types of software tasks.

Because the old metric was flawed, the team developed a new approach. Instead of guessing energy use based on math counts, they built a system that learns from real measurements. They trained a helper model to predict energy consumption based on the specific settings of the student model, using actual data gathered from the hardware. When they used this new, energy-aware system to design the smaller models, the results improved dramatically. For the duplicate code task, the new method found models that used 39 percent less energy than those designed with the old method, without losing any accuracy. For the security prediction task, the results were similar, showing that the new approach could find efficient solutions even when the old metric seemed to work.

The researchers then pushed this method further, applying it to a much more complex challenge: code summarization. This task involves generating a human-readable description of what a piece of code does, which is far more difficult than simple classification. They used a large model called CodeT5+ as the teacher and tried to distill it into a smaller student. The results were striking. The new, energy-focused method produced student models that were 86 percent smaller in memory size and used up to 90 percent less energy to run than the original teacher. The trade-off was a small drop in accuracy, but the summaries remained coherent and useful. The researchers found that for these complex generation tasks, the standard rule of making models as small as possible did not work; models needed to be large enough to understand the code, but the new method found the sweet spot where they were small enough to be efficient.

This work highlights a crucial shift in how we should think about artificial intelligence and sustainability. The study demonstrates that relying on theoretical numbers to estimate energy use can lead engineers to choose models that are not actually the most efficient. By measuring real energy consumption and using that data to guide the design of smaller models, it is possible to create software tools that are both powerful and sustainable. These smaller, energy-efficient models can eventually run on consumer hardware, bringing the benefits of advanced artificial intelligence to local devices without the massive environmental cost of cloud computing. The findings suggest that for a greener future in software, we must look beyond simple counts of operations and measure the actual energy footprint of the tools we build.

Technical Summary: Beyond FLOPs: Energy-Aware Knowledge Distillation for Sustainable LLMs on Code-Related Tasks

Problem Statement

Large Language Models (LLMs) have achieved high accuracy in Software Engineering (SE) tasks such as clone detection, vulnerability prediction, and code summarization. However, their massive scale and computational demands raise significant sustainability concerns, particularly regarding energy consumption and the inability to deploy these models on consumer hardware or resource-constrained platforms.

A prevalent method for estimating the computational cost of LLMs in both literature and industry is the use of Floating Point Operations (FLOPs). FLOPs are attractive due to their simplicity and hardware-agnostic nature, serving as a proxy for energy consumption. However, the validity of FLOPs as a reliable energy metric is debated. FLOPs typically capture only the theoretical count of mathematical operations, ignoring runtime factors that heavily influence energy usage, such as memory access patterns, data movement overheads, and hardware-level optimizations (e.g., on GPUs or TPUs). Furthermore, existing knowledge distillation methods for SE, such as Morph, rely on FLOPs to optimize model efficiency, potentially leading to suboptimal energy savings. Additionally, these methods have been limited to binary classification tasks using older architectures (e.g., BERT), leaving a gap in understanding their applicability to complex generative tasks and newer architectures (e.g., CodeT5+).

Methodology

This paper investigates the implications of energy-aware knowledge distillation for SE by addressing three Research Questions (RQs) through controlled experiments:

RQ1: Validity of FLOPs as an Energy Proxy.
The authors replicated the original Morph experiments using GraphCodeBERT for Clone Detection and Vulnerability Prediction tasks. They generated a Pareto front of student models with varying FLOPs and empirically measured their actual CPU and GPU energy consumption during inference using the EnergiBridge profiler. Statistical analysis (ANOVA) was performed to determine the correlation between FLOPs and measured energy.
RQ2: Energy Efficiency of Morph with Direct Energy Optimization.
The authors modified the Morph methodology to replace FLOPs with direct energy consumption as an optimization objective. Instead of minimizing theoretical FLOPs, they introduced a surrogate model approach:
- They sampled hyperparameter configurations using Latin Hypercube Sampling (LHS).
- They measured actual energy consumption for these configurations.
- They trained regression models (Gradient Boosting) to predict energy consumption based on hyperparameters.
- The Many-Objective Evolutionary Algorithm (AGE-MOEA) then optimized for accuracy, robustness, size, and the predicted energy consumption.
  This approach was tested on the same classification tasks to compare the resulting models against those optimized for FLOPs.
RQ3: Applicability to Generative Tasks.
The authors extended the modified Morph methodology to a generative task: Code Summarization, using the CodeT5+ model (220M parameters).
- Weight Subcloning: To address the slower convergence of randomly initialized students in generative tasks, they applied weight subcloning, copying the most important weights from the teacher to the student based on input/output magnitudes.
- Online Distillation: Due to the infeasibility of storing logits for large vocabularies, they performed online distillation, running teacher and student inference simultaneously on the same GPU.
- Loss Function: The distillation loss combined Kullback-Leibler Divergence (KLD) between student and teacher outputs with cross-entropy against the ground truth.
- Objectives: The optimization targeted minimizing model size and inference energy (in Joules) while maximizing effectiveness (measured by ROUGE-L).

Key Contributions

Empirical Study of FLOPs vs. Energy: The paper provides empirical evidence that FLOPs are not a consistent proxy for energy consumption across different SE tasks.
Energy-Aware Extension of Morph: The authors propose and implement a modification to the Morph framework that utilizes measured energy and surrogate models as optimization objectives, rather than relying solely on FLOPs.
Application to Code Summarization: The study successfully applies Many-Objective Optimized Distillation to a complex generative task (Code Summarization) using CodeT5+, demonstrating significant reductions in model size and energy consumption with manageable accuracy trade-offs.

Results

RQ1: Correlation of FLOPs and Energy

Clone Detection: FLOPs showed no statistical correlation with actual energy consumption ( $p = 0.17$ ). Models with similar FLOPs exhibited vastly different energy usage, and models with higher FLOPs sometimes consumed less energy.
Vulnerability Prediction: FLOPs showed a statistically significant correlation with energy ( $p = 6.48 \times 10^{-7}$ ), though the relationship was not strictly linear or proportional.
Conclusion: FLOPs are not a reliable, task-agnostic proxy for energy. Other hyperparameters (e.g., Hidden Size, Attention Heads) often correlate more strongly with energy than the FLOPs metric itself.

RQ2: Substituting FLOPs with Surrogate Energy

Clone Detection: Optimizing for energy (via surrogates) instead of FLOPs resulted in a 39% reduction in median inference energy (from ~5,355 J to ~3,604 J) with a statistically significant effect size ( $\hat{A}_{12} = 0.925$ ). Interestingly, these energy-efficient models had higher FLOPs (21% increase), highlighting the disconnect between the two metrics.
Vulnerability Prediction: In this task where FLOPs correlated with energy, the energy-based optimization did not yield statistically significant improvements or degradations compared to FLOPs optimization.
Conclusion: Surrogate energy models are superior when FLOPs fail to reflect actual energy consumption, bridging the gap to find more efficient configurations without sacrificing accuracy.

RQ3: Application to Generation Tasks

Compression: The distillation process reduced the model size from 830 MB (Teacher) to 114 MB (Most Accurate Student), an 86% reduction.
Energy: Inference energy was reduced from 82,396 J to 13,600 J for the most accurate student (83% reduction) and down to 8,059 J for the most efficient student (90% reduction).
Accuracy: The ROUGE-L score dropped from 0.229 (Teacher) to 0.199 (Most Accurate Student, -13%) and 0.194 (Most Efficient Student, -15%). Manual inspection confirmed that the summaries remained coherent, though performance dipped on code with specialized nomenclature.
Conclusion: Many-objective distillation is effective for generative tasks, achieving substantial efficiency gains with modest accuracy trade-offs.

Significance and Claims

The paper claims that energy-aware knowledge distillation guided by direct energy surrogates (rather than FLOPs) can significantly improve the sustainability and deployability of LLMs for SE applications.

Reliability: The study demonstrates that relying on FLOPs can lead to suboptimal energy choices, particularly in tasks like Clone Detection where runtime factors (e.g., padding, sparsity optimizations) decouple theoretical operations from actual energy use.
Practicality: By using surrogate models, developers can identify configurations that minimize energy consumption directly, enabling the deployment of efficient models on consumer hardware (e.g., local IDEs) without the need for energy-intensive calls to remote cloud servers.
Generalizability: The extension to CodeT5+ and Code Summarization proves that these optimization techniques are not limited to simple classification tasks but can be adapted for complex generative workflows, provided that appropriate distillation strategies (like weight subcloning and online distillation) are employed.

The authors conclude that while FLOPs may be useful for solving optimization problems due to their ease of computation, actual energy measurements should be prioritized when selecting final student models, especially for systems with strict energy constraints.