🤖 machine learning

Optimising for Energy Efficiency and Performance in Machine Learning

This paper introduces ECOpt, a hyperparameter tuning tool that optimizes the trade-off between energy efficiency and model performance by providing actionable feedback and an interpretable Pareto frontier, ultimately enabling the discovery of more sustainable and accurate machine learning models.

Original authors: Emile Dos Santos Ferreira, Andrei Paleyes, Neil D. Lawrence

Published 2026-01-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Emile Dos Santos Ferreira, Andrei Paleyes, Neil D. Lawrence

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a bakery. For years, you've only cared about one thing: how delicious your cakes taste (performance). You've been making them bigger and bigger, using more flour and sugar (parameters), assuming that bigger automatically means better.

But recently, you've realized something scary: your bakery is burning through electricity at an alarming rate, and the carbon footprint of your ovens is hurting the planet. You also realize that while you spent a fortune baking the cakes (training), you actually spend even more money just delivering them to customers every day (inference).

This paper introduces a new tool called ECOpt (Energy Consumption Optimiser) to help bakers (machine learning engineers) solve this problem. Here is how it works, using simple analogies:

1. The Problem: The "Bigger is Better" Myth

For a long time, the tech world thought that if you just counted the number of ingredients (parameters) or the number of mixing steps (math operations called FLOPs), you could guess how much energy your cake would cost to bake.

The authors tested this and found it was like guessing a car's gas mileage just by counting the number of bolts in the engine. It didn't work.

They found that changing how you move the ingredients (like changing the "stride" in a recipe) could save a lot of energy without changing the number of ingredients at all.
They also found that just because a machine is powerful (like a giant industrial oven) doesn't mean it's efficient. If you put a tiny cake in a giant oven, the oven wastes a huge amount of energy just staying hot, even if the cake is small.

2. The Solution: The "Smart Menu" (ECOpt)

Instead of guessing, the authors built a tool called ECOpt. Think of this as a smart menu planner for your bakery.

The Balancing Act: Usually, you have to choose: "Do I want the tastiest cake (high performance) or the cheapest cake to make (high energy efficiency)?"
The Pareto Frontier: ECOpt draws a map called a "Pareto Frontier." Imagine a graph where the X-axis is "Taste" and the Y-axis is "Energy Cost." The tool finds the "sweet spot" curve. Every point on this curve represents a cake where you can't get more taste without paying more energy, and you can't save more energy without losing some taste.
The Result: It gives the baker a list of options. "Here is a cake that is 90% as tasty but uses 50% less energy. Here is another that is 95% as tasty but uses 20% less energy." You get to choose what trade-off you are comfortable with.

3. Key Discoveries from the Experiments

The authors tested this tool on real "cakes" (AI models) and found some surprising things:

Hardware Consistency: They tested the same model on different machines (a laptop, a desktop, a supercomputer). Surprisingly, the energy efficiency (how much "work" you get per drop of electricity) was surprisingly similar across all these machines. This means we should start publishing energy numbers for our models, just like we publish calorie counts on food, because the numbers are reliable enough to compare.
The "Empty Oven" Effect: They found that if you run a small model on a massive, powerful computer, the computer wastes energy because it's not fully utilized. It's like using a jet engine to power a bicycle. They used ECOpt to adjust the "batch size" (how many cakes you bake at once) to fill the oven properly. This single change made the process 38 times more efficient.
The Payback: The tool itself uses a tiny bit of energy to run the optimization. However, the authors calculated that after generating just 4,802 tokens (pieces of text), the energy saved by using the optimized settings paid back the energy cost of running the optimization tool. It's a quick win for the environment.

4. Finding New Recipes (Neural Architecture Search)

Finally, they used ECOpt to invent new cake recipes from scratch (a process called Neural Architecture Search) for a specific task (classifying images of animals).

They didn't just look for the "best" cake; they looked for the best balance.
They found seven new recipes that were better than the current "State of the Art" (the best known recipes) when you considered both taste and energy cost together.
They discovered that the "best" recipe for speed (latency) is different from the "best" recipe for energy. A wide, shallow cake might be fast, but a deeper, narrower one might use less power. ECOpt helps you find the right shape for your specific needs.

The Bottom Line

The paper argues that we can't just keep making bigger, hungrier AI models. We need to be smart about how we bake them. ECOpt is a tool that helps engineers stop guessing and start making informed choices, ensuring their models are not only smart but also energy-efficient and environmentally friendly. It turns the vague idea of "being green" into a concrete, measurable part of the design process.

Technical Summary: Optimising for Energy Efficiency and Performance in Machine Learning

Problem Statement
The exponential growth in the computational complexity of machine learning (ML) models has led to a significant increase in energy consumption and environmental impact. While recent regulations, such as the EU AI Act, mandate the publication of energy usage, current ML research predominantly focuses on model performance, often neglecting energy metrics. Existing literature frequently relies on parameter counts and floating-point operation (FLOP) counts as proxies for energy consumption, yet these metrics are often uncorrelated with actual energy usage. Furthermore, prior research on the environmental impact of ML largely ignores inference costs, despite inference often accounting for the majority of ML compute costs (e.g., up to 90% for AWS). Current tools for estimating carbon footprints often require high code coupling and fail to provide actionable feedback for improving efficiency, while hyperparameter tuners generally do not account for environmental impact or inference costs.

Methodology
To address these gaps, the authors developed ECOpt (Energy Consumption Optimiser), a Python framework designed to simultaneously optimise model hyperparameters for both energy efficiency and performance.

Optimization Formulation: The problem is framed as a multi-objective optimisation task. The system searches a hyperparameter space $S$ to find the Pareto frontier between a performance metric ( $f_p$ , e.g., accuracy) and an energy efficiency metric ( $f_e$ ).
Energy Measurement: ECOpt utilises software-based energy meters, specifically CodeCarbon, which aggregates power usage data from device drivers (e.g., NVIDIA nvidia-smi for GPUs and RAPL for CPUs). The framework focuses on compute energy usage, quantifying efficiency as "samples per Joule" (or tokens per Joule for text generation). It accounts for the power draw of all active components (CPU, GPU, RAM) rather than just the accelerator.
Optimization Algorithm: The framework employs Multi-Objective Bayesian Optimisation (MOBO) using the Expected Hypervolume Improvement (EHVI) acquisition function (specifically parallel noisy EHVI, or $q$ NEHVI). This approach is chosen for its sample efficiency, allowing the discovery of the trade-off curve (Pareto frontier) with fewer evaluations than random or grid search.
Workflow: The process involves splitting a dataset into training and evaluation sets. For each iteration, ECOpt constructs and trains a model with sampled hyperparameters, performs inference on the evaluation set, measures the objectives, and updates the surrogate models (Gaussian Processes) to guide the next search step.

Key Contributions

Formulation of Trade-off: The authors formalise the search for the performance-energy trade-off in ML as a multi-objective optimisation problem.
Automated Procedure: They describe a fully automated procedure based on MOBO to discover an empirical Pareto frontier characterising this trade-off.
Open-Source Framework: ECOpt is released as a reusable Python package, including experiment scripts for reproducibility.
Empirical Findings: The work provides extensive experimental evaluations demonstrating that:
- Parameter counts and FLOPs are unreliable proxies for energy consumption.
- Transformer models for text generation exhibit relatively consistent energy efficiency across diverse hardware configurations.
- An energy scaling law exists for Transformers, where energy efficiency decreases log-linearly with parameter count.
- Optimising for energy efficiency can yield a net positive environmental impact by recovering the energy spent on optimisation quickly.

Experimental Results
The authors conducted experiments across various hardware (VMs, laptops, desktops, servers, and HPC nodes) and model types (CNNs, Transformers).

Metric Validity: In experiments scaling neural network layers and adjusting CNN strides, the authors observed that parameter counts remained constant while energy consumption and FLOPs varied significantly. This confirms that parameter count is an unsuitable proxy for energy efficiency. Similarly, FLOPs showed weak correlation with actual energy usage in CNNs.
Hardware Consistency: Testing Transformer models (GPT-2, Qwen3, Gemma 3, Llama 3.1) across five different machines revealed that while absolute energy efficiency varied by model size, the efficiency of a specific model remained relatively consistent across different hardware (standard deviation of 0.2372 across machines vs. 0.4608 across models). This suggests model energy metrics are valuable to publish.
GPU Utilisation: In an experiment optimising the batch size for Gemma 3 on a server, ECOpt identified an optimal batch size of 831 (yielding 2.67 tokens/J), which was lower than the maximum VRAM-fitting size. This configuration achieved a 38-fold increase in energy efficiency compared to a default batch size of 1. The authors calculated that the energy spent on this optimisation would be recovered after generating just 4,802 tokens.
Neural Architecture Search (NAS): Applying ECOpt to a CNN NAS task on CIFAR-10, the framework identified seven Pareto-optimal architectures. The best model achieved 76.09% accuracy with 20.12 samples/J. This outperformed a random search baseline (71.17% accuracy) and a manual tuning baseline (70.07% accuracy) in both accuracy and time efficiency (61 minutes vs. three days). Notably, the authors found that while their models were not State-of-the-Art (SOTA) in accuracy alone, they offered superior accuracy-per-energy profiles compared to existing SOTA models when considering energy efficiency.

Significance and Claims
The paper claims that by optimising hyperparameters for both energy efficiency and performance, practitioners can reduce the energy cost of ML without sacrificing inference quality. The authors argue that the current reliance on parameter counts and FLOPs obscures the true environmental cost of models. By providing a tool that quantifies the trade-off via an interpretable Pareto frontier, ECOpt enables informed decision-making regarding deployment costs and environmental impact.

The authors modestly note that their findings on CIFAR-10 improving upon SOTA are largely due to the scarcity of published energy metrics in the field, rather than a fundamental breakthrough in model architecture. They advocate for the broader adoption of energy metrics in model publication to foster competition and awareness. The work also highlights that optimising for energy efficiency differs from optimising for latency; while latency often favours wide networks to maximise hardware utilisation, energy efficiency may require balancing width and depth to minimise power draw, particularly when hardware is not fully saturated.

1. The Problem: The "Bigger is Better" Myth

2. The Solution: The "Smart Menu" (ECOpt)

3. Key Discoveries from the Experiments

4. Finding New Recipes (Neural Architecture Search)

The Bottom Line

More like this