💻 computer science

Green Optimization: Energy-aware Design of Metaheuristics by Using Machine Learning Surrogates to Cope with Real Problems

This paper investigates the integration of machine learning surrogates into metaheuristics to create energy-aware optimization algorithms, demonstrating that while pre-trained surrogates can drastically reduce energy consumption, execution time, and memory usage, their adoption requires a nuanced trade-off analysis between efficiency gains and potential impacts on solution accuracy.

Original authors: Tomohiro Harada, Enrique Alba, Gabriel Luque

Published 2026-03-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Tomohiro Harada, Enrique Alba, Gabriel Luque

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the absolute best route for a city's traffic lights to prevent jams. You have a super-smart computer simulation (let's call it the "Traffic Simulator") that can tell you exactly how good a specific traffic light plan is.

However, there's a catch: The Traffic Simulator is like a giant, slow-moving elephant. Every time you ask it to test a plan, it takes a long time, eats up a massive amount of electricity, and uses a lot of memory. If you try to test millions of plans to find the perfect one, you'll run out of time, money, and energy before you even get close to a solution.

This paper is about teaching the computer to use a "Smart Assistant" (a Machine Learning model) to do the heavy lifting, but with a twist: the authors wanted to know if this assistant actually saves energy or if it just moves the energy cost to a different part of the process.

Here is the breakdown of their findings using simple analogies:

1. The Problem: The Expensive Elephant

In the real world, testing a traffic light plan involves running a complex simulation.

The Cost: It's like asking a master chef to cook a full 5-course meal just to taste one ingredient. It takes forever (time), burns a lot of gas (energy), and fills up the kitchen (memory).
The Goal: The researchers wanted to optimize traffic lights in cities like Málaga, Stockholm, and Paris without burning out the power grid or waiting years for results.

2. The Solution: The "Smart Assistant" (Surrogate)

Instead of asking the "Elephant" (the real simulator) to check every single idea, they trained a "Smart Assistant" (a Neural Network).

How it works: The Assistant watches the Elephant cook a few meals, learns the patterns, and then starts guessing the taste of new dishes.
The Benefit: The Assistant is incredibly fast. It's like a microwave compared to the slow oven. It can guess the result in milliseconds instead of minutes.

3. The Big Discovery: It's Not Just About Speed

Usually, people think, "If the Assistant is faster, it must be better." But this paper asked a deeper question: "Does the Assistant actually save energy, or does it just shift the cost?"

They found three surprising things:

A. The "Training" Cost vs. The "Usage" Cost

The Analogy: Imagine you hire a tutor to teach you math.
- Scenario 1 (Small Dataset): You hire a tutor for 1 hour. They give you a quick cheat sheet. It's cheap to hire them, but their cheat sheet is full of mistakes. You have to keep asking the "Elephant" to double-check your work, which is expensive.
- Scenario 2 (Large Dataset): You hire a tutor for 100 hours. This costs a lot of energy upfront (training). But now, the tutor is a genius. Their cheat sheet is perfect. You rarely need to ask the Elephant for help.
The Result: Even though training the "Genius Tutor" (Large Dataset) took a lot of energy at the start, it saved massive amounts of energy in the long run because the Assistant became so efficient at guessing that it barely needed to "think" hard anymore. In fact, the more data they fed the Assistant, the "sparser" (simpler) its brain became, making it even more energy-efficient to use later.

B. The Memory Trap

The Analogy: The "Elephant" (the real simulator) is like a minimalist hiker carrying a tiny backpack. The "Smart Assistant" is like a librarian who needs a huge library to store all the books (data) it learned from.
The Result: While the Assistant saved huge amounts of Time (98% faster) and Energy (up to 98% less), it gobbled up a lot more Memory (RAM). If you have a computer with limited memory, the Assistant might crash your system even if it saves you electricity.

C. Static vs. Dynamic (The "Set It and Forget It" vs. "Keep Learning")

Static (Pre-trained): You train the Assistant once and lock the door. It's fast, but if the city traffic changes (e.g., a new road opens), the Assistant might get confused because it's stuck in the past.
Dynamic (Retraining): You let the Assistant keep learning while it works. It updates its cheat sheet constantly.
The Winner: For the traffic light problem, the Dynamic approach worked best for finding the best solution, but the Static approach was the most energy-efficient if you just needed a "good enough" answer quickly.

4. The "Green" Verdict

The paper concludes that we need to stop looking at algorithms just by how fast they are. We need to look at them like a hybrid car:

Old Way: Just drive the gas car (the real simulator). It works, but it pollutes and costs a fortune.
New Way: Use the electric motor (the AI Assistant).
- The Catch: You have to charge the battery (train the model) first.
- The Payoff: If you drive enough miles (solve enough problems), the electric motor saves you a fortune in gas.

Summary for the Everyday Person

This research tells us that using AI to speed up complex problems is a great idea, but we have to be smart about how we use it.

Don't just train a little: If you want to save energy in the long run, train your AI model on a lot of data first. It makes the model "simpler" and more efficient later.
Watch your memory: These smart assistants are hungry for computer memory. Make sure your computer has enough RAM before you start.
Green Computing: By using these smart assistants, we can solve real-world problems (like traffic jams) using a fraction of the electricity, making our technology more "green" and sustainable.

In short: Teach the AI well once, and it will save you energy forever. But don't forget to check if your computer has enough room in its "brain" to hold the lesson!

1. Problem Statement

Real-world optimization problems, such as traffic light scheduling in smart cities, often involve computationally expensive objective functions (e.g., detailed traffic simulations using SUMO). While Surrogate-Assisted Evolutionary Algorithms (SAEAs) are commonly used to reduce computational time by approximating these expensive functions with Machine Learning (ML) models (specifically Neural Networks), existing research focuses almost exclusively on runtime reduction.

There is a significant gap in understanding the energy efficiency and memory footprint of these surrogate-assisted approaches. Specifically, the trade-offs between:

The energy cost of training the surrogate.
The energy cost of using the surrogate during the search.
The impact of training strategies (static pre-training vs. dynamic retraining).
The impact of dataset size on inference energy.

The authors argue that "efficiency" must be redefined to include energy consumption and memory usage, not just execution time, to align with Green AI principles.

2. Methodology

2.1. Real-World Application

The study uses Traffic Light Scheduling as a case study. The goal is to optimize the duration of traffic light phases across multiple intersections to minimize total travel time, waiting time, and queue lengths while maximizing vehicle throughput.

Instances: Three real-world datasets from Málaga (Spain), Stockholm (Sweden), and Paris (France), varying in network size (56–75 intersections) and simulation complexity.
Evaluator: The SUMO (Simulation of Urban MObility) simulator, which is computationally intensive (taking seconds to minutes per evaluation).

2.2. Algorithms and Configurations

The authors integrated Neural Network (NN) surrogates into two metaheuristics: Particle Swarm Optimization (PSO) and Genetic Algorithms (GA). They tested four distinct configurations for each:

Baseline: No surrogate (direct evaluation via SUMO).
Pre-trained (Small Dataset): Surrogate trained once on a small initial dataset ( $N_t=100$ ) and used statically.
Pre-trained (Large Dataset): Surrogate trained once on a large dataset ( $N_t=8192$ ) and used statically.
Iterative Retraining: Surrogate is retrained periodically during the search using new data points (Small or Large initial datasets).

2.3. Experimental Setup & Metrics

Hardware: Intel Xeon CPU, 16GB RAM, Ubuntu 22.04.
Measurement Tools:
- Energy: pyRAPL (Intel RAPL technology) to measure CPU and DRAM energy.
- Memory: Python tracemalloc and time command.
- Time: Execution time.
Statistical Analysis: Log-normal distribution fitting for cost modeling and Kruskal-Wallis tests for significance.

3. Key Contributions

Holistic Energy-Aware Framework: The paper proposes a comprehensive evaluation framework that integrates energy consumption, execution time, and memory usage into the performance assessment of metaheuristics.
Surrogate Training Strategy Analysis: It rigorously compares static pre-training vs. dynamic retraining, revealing that the "best" strategy depends on the specific constraints (time vs. energy vs. accuracy).
Dataset Size vs. Inference Energy Paradox: A novel finding that larger training datasets lead to lower energy consumption during inference. This is attributed to increased sparsity in the trained Neural Networks (more ReLU neurons outputting zero) when trained on larger datasets, reducing computational complexity during prediction.
Algorithm-Specific Insights:
- PSO benefits significantly from retraining due to its ability to maintain population diversity.
- GA tends to converge prematurely; retraining offers limited benefit if the search space is exhausted before the surrogate is updated.

4. Key Results

4.1. Energy and Time Reductions

Massive Savings: Using a state-of-the-art pre-trained surrogate reduced energy consumption by up to 98%, execution time by ~98%, and memory usage by ~99% compared to the baseline (direct SUMO evaluation).
Trade-offs:
- Small Pre-training (SAPSO-ps/SAGA-ps): Fastest execution and highest energy reduction (>95%), but suffers from poor solution quality (stagnation) due to low surrogate accuracy.
- Large Pre-training (SAPSO-pl/SAGA-pl): Good balance of speed and accuracy, reducing energy by ~70%.
- Retraining (SAPSO-rs/SAGA-rs): Achieves high accuracy and good energy savings (~85% reduction), but incurs higher memory overhead due to data archiving and repeated model construction.

4.2. The "Break-Even" Point

The study calculated the break-even point for training large datasets. While training on 8192 samples costs significantly more energy initially (~2356 J more than training on 128 samples), the reduced energy per inference (due to sparsity) means that after approximately 2,874 surrogate uses, the large-dataset approach becomes more energy-efficient overall than the small-dataset approach.

4.3. Memory Usage

Surrogate-assisted algorithms significantly increase memory usage (up to 300-500x the baseline) due to the storage of the NN model, training data, and solution archives.
Retraining strategies consume the most memory, while pre-training consumes less.
Memory usage is largely dictated by problem dimensionality and NN architecture, not the number of evaluations.

4.4. Accuracy vs. Efficiency

PSO: Retraining (SAPSO-rs) yielded the best balance, maintaining solution quality close to the baseline while drastically cutting energy.
GA: Pre-training on large datasets (SAGA-pl) performed best. Retraining (SAGA-rs) often led to stagnation because GA populations converged too quickly to generate diverse data for the surrogate to learn from.

5. Significance and Conclusion

This paper fundamentally shifts the perspective on surrogate-assisted optimization from a purely "time-saving" metric to a "Green Computing" metric.

Redefining Efficiency: It demonstrates that minimizing runtime does not automatically minimize energy. In some cases, the energy cost of training a complex model can outweigh the savings, but in others (large datasets), the inference efficiency gains are substantial.
Practical Guidelines:
- For strict energy constraints and long-running tasks, large-dataset pre-training or retraining is preferred to leverage inference sparsity.
- For short tasks or where memory is limited, small-dataset pre-training is viable but risks solution quality.
- PSO is generally more robust to surrogate retraining than GA in these contexts due to diversity maintenance.
Future Impact: The findings support the development of "energy-aware" AI systems where algorithm selection and hyperparameter tuning are driven by energy profiles, not just convergence speed. The discovery that larger training sets induce sparsity (and thus lower inference energy) is a counter-intuitive but critical insight for sustainable AI design.