← Latest papers
💻 computer science

Energy-Efficient GPU Resource Management for Generative AI on HPC Systems: A Roadmap Towards Cross-Layer Co-Design Approaches

This paper reviews 46 studies on energy-efficient GPU management for Generative AI in HPC systems, identifies critical gaps in cross-layer coordination and dynamic workload support, and proposes the GenAI-XEnergy framework to unify telemetry, prediction, and control for sustainable, energy-proportional execution.

Original authors: Atha Shamas, M. Usman Ashraf

Published 2026-11-17✓ Author reviewed ⓘ
📖 5 min read🧠 Deep dive

Original authors: Atha Shamas, M. Usman Ashraf

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The machines that power the most advanced artificial intelligence today are not just smart; they are voracious. These systems, known as generative AI, create new text, images, and code by learning from massive amounts of data. To do this, they rely on specialized computer chips called GPUs, which act as the engine rooms for these calculations. However, running these engines requires a tremendous amount of electricity. The training of a single large model can consume as much energy as a small town uses in a year, and the daily operation of these systems continues to draw power at a staggering rate. As these models grow larger and more complex, the challenge shifts from simply making them faster to making them efficient enough to run without draining the planet's resources. The question facing engineers is no longer just about performance, but about how to manage the flow of energy through these massive computer systems so that every watt of electricity does useful work.

A team of researchers has taken a close look at how we currently manage these energy-hungry systems and found that our approach is broken into too many separate pieces. In a comprehensive review of forty-six recent studies, they discovered that experts are trying to solve the energy problem in isolation. Some researchers focus on the hardware itself, trying to slow down chips when they are idle. Others look at the software that runs the programs, trying to organize tasks better. Still others study the scheduling systems that decide which computer gets which job. While each of these efforts has shown some success on its own, the researchers argue that they are failing to work together. Because the different layers of the system do not talk to each other, the overall result is often wasted energy. A chip might be slowed down to save power, but the software might not know about it and keep sending work anyway, or a scheduler might assign a job to a computer that is already overheating because it lacks real-time data from the hardware.

The researchers identified that the nature of generative AI workloads makes this fragmentation especially dangerous. Unlike traditional computer tasks that run at a steady pace, these AI jobs are highly unpredictable. They can burst with intense activity for a few seconds and then go quiet, or they can demand massive amounts of memory in a way that traditional systems were not designed to handle. The review showed that current methods often treat these jobs as if they were static and predictable, leading to inefficient use of resources. For instance, many systems allocate a fixed amount of computing power to a job regardless of whether it actually needs it at that moment, leaving expensive hardware sitting idle while still drawing power. The study found that while there are many clever tricks to save energy at the hardware level or the software level, there is a distinct lack of strategies that connect the two.

To address this, the authors propose a new way of thinking called cross-layer co-design. Instead of fixing the hardware, the software, or the scheduler separately, they suggest building a system where all these parts work as a single, coordinated unit. They introduce a conceptual framework they call GenAI-XEnergy. This idea envisions a system that constantly watches the computer's energy use, predicts how much power a job will need before it even starts, and then instantly adjusts the hardware and software to match. Imagine a conductor leading an orchestra where every musician can hear the others and adjust their volume in real time to create a perfect harmony; in this case, the "music" is the flow of electricity, and the "musicians" are the computer chips, the operating system, and the job scheduler. By linking these layers, the system could turn off parts of a chip the moment they are not needed, or move a heavy task to a different computer that has cheaper, cleaner energy available.

The review highlights that while we have the individual tools to make these adjustments, we lack the central brain to coordinate them. The researchers point out that current systems often rely on old rules that were written for different types of computer work. They argue that to truly solve the energy crisis in artificial intelligence, we need to move away from these static rules and toward a dynamic system that learns and adapts. The proposed framework suggests using real-time data to make decisions, ensuring that energy is only used when it is absolutely necessary. This approach does not require inventing entirely new types of computers, but rather using the capabilities we already have in a smarter, more connected way. The findings suggest that by bringing these separate layers together, we could significantly reduce the energy footprint of artificial intelligence without sacrificing its speed or ability to learn.

The path forward, according to the paper, involves building these connections between the hardware and the software. The researchers suggest that future systems should be able to sense their own energy consumption and adjust their behavior accordingly, a concept known as telemetry-driven management. This means the computer would know exactly how much power it is using at any given second and could make immediate changes to save energy. They also emphasize the need to consider the source of the electricity, suggesting that systems should try to run heavy tasks when the power grid is using cleaner energy sources. While the paper acknowledges that building such a unified system is a complex challenge that has not yet been fully realized, it provides a clear roadmap for how to get there. The ultimate goal is to create an artificial intelligence infrastructure that is not only powerful but also sustainable, ensuring that the technology can continue to grow without overwhelming the energy systems that support it.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →