💻 computer science

LLM-Powered Predictive Decision-Making for Sustainable Data Center Operations

This paper presents an LLM-powered predictive scheduling system that forecasts execution time and energy consumption from source code to optimize GPU resource allocation, achieving a 32% reduction in energy usage and a 30% decrease in waiting time for sustainable data center operations.

Original authors: Hanzhao Wang, Jingxuan Wu, Yumeng Li, Yu Pan, Guanting Chen

Published 2026-08-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Hanzhao Wang, Jingxuan Wu, Yumeng Li, Yu Pan, Guanting Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The modern world runs on a quiet, invisible engine: the data center. These vast warehouses of computer servers power everything from streaming movies to the artificial intelligence tools that are reshaping how we work and think. However, this digital convenience comes with a heavy physical cost. Training the most advanced artificial intelligence models requires immense amounts of computing power, which in turn demands staggering amounts of electricity and water for cooling. As the demand for these intelligent systems grows, so does the strain on our energy grids and the environment. The challenge for engineers is no longer just about making these systems faster; it is about making them sustainable, ensuring that the digital future does not consume the physical resources needed to support it.

To address this growing tension, a team of researchers has developed a new way to manage the flow of work inside these data centers. Instead of guessing how long a computer task will take or how much power it will burn, they created a system that uses a sophisticated language model to read the computer code itself and predict the outcome before the work even begins. This approach treats the source code—the instructions written by human programmers—as a story that the computer can understand. By analyzing the structure and logic of the code, the system can estimate the time required to finish a job and the energy it will consume, all without needing to run the task first. This predictive ability allows the data center to make smarter choices about which powerful computer chips to assign to which jobs, balancing the load to save energy and reduce wait times.

The researchers built a prototype system that operates in two distinct stages. First, a pre-trained artificial intelligence model, designed to understand human language and code, reads the instructions submitted by a user. It translates the complex code into a mathematical representation that captures the essence of the task. This model does not rely on rigid, hand-written rules or specific features that engineers have to manually define for every new type of program. Instead, it uses its broad training to recognize patterns and understand the context of the code, much like a skilled reader can grasp the plot of a story without needing a summary. Once the system understands the task, it predicts two critical numbers: how long the job will take to complete and how much electricity it will use. These predictions are made in less than a second, fast enough to be used in real-time decision-making.

With these predictions in hand, the second stage of the system takes over: a scheduling algorithm that decides how to distribute the available computer resources. In a typical data center, tasks arrive in a continuous stream, and the manager must decide which computer chip to assign to each one. If the manager assigns a task to a chip that is too powerful, energy is wasted. If they assign it to a chip that is too weak or if they wait too long to assign it, the task sits in a queue, delaying the results. The new system uses the predictions from the first stage to calculate the best possible assignment for every incoming task. It considers the current availability of different types of computer chips, the estimated energy cost, and the time the task will take. The goal is to minimize the total energy used while keeping the waiting time for all tasks as short as possible.

To test if this idea works in the real world, the researchers collaborated with a data center to run a simulation using two months of actual operational data. They compared their new, predictive system against the standard method used by the facility, which relies on simpler, rule-based decisions. The results were significant. The new system reduced the total energy consumption of the data center by 32 percent. At the same time, it cut the time tasks spent waiting in line by 30 percent. These improvements were achieved not by changing the hardware or the physical infrastructure, but by changing how the data center thinks about the work it is doing. The system proved that by understanding the code before execution, the facility could operate with much greater efficiency.

One of the most striking aspects of this work is its flexibility. Traditional methods for predicting energy use often require a separate, custom-built model for every different type of computer program. If a new kind of task appeared, the old system would struggle to predict its needs. The new approach, however, uses a single, unified model that can handle a wide variety of tasks, from training complex image recognition systems to running language models. This is because the underlying artificial intelligence model has learned to understand the fundamental structure of code, rather than just memorizing specific examples. The researchers found that even when the code was written in a slightly different style or by a different person, the system could still make accurate predictions. To further improve this, they added a secondary step where the system can rewrite unfamiliar code into a style that matches its training data, ensuring that the predictions remain accurate even for tasks the system has never seen before.

The study also highlights a shift in how we might think about resource management in the digital age. For years, data centers have operated on estimates provided by users or on historical averages that often miss the mark. This new method replaces guesswork with informed prediction. It suggests that the key to a more sustainable digital future lies in better decision-making tools that can adapt to the complexity of modern computing. While the current system focuses on time and energy, the researchers note that the same framework could be extended to predict other environmental impacts, such as water usage for cooling or carbon emissions, provided the data center tracks those metrics.

The success of this prototype demonstrates that artificial intelligence can be used not just to create new technologies, but to optimize the infrastructure that supports them. By reading the code and predicting the cost of the work, the system turns the data center into a more responsive and efficient organism. The 32 percent reduction in energy and 30 percent reduction in waiting time are not just numbers; they represent a tangible step toward reducing the environmental footprint of the AI revolution. As the demand for artificial intelligence continues to surge, methods like this offer a practical path forward, proving that we can build a smarter, more sustainable digital world without sacrificing performance. The work shows that with the right tools, the invisible engine of the internet can run cleaner and more efficiently than ever before.

Technical Summary: LLM-Powered Predictive Decision-Making for Sustainable Data Center Operations

1. Problem Formulation

The rapid expansion of AI-driven workloads, particularly Large Language Models (LLMs), has intensified the energy and resource consumption of data centers. Current operational pipelines in data centers often rely on heuristic algorithms that allocate resources (e.g., GPUs) based on user-provided estimates of task duration and resource needs. However, accurately predicting execution time and energy consumption is inherently difficult due to the complexity of modern computing systems (including bandwidth, memory, software stacks, and hardware variations). Consequently, neither users nor data centers possess precise knowledge of optimal resource requirements, leading to inefficiencies, resource wastage, and increased environmental footprints.

The authors formulate a two-part problem:

Prediction: Given a user-submitted source code $x$ and a potential GPU configuration $z$ , predict the execution time $t$ and energy consumption $e$ . These are modeled as unknown functions $t = f(x, z)$ and $e = g(x, z)$ that are specific to the data center's infrastructure.
Decision-Making: Develop a policy $\pi$ to sequentially allocate GPU resources to arriving tasks to minimize a weighted sum of execution time, waiting time, and energy cost, subject to limited GPU inventory constraints. This problem is complicated by the presence of a waiting queue and the need to balance multiple objectives (multi-purpose optimization), for which no analytical solution exists.

2. Methodology

A. LLM-Based Predictive Model

The core of the proposed system is a unified, end-to-end predictive model that utilizes a pre-trained Large Language Model (LLM) to process source code.

Architecture: The model consists of two stages:
1. Representation Extraction: A pre-trained LLM (denoted as $l_{\theta_1}$ ) takes the source code $x$ as input and outputs a $d$ -dimensional vector representation. The LLM parameters $\theta_1$ are kept fixed.
2. Probing: A lightweight supervised model (a "probe," denoted as $h_{\theta_2}$ or $q_{\theta_2}$ ) takes the extracted representation and the GPU configuration $z$ as input to predict the target metric (time or energy).
Training: The system is trained on a dataset of source code paired with actual execution results (time and energy) across various GPU configurations. Notably, the model requires only a small dataset (approximately 500 source code examples) because the pre-trained LLM already possesses extensive contextual understanding of code.
Advantages over Traditional Methods:
- Generalization: Unlike previous methods that rely on handcrafted features (e.g., layer depth, number of passes) specific to certain model types (e.g., CNN-only), the LLM approach generalizes across diverse task types and composite tasks (e.g., CNN followed by LSTM).
- Data Efficiency: The pre-trained LLM acts as a powerful feature extractor, reducing the need for massive training datasets.
- Flexibility: The same representation can be used for different prediction targets (time, energy, and potentially water/carbon metrics) by simply training different probes.

B. Sequential Decision-Making Algorithms

To optimize resource allocation, the authors propose two data-driven algorithms that utilize the predictions from the LLM model:

Greedy Algorithm: Assigns tasks based on a first-come-first-served rule, selecting the GPU type that minimizes the estimated objective value ( $\alpha \hat{t} + \beta w + \gamma \hat{e}$ ) for the current task.
Value-Based Algorithm: Inspired by the multiple knapsack problem, this algorithm considers the entire waiting queue. It assigns tasks based on their "value" in reducing overall waiting time and energy costs, rather than strictly following arrival order.

C. Handling Out-of-Distribution (OOD) Tasks

The authors identify a potential inconsistency where functionally equivalent code written by different engineers might yield different LLM representations, leading to prediction errors. To mitigate this, they propose an extended pipeline:

Align-LLM: If a new task's representation deviates significantly from the training clusters, an auxiliary "Align-LLM" rewrites the source code to match the style of similar tasks in the local codebase. This aligns the input representation with the training distribution, improving prediction accuracy without altering the code's functionality.

3. Key Contributions

Methodological Contributions

LLM-based Versatile Predictive Model: The first comprehensive, end-to-end solution that takes raw source code as input to estimate execution time and energy consumption. It eliminates the need for task-specific handcrafted features.
Fully Automated Predictive Scheduling System: A unified framework capable of handling any user-submitted task type and providing immediate estimations, overcoming the limitations of traditional, rigid prediction models.
Sequential Decision-Making: A data-driven algorithm that addresses the complex, multi-objective resource allocation problem with waiting queues, a scenario where standard analytical solutions fail.

Application-Level Contributions

Practical Deployment: The pipeline executes within one second, enabling real-time, cost-effective scheduling. It is effective even with limited training data.
Multi-Purpose Sustainability: The framework is designed to be extensible, capable of predicting various sustainability metrics (e.g., carbon emissions, water usage) if the necessary data is available.

4. Experimental Results

The authors validated their approach through a collaboration with a data center, utilizing two months of operational data (July 1, 2024 – September 1, 2024).

Performance Metrics: Compared to the data center's baseline "Simple Rule" (first-come-first-served with preference for the most powerful GPU), the proposed Value-based algorithm achieved:
- 32% reduction in total energy consumption (TEC).
- 30% reduction in total waiting time (TWT).
- 24% reduction in the number of delayed tasks.
Comparison: The Value-based algorithm significantly outperformed the Greedy algorithm by considering the global state of the waiting queue rather than just the immediate task.
Robustness: The extended pipeline with the Align-LLM successfully mitigated prediction errors caused by coding style variations, bringing out-of-distribution tasks closer to the training distribution and improving prediction accuracy.

5. Significance and Claims

The paper claims that this work offers a practical, scalable solution for modernizing data center operations to meet sustainability goals. By leveraging the contextual understanding and generalization capabilities of LLMs, the proposed system overcomes the rigidity of traditional heuristic and physics-based prediction methods.

The authors emphasize that their approach:

Advances Infrastructure: It provides a pathway to more efficient and environmentally friendly AI infrastructure.
Bridges Theory and Practice: It demonstrates that LLMs can be effectively integrated into real-time operational decision-making loops with minimal data requirements.
Scalability: The unified nature of the model allows it to adapt to diverse task types and future sustainability metrics without requiring a complete redesign of the prediction architecture.

The work concludes that integrating LLM-based predictive modeling with sequential decision-making is a viable strategy to significantly reduce the environmental impact of AI-driven workloads while improving operational efficiency.