Measurement of Generative AI Workload Power Profiles for Whole-Facility Data Center Infrastructure Planning
This paper presents a methodology and publicly available dataset that links high-resolution power measurements of generative AI workloads on NVIDIA H100 GPUs to whole-facility energy demand, enabling realistic infrastructure planning for data centers.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are planning a massive city. You know people are going to move in, and you need to build power plants, roads, and water systems to support them. But here's the problem: you don't know exactly how the new residents will behave. Will they all wake up at 6 AM to cook breakfast? Will they all turn on their air conditioners at the exact same time in July? Or will they work weird hours, like a vampire shift?
If you guess wrong, you might build a power plant that's too small (causing blackouts) or way too big (wasting billions of dollars).
This paper is about solving that guessing game for Data Centers—the giant warehouses full of computers that run Artificial Intelligence (AI). Specifically, it's about the new, super-powerful "Generative AI" (like the chatbots that write stories or make images).
Here is the breakdown of what the researchers did, using some everyday analogies:
1. The Problem: The "Black Box" Mystery
Data centers are like black boxes. Big tech companies (the "hyperscalers") guard their energy usage secrets like state secrets. We know AI is hungry for electricity, but we don't know how it eats.
- The Issue: We have old maps of how data centers use power, but they are like maps from 1990. They don't show the new "AI traffic."
- The Risk: If we build a new data center based on old maps, we might get hit by a "power surge" that the grid can't handle, or we might pay for cooling systems we don't actually need.
2. The Experiment: The "Power Meter" on the Muscle
The researchers went to their own high-tech lab (the National Laboratory of the Rockies) and hooked up super-fine power meters to their AI computers (specifically NVIDIA H100 GPUs, which are the "muscle" of AI).
They didn't just look at the total bill at the end of the month. They measured the power 10 times every second.
- The Analogy: Imagine watching a weightlifter. A normal measurement tells you, "He lifted 200 pounds." This study measured the heartbeat of the weightlifter. It saw that he breathes hard, pauses, lifts fast, then rests.
- What they found:
- Training (Teaching the AI): When the AI is learning, it's like a marathon runner. It runs hard, but it has to stop to catch its breath (synchronize with other computers). The power goes up and down in a rhythmic pattern.
- Inference (Using the AI): When you ask the AI a question, it's like a sprinter. It bursts into action, answers quickly, and then goes idle. But if you ask 1,000 questions at once, the sprinters get tired and the power usage changes shape.
3. The Tool: The "Digital Twin" Simulator
Once they had these detailed "heartbeat" measurements, they built a computer simulation called DIPLOEE. Think of this as a Flight Simulator for Data Centers.
- How it works: They took their real-world power data and fed it into a model that simulates a whole city of computers.
- The Scenarios: They ran two different "cities":
- The Colocation City (10 MW): A big warehouse where many different companies rent space to train their AI models. It's like a busy office park where everyone comes and goes at different times.
- The Inference City (1 MW): A smaller, faster facility right next to the users (like in a city) that answers questions in real-time. It's like a 24-hour emergency room.
4. The Surprising Results
The simulation revealed some counter-intuitive truths:
The "Full House" Myth: Even when the data center was "full" (100% of the computers were working), the total power usage only reached about 73% to 80% of the maximum power the building was designed to handle.
- The Metaphor: Imagine a stadium designed for 50,000 people. Even when every seat is filled, the crowd isn't screaming at the top of their lungs at the exact same second. They are whispering, cheering, and clapping at different times. The "noise" (power) never hits the theoretical maximum.
- Why it matters: We might be able to build smaller, cheaper power infrastructure than we thought because the computers rarely hit their absolute maximum power limit simultaneously.
The "Rush Hour" Effect:
- In the big warehouse (Colocation), power usage followed a "business day" pattern (high during the day, low at night).
- In the real-time city (Inference), the power usage was more stable, but if too many people asked questions at once, the system had to slow down to keep the "latency" (wait time) low. This created a safety valve that prevented the power grid from crashing.
5. Why This Matters to You
This paper is a blueprint for the future.
- For the Power Grid: It helps utility companies know exactly how much electricity to generate so they don't have brownouts when everyone asks their AI for help at 5 PM.
- For Builders: It stops them from overspending on massive cooling systems and transformers that will sit idle 20% of the time.
- For the Planet: By understanding exactly how much energy AI uses, we can build more efficient systems and reduce waste.
In a nutshell: The researchers took the mystery out of AI energy usage. They measured the "heartbeat" of AI computers and built a simulator to show us that while AI is a power-hungry beast, it's not as unpredictable as we feared. We can now plan our power cities with a much clearer map.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.