⚡ electrical engineering

Data Center Life Cycle Co-Design Optimization

This paper presents a co-design optimization framework that integrates operational energy, embodied carbon, and reliability models to determine the optimal number of cooling subloops for liquid-cooled supercomputers, demonstrating that reducing the Frontier supercomputer's configuration from four to two subloops significantly lowers life-cycle costs and carbon emissions while maintaining operational reliability.

Original authors: Shrenik Jadhav, Vidhyashree Nagaraju, Zheng Liu

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shrenik Jadhav, Vidhyashree Nagaraju, Zheng Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a supercomputer like Frontier is a giant, high-performance race car engine. To keep it from melting, it needs a massive cooling system that pumps liquid through pipes, much like a car's radiator system.

For years, engineers built these cooling systems based on tradition and supplier habits, not on a deep mathematical calculation of what was actually best. They usually built four separate loops (like four separate radiator circuits) to pump the coolant.

This paper asks a simple but powerful question: "Is four loops actually the best number, or could we do better with fewer?"

The authors didn't just look at how much electricity the cooling uses while running (operational energy). They looked at the entire life of the machine, from the moment the steel and copper are mined to build the pipes, all the way to the day the machine is turned off. They call this a "Life Cycle Co-Design."

Here is the breakdown of their findings using simple analogies:

1. The "Embodied Carbon" vs. "Running Cost" Battle

Think of building a cooling system like building a house.

Embodied Carbon: This is the "carbon footprint" of the bricks, wood, and nails used to build the house. In this study, it's the carbon emitted to mine the steel and copper for the pipes and pumps.
Operational Cost: This is the electricity bill you pay every month to keep the AC running.

The Surprise: The authors found that for a supercomputer the size of Frontier, the "building cost" (embodied carbon) matters much more than the "electricity bill" (operational cost).

When you add more loops (going from 2 to 4), you need more pipes, more pumps, and more valves. This adds a lot of "embodied carbon" right at the start. While having more loops might save a tiny bit of electricity by giving the pumps more flexibility, the study found that a smart computer algorithm (the "optimizer") can already squeeze out almost all the efficiency with just two loops.

So, adding loops 3 and 4 is like buying a bigger, heavier truck to deliver the same amount of groceries. You pay more for the truck (embodied carbon) but don't save enough on gas (operational energy) to make it worth it.

2. The "Goldilocks" Result

The researchers tested every possible way to split the 25 cooling units into 2, 3, 4, 5, or 6 loops.

The Winner: Two loops (specifically, one loop with 14 units and one with 11).
The Loser (Current Design): The actual Frontier supercomputer was built with four loops.

The Savings: By switching from the built-in 4 loops to the optimal 2 loops, the facility would save:

50.2 tonnes of CO2 (about the weight of 8 elephants).
$100,000 over 7 years.

It's not a massive fortune, but in the world of supercomputers, saving money and carbon without changing how the computer works is a huge win.

3. The "Reliability" Catch (The Safety Net)

You might ask, "If 2 loops are cheaper and greener, why did they build 4?"

The answer is reliability (safety).

Imagine a bridge with 4 lanes. If one lane collapses, you still have 3 lanes to get across.
If you have a bridge with only 2 lanes and one collapses, you have a 50% chance of being stuck.

The study found that while 2 loops are the best for money and carbon, 4 loops are better for safety. If a pump breaks in a 2-loop system, the whole system is at risk. In a 4-loop system, the backup pumps can take over easily.

The authors suggest a new way to think about this: Don't treat safety as a "nice-to-have" goal to be traded off. Instead, treat it as a hard rule.

If your rule is "We must have Tier IV safety (very high reliability)," then 4 loops is the correct answer.
If your rule is "We can tolerate a little risk," then 2 loops is the winner.

The paper argues that Frontier's current 4-loop design isn't "wrong"; it's just the result of following a strict safety rule. But if you are building a new system and your safety rules allow for it, you should build with 2 loops.

4. The "Decision Map" for Other Supercomputers

The authors didn't just solve the problem for Frontier. They created a map (a decision rule) that other engineers can use.

Frontier & El Capitan: Medium-sized flow -> 2 loops is best.
Aurora: Huge flow (twice as big) -> 4 loops is best (because the pipes get too long and heavy if you try to do it in fewer loops).
LUMI: Small flow -> 1 loop is best (it's so small, you don't even need a second loop).

Summary

This paper is like a mechanic telling you: "You've been building your supercomputer cooling system with four separate pipes because that's how it's always been done. But if you look at the total cost of the metal and the electricity, two pipes are actually the sweet spot."

However, if your boss says, "I need a safety net that never fails," then you stick with four pipes. The paper's main job is to show engineers exactly where that line is drawn, so they don't waste money and carbon on unnecessary pipes, but also don't cut corners on safety.

Technical Summary: Data Center Life Cycle Co-Design Optimization

Problem Statement

The design of cooling infrastructure for exascale supercomputers, such as the Frontier system at Oak Ridge National Laboratory (ORNL), traditionally relies on fixed topological decisions made during construction, specifically the number of parallel cooling subloops ( $K$ ) and the assignment of Coolant Distribution Units (CDUs) to these loops. While prior research has focused on optimizing operational control (e.g., flow rates and temperatures) within a fixed topology, these studies often ignore the manufacturing and construction phases of the facility's life cycle.

As electricity grids decarbonize, the relative importance of "embodied carbon" (emissions from materials and construction) increases, potentially accounting for 38% to 69% of a facility's total life cycle emissions. Current configurations for exascale systems (e.g., Frontier with $K=4$ , Aurora with $K \approx 10$ ) appear to reflect convention and supplier practices rather than systematic optimization. There is a lack of published studies that systematically enumerate feasible subloop configurations, evaluate them against both operational energy and embodied carbon, and identify a joint optimum. Furthermore, the trade-off between cost, carbon, and system reliability (downtime) has not been comprehensively analyzed for these topological choices.

Methodology

The authors propose a three-layer life cycle co-design optimization framework that integrates operational energy, embodied carbon, capital cost, and reliability. The framework is applied to the Frontier supercomputer, evaluating all 611 possible integer partitions of its 25 CDUs into $K$ subloops, where $K \in \{2, 3, 4, 5, 6\}$ .

Framework Components

Input Data: The model utilizes 2023 operational data (49,353 ten-minute intervals), hydraulic configurations, embodied carbon factors from the Inventory of Carbon and Energy (ICE) database and manufacturer EPDs, and a projected Tennessee Valley Authority (TVA) grid intensity trajectory.
Physics Models:
- Thermohydraulic Model: Computes heat exchange using the $\varepsilon$ -NTU method and sizes pipes based on ASHRAE standards.
- Equipment Sizing Model: Determines pump, Variable Frequency Drive (VFD), heat exchanger, and header dimensions required for peak loads.
- Reliability Model: Estimates unplanned downtime based on IEEE Std 493 component reliabilities (MTBF/MTTR) and maps subloop counts to Uptime Institute tier availability targets.
Optimization Layers:
- Layer 1: Enumerates all 611 partitions.
- Layer 2: Executes a validated Sequential Least Squares Programming (SLSQP) optimizer to determine optimal flow fractions and supply temperatures for each partition at every timestep, minimizing pump and fan power.
- Layer 3: Integrates operational energy with embodied carbon and reliability metrics to calculate Life Cycle Cost (LCC), Life Cycle Carbon ( $CO_2e$ ), and expected downtime over a 7-year horizon.

Analytical Generalization

To extend findings beyond Frontier, the authors derive a semi-analytical decision rule (Equation 2) that predicts the optimal $K^*$ as a function of total peak flow rate ( $Q$ ) and drop budget ( $N_{CDU} \times L_{drop}$ ). This rule balances operational efficiency penalties against embodied carbon costs (specifically piping).

Key Contributions

Integrated Framework: Development of a three-layer framework that unifies operational control, embodied carbon, and reliability into a single life cycle objective for cooling plant topology.
Identification of Optimum: Determination that the life cycle cost and carbon optimum for the Frontier cooling plant is achieved at $K=2$ with a partition of (14, 11). This configuration yields 3,320.7 tonnes of $CO_2e$ and a Net Present Value (NPV) of $3.99 million over seven years.
Semi-Analytical Decision Rule: Creation of a portable decision map that predicts optimal subloop counts for other exascale systems based on flow and drop budget parameters.
Pareto Analysis: A three-objective analysis (cost, carbon, downtime) demonstrating that treating reliability as a hard constraint (set by operations policy) rather than a soft objective significantly alters the design space, expanding the Pareto front from a single point to a "staircase" frontier.

Results

Optimal Configuration vs. As-Built

The study finds that the as-built Frontier configuration ( $K=4$ , partition 7-6-6-6) is suboptimal compared to the $K=2$ configuration (14-11).

Carbon Savings: 50.2 tonnes of $CO_2e$ (a 1.5% reduction).
Cost Savings: $100,000 NPV (a 2.5% reduction).
Driver of Savings: The savings are driven almost entirely by embodied carbon, specifically the reduction in primary piping length required for fewer loops. Operational energy savings are negligible because the SLSQP optimizer extracts nearly all available efficiency gains even at $K=2$ .

Sensitivity and Robustness

The $K=2$ optimum was tested against 15 sensitivity scenarios involving variations in operational horizon (5–10 years), grid trajectory (optimistic to pessimistic), discount rates, material costs, and embodied carbon factors.

The $K=2$ (14, 11) partition remained on the Pareto front in all 15 scenarios.
While the magnitude of savings fluctuates with parameter uncertainty, the identity of the optimal configuration does not change.

Generalization to Other Systems

Applying the decision rule to other systems yields distinct predictions:

Aurora: Predicted optimal $K^* = 4$ (due to high peak flow requiring more loops to manage pump capacity without excessive piping).
El Capitan: Predicted optimal $K^* = 2$ (similar flow/drop profile to Frontier).
LUMI: Predicted optimal $K^* = 1$ (low flow and drop budget make the embodied penalty of a second loop unjustified).

Reliability and Pareto Fronts

When reliability is treated as a hard constraint:

For Frontier, the $K=2$ solution offers the lowest cost/carbon but carries ~22 hours of annual downtime (Tier II).
To meet Tier IV availability (0.4 hours downtime), the design must shift to $K=4$ .
The study argues that the deployed $K=4$ configuration is consistent with a constrained optimum (meeting Tier IV requirements) rather than a flawed design.

Significance and Claims

The paper claims that the conventional assumption—that more subloops yield better operational efficiency—is structurally incorrect for systems where control policies (like SLSQP) can already maximize efficiency at lower loop counts. The marginal benefit of additional loops is consumed by the embodied carbon and capital cost of extra pumps, drives, and piping.

The primary significance of this work is the reversal of the design priority order: for clean or decarbonizing grids, minimizing embodied carbon (by reducing loop count) becomes more critical than seeking marginal operational gains. The authors position their findings not merely as a numerical optimization for Frontier, but as a corrective framing for the broader field of HPC facility design. They advocate for treating availability tiers as hard constraints set by operations policy, within which the partition optimization should then be performed to minimize life cycle cost and carbon.

The study concludes that for organizations planning multiple exascale builds, identifying the optimal configuration before construction can yield significant cumulative savings, and that the proposed decision map provides a portable tool for other plant designers to apply these principles to their specific system parameters.