💻 computer science

The Hidden Cost of Thinking: Energy Use and Environmental Impact of LMs Beyond Pretraining

This paper reveals that the environmental impact of modern language model development is dominated by post-training experimentation and complex reasoning pipelines—accounting for 82.2% of total compute and making reasoning models 17 times more expensive to post-train than instruction-tuned ones—highlighting a critical, unreported gap in current environmental reporting standards.

Original authors: Jacob Morrison, Noah A. Smith, Emma Strubell

Published 2026-05-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jacob Morrison, Noah A. Smith, Emma Strubell

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The Iceberg of AI Costs

Imagine you are building a massive, high-tech factory to bake the perfect cake (a smart AI model). For years, everyone has only been counting the electricity used to bake the final cake and putting that number on the label.

This paper argues that this is like looking at an iceberg and only counting the tip above the water. The authors went underwater to measure the entire iceberg. They found that the "baking" of the final model is actually a tiny part of the story. The real energy, carbon emissions, and water usage happen during the messy, experimental process of figuring out how to bake the cake in the first place.

1. The "Thinking" Models Are Energy Hungry

The researchers studied a new family of AI models called Olmo 3. They built two types:

The "Instruct" Model: Follows orders well (like a helpful assistant).
The "Think" Model: Solves complex problems by "thinking" through steps (like a mathematician working out a proof).

The Analogy:
Imagine the "Instruct" model is a sprinter. It runs a short race and finishes. The "Think" model is a marathon runner who stops to solve puzzles along the way.

The Finding: Training the "Think" model required 17 times more energy than the "Instruct" model.
Why? The "Think" model has to generate massive amounts of "practice runs" (called rollouts) to learn how to reason. It's like a student who has to write out 100 practice essays just to learn how to write one good one. The paper calls this the "cost of thinking."

2. The "Hidden" Cost: Trial and Error

The biggest surprise in the paper is about development costs.

Old Belief: We thought most energy was used for the final training run (the "final exam").
New Reality: 82% of the total energy was used for experimentation. This includes failed attempts, testing different settings, and trying out different data mixes before the final model was even built.

The Analogy:
Think of it like a chef trying to create a new signature dish.

The Final Dish: The one served to the customer (the final model).
The Development: The chef burning 100 pots of soup, throwing away 50 batches of cookies, and tasting 20 different spice blends to get it right.
The Paper's Point: If you only count the electricity used to cook the one final dish served to the customer, you are ignoring the 100 pots of soup that were burned in the kitchen. The paper found that for modern AI, the "burned soup" (failed experiments) costs 5 times more than the final dish.

3. The Water Problem: It's Not the Data Center's Fault

The paper also looked at water usage. They found the process used a huge amount of water (enough for one person to drink for 140 years).

The Analogy:
Many people think AI uses water because the computers get hot and need to be cooled down with water (like a car radiator).

The Reality: The data center used a "closed-loop" system, meaning it didn't lose water to cooling.
The Real Culprit: The water was used to make the electricity that powered the computers. Power plants (coal, gas, nuclear) use massive amounts of water to generate the power needed to run the AI.
The Lesson: If you want to save water, you don't just need to build better computers; you need to change how we generate the electricity that powers them.

4. The Total Bill

When the researchers added up everything—the final training, the failed experiments, the data generation, and the hardware manufacturing—the total cost for building the Olmo 3 models was:

Energy: About 12.3 million kilowatt-hours (enough to power 847 US homes for a whole year).
Carbon: About 4,251 tons of CO2.
Water: About 15,887,000 liters.

Why This Matters

The paper concludes that the way we currently report AI's environmental impact is broken. We are only reporting the "final run" cost, which makes the problem look much smaller than it actually is.

As AI models get smarter and require more "thinking" (reasoning), and as the development process gets more complex, these hidden costs will skyrocket. The authors are asking developers to stop hiding the "burned soup" and start reporting the full cost of the entire kitchen, not just the final meal.

In short: Building smart AI is expensive, but the process of figuring out how to build it is even more expensive, and the "thinking" models are the most energy-hungry of all.

Technical Summary: The Hidden Cost of Thinking

Problem Statement

Current environmental reporting for Large Language Models (LLMs) is narrowly focused on the energy and carbon costs of pretraining a single final model. This approach renders the majority of the modern development lifecycle invisible. As model pipelines have evolved to include mid-training, long-context extension, synthetic data generation, supervised fine-tuning (SFT), preference optimization (DPO), and reinforcement learning (RL), the environmental footprint has expanded significantly. Furthermore, the emergence of "reasoning" models, which rely heavily on Reinforcement Learning with Verifiable Rewards (RLVR), introduces compute-intensive workloads (specifically rollout generation) that are not captured in standard pretraining-centric reports. Additionally, prior work has suggested that development costs (experimentation, failed runs, ablations) are substantial, but the extent of these costs in complex, multi-stage pipelines remains unquantified.

Methodology

The authors provide a detailed environmental accounting for the Olmo 3 family of models (7B and 32B parameters), released in both instruction-following ("Instruct") and reasoning ("Think") variants.

Scope: The study covers the entire development pipeline: data generation, pretraining, mid-training, long-context extension, and all post-training stages (SFT, DPO, RLVR), including both final training runs and the development iterations (hyperparameter searches, ablations, failed runs) preceding them.
Hardware & Infrastructure: Training occurred on the Jupiter cluster (Cirrascale, Texas) using NVIDIA H100-80GB GPUs. Data generation utilized the Frontier supercomputer with AMD MI250X GPUs.
Energy & Emission Calculation:
- Power: Total datacenter power ( $P_{DC}$ ) is calculated as $P_{GPU} \times 1.74 \times PUE$ . The 1.74 factor accounts for non-GPU server and IT infrastructure overhead (based on Epoch AI, 2025), improving upon prior methods that only multiplied GPU power by PUE.
- Carbon: Emissions are calculated as $CO_2e = P_{DC} \times CI$ , where Carbon Intensity (CI) is 0.332 kg CO2/kWh.
- Water: Consumption is calculated as $Water = P_{DC} \times (WUE_{onsite} + WUE_{offsite})$ . The datacenter uses closed-loop cooling ( $WUE_{onsite} = 0$ ), meaning all water consumption is attributed to power generation ( $WUE_{offsite} = 1.29$ L/kWh).
- Embodied Emissions: Hardware manufacturing impacts are amortized over a 4-year lifespan.
Comparison: The study compares the "Think" (reasoning) and "Instruct" variants, which share a common base model but diverge in post-training strategies.

Key Contributions

First Detailed Post-Training Breakdown: This work is the first to measure and report the environmental costs of specific post-training stages (SFT, DPO, RL) and synthetic data generation, enabling direct comparison across stages and model variants.
Quantification of "Thinking" Costs: The authors demonstrate that reasoning models are significantly more expensive to post-train than instruction-tuned models, driven specifically by RLVR rollout generation.
Revised Development Cost Estimates: The study updates the estimate of development costs (experimentation and failed runs) from the previously reported ~50% of total compute to 82.2%, highlighting that reporting only final runs understates the true impact by approximately 5×.
Water Attribution Analysis: The paper clarifies that water consumption in this specific setup is driven entirely by power generation infrastructure rather than datacenter cooling, challenging common narratives that attribute water use primarily to AI workload cooling.

Results

Total Environmental Impact

The development of the Olmo 3 series (including final runs, development iterations, and data generation) resulted in:

Energy: ~12.3 GWh of datacenter energy.
Carbon: ~4,251 tCO2eq (equivalent to ~847 US homes' electricity for one year).
Water: ~15,887 kL (equivalent to ~140 years of water use by an average US person).
Compute: ~13.2 million GPU hours total.

The Cost of Reasoning (Think vs. Instruct)

Energy Disparity: The 32B "Think" model's post-training phase consumes 17× more datacenter energy than the 32B "Instruct" model.
Driver: This disparity is driven by RLVR rollout generation. In the 32B Think model, RLVR rollout generation accounts for 87% of the post-training energy.
Mechanism: The RLVR generator (autoregressive inference) operates at ~600W per GPU, while the trainer operates at 130–220W. The generator is functionally identical to large-scale inference but requires generating extremely long traces (averaging >10,000 tokens, max 32,768) across many iterations.
Token Volume: The RLVR trainer for the 32B Think model processes ~18.7B tokens, compared to ~564M for the Instruct variant (a 33× difference).

Development Costs

Proportion: Development activities (hyperparameter searches, ablations, failed runs) accounted for 82.2% of total GPU hours and 80.9% of total GPU energy (excluding data generation).
Stage Variation: Post-training stages are particularly iteration-heavy. Development fractions for post-training stages exceeded 91% (e.g., 97.5% for mid-training/data mixing, 97.1% for DPO).
Comparison: This represents a significant increase from the ~50% development fraction reported in prior work (Morrison et al., 2025) for pretraining-focused pipelines.

Water Consumption

Source: 100% of the water consumption was attributed to power generation ( $WUE_{offsite}$ ), as the datacenter utilized closed-loop cooling ( $WUE_{onsite} = 0$ ).
Implication: Water costs are strictly proportional to energy consumption in this setup, suggesting that reducing energy use is the primary lever for reducing water impact, rather than changing cooling infrastructure.

Significance and Claims

The paper argues that the current standard of reporting only pretraining costs is insufficient for accurate environmental accounting. As pipelines become more complex and reasoning models become the default paradigm, the "hidden costs" of post-training and development are growing rapidly.

Transparency: The authors claim that accurate, measurement-based accounting of the full development lifecycle is essential for informed regulatory guidelines and for the research community to develop effective strategies to reduce AI's environmental footprint.
Feasibility: Contrary to claims that such detailed reporting is impractical, the authors demonstrate that it is feasible to break down costs across multiple stages and variants.
Policy Recommendations: The paper concludes with four recommendations:
1. Report full pipeline costs (post-training and data generation), not just pretraining.
2. Include development costs (at minimum as a multiplier).
3. Disclose datacenter efficiency metrics (PUE, WUE, cooling type, power mix).
4. GPU manufacturers should disclose manufacturing impacts (embodied emissions).

The work positions itself as a foundational step toward understanding the true environmental footprint of modern AI, emphasizing that the "cost of thinking" is a rapidly escalating factor that must be addressed in sustainability standards.