Transparent Screening for LLM Inference and Training Impacts

1. Problem Statement

The rapid adoption of Large Language Models (LLMs) has created a significant gap in environmental accountability. While LLMs dominate public discourse and practical application, direct environmental telemetry (energy consumption and carbon emissions) from proprietary service providers is generally unavailable.

• The Challenge: Without direct data, discussions regarding LLM environmental impacts rely on opaque claims, unsupported comparisons, or contradictory narratives.
• The Limitation of Current Approaches: Simply avoiding approximation does not improve rigor; it leaves a vacuum for unverified assertions. Existing tools often lack transparency regarding their assumptions or fail to distinguish between inference and training impacts.
• The Goal: To create a framework that provides auditable, source-linked, and reproducible estimates for LLM impacts under conditions of limited observability, enabling comparative analysis without claiming false precision.

2. Methodology

The authors propose a Transparent Screening Framework that converts natural-language application descriptions into bounded environmental estimates. The methodology relies on "proxy" models anchored in observed literature rather than provider claims.

A. Inference Screening Method

The inference estimator is designed for speed and interpretability, focusing on the operational phase of LLMs.

• Input: Users provide natural-language descriptions (e.g., "We use GPT-4o-mini for customer support, 4,000 uses/month"). A parser extracts model, request type, token volume, usage, and country context.
• Core Anchor: The method uses a literature-based anchor: 0.24 Wh/prompt (median prompt energy from Elsworth et al. for Gemini Apps).
• Calculation Logic:
  1. Weighted Prompt Volume ($V_t$): Accounts for the asymmetry between input processing and output generation.
     $$V_t = T_{in} + \omega T_{out}$$
     Where $\omega = 1.8$ (an internal prior reflecting that output generation is more computationally intensive).
  2. Effective Active Parameters ($P^{eff}_{t,s}$): Adjusts the raw parameter count ($P_t$) based on context window, serving mode, modality, and architecture.
     $$P^{eff}_{t,s} = P_t \times F_{ctx} \times F_{srv} \times F_{mod} \times F_{arch}$$
  3. Energy Estimation ($\hat{E}_{t,s}$): Scales the anchor energy based on the ratio of effective parameters and token volume, using exponents ($\alpha, \beta$) to define a low-central-high interval rather than a single point estimate.
     $$\hat{E}_{t,s} = E_a \times \left(\frac{P^{eff}_{t,s}}{P_a}\right)^{\alpha_s} \times \left(\frac{V_t}{V_{ref}}\right)^{\beta_s}$$
  4. Carbon Calculation: Converts energy to carbon ($\hat{C}$) using the specific electricity carbon intensity ($CI_c$) of the user's selected country.

B. Training Screening Method

Training estimates face higher uncertainty due to sparse public data.

• Approach: Uses a bounded proxy combining parameter counts with training priors.
• Formula:
  $$\hat{E}^{train}_{t,s} = E^{train}_a \times \left(\frac{P_t}{P_a}\right)^{\alpha_s} \times \left(\frac{Tok_t}{Tok_a}\right)^{\beta_s} \times F_{reg} \times F_{arch-tr} \times F_{hw}$$
• Key Assumptions:
  • Token Prior: Defaults to 20 training tokens per parameter (based on the Chinchilla scaling law) when specific data is missing.
  • Regime: Defaults to a "foundation-pretraining" regime unless narrower public data exists.
  • Scope: This training proxy is used for the observatory's comparative tables but is excluded from the user-facing application estimator to maintain focus on inference.

3. Key Contributions

1. Transparent Screening Framework: Introduces a methodology for estimating LLM impacts under limited observability that prioritizes audibility over false precision.
2. Bounded Multi-Factor Proxy: Separates inference and training estimates, explicitly stating assumptions (e.g., token weights, parameter adjustments) and providing uncertainty intervals (low-central-high) rather than single-point predictions.
3. Operational Observatory: Implements a working system (ImpactLLM Observatory) covering 41 market models, allowing users to input natural language and receive immediate, source-linked comparative data.
4. Contextualization: Explicitly integrates regional electricity mixes, demonstrating how the same model can have vastly different carbon footprints depending on the hosting country.

4. Results and Observations

The paper presents results from the ImpactLLM Observatory, highlighting several key findings:

• Scale of Impact: There is a massive divergence in screening values across models. For example, a standardized request for Ministral 3B yields ~0.0056 Wh, whereas Claude Opus 4.1 yields ~2.99 Wh (a ~500x difference).
• Annualization Matters: Small per-request values become significant when multiplied by real-world usage volumes. A chatbot using a lightweight model can still generate substantial annual emissions if usage is high.
• Electricity Context Dominance: The carbon intensity of the hosting country often dominates the interpretation. A model hosted in France (low-carbon grid) may have significantly lower emissions than the same model hosted in the US (higher-carbon grid).
• Training vs. Inference: While inference is the focus of the user tool, training estimates for frontier models (e.g., Claude Opus 4.1) are orders of magnitude higher (e.g., ~125 GWh) than inference costs, though these remain highly uncertain proxies.

5. Significance and Limitations

Significance:

• Policy and Governance: The framework serves as a "discussion tool" to make structural trends legible. It highlights that model-scale growth may be outpacing public environmental reporting, motivating better disclosure standards.
• Scientific Rigor: By replacing opaque claims with "source-linked, inspectable proxies," the paper argues for a more disciplined debate on AI sustainability. It organizes approximation rather than pretending to eliminate uncertainty.
• Reproducibility: Every value in the observatory is linked to a specific document, system boundary, and assumption, allowing for independent verification.

Limitations:

• Data Scarcity: The method relies on scarce literature anchors; uncertainty intervals are wide to remain honest about this lack of data.
• Opacity of Proprietary Systems: The method must rely on contextual proxies for architecture and hardware configurations of closed-source models.
• Training Uncertainty: Training emissions are reconstructed from sparse reports and assumed token priors; more public data is needed to reduce this uncertainty.
• Parser Ambiguity: Natural language inputs can be ambiguous, requiring the system to surface extracted assumptions to the user for validation.

Conclusion

The paper concludes that for proprietary LLM services, uncertainty is structurally unavoidable. The most valuable approach is not to claim direct measurement but to provide a transparent, reproducible screening proxy. This framework enables stakeholders to compare models, understand the impact of context (like electricity grids), and drive a more informed policy discussion regarding the environmental costs of AI.