Wasted large language models: A life cycle thinking approach

Technical Summary: Wasted Large Language Models: A Life Cycle Thinking Approach

Problem Statement
Large Language Models (LLMs) possess a rapidly growing carbon footprint due to the immense energy required for their development, storage, transmission, and inference. While the field of "Green AI" focuses on improving energy efficiency during training and deployment, the authors argue that efficiency gains alone are insufficient. Citing the Jevons Paradox, the paper notes that increased efficiency often drives increased consumption, negating environmental benefits. Furthermore, the rapid iteration of LLMs—driven by frontier labs and a vast ecosystem of open-weight models—leads to frequent deprecation of older models. This cycle creates a scenario where the costs of training numerous models may outweigh their benefits if their lifespans are artificially shortened by the constant release of new versions. The core problem is the lack of a framework to treat LLMs as products that can become "waste," leading to unnecessary resource expenditure.

Methodology
The paper employs a conceptual framework based on Life Cycle Thinking (LCT) and applies the Waste Hierarchy from the European Union's Waste Framework Directive (WFD) to the domain of software and machine learning.

1. Reframing LLMs as Products: The authors define LLMs as "Software 2.0" (following Andrej Karpathy's distinction from "Software 1.0" code), where instructions are encoded as neural network weights. They adapt the WFD definition of waste ("any substance or object which the holder discards") to LLMs, defining an LLM as waste when it is discarded and no longer used, typically due to the release of superior models.
2. Application of the Waste Hierarchy: The authors systematically map the five measures of the WFD hierarchy—Prevention, Preparing for Re-use, Recycling, Recovery, and Disposal—to the lifecycle of LLMs.
3. Scope Definition: The analysis focuses on the energy consumption associated with developing (training, validation), storing, moving (transmission), and using LLMs. It treats hardware production and data center infrastructure impacts as secondary, as the primary goal is to reduce the need for new model production through better management of existing artifacts.

Key Contributions and Proposed Measures
The paper proposes specific technical and strategic approaches for each level of the waste hierarchy:

• Prevention and Reuse:

  • Definition: Preventing waste by extending the lifespan of models and reusing existing artifacts.
  • Approaches: Shifting the industry focus from "performance at all costs" to "sufficiency" (using models that are "good enough"). The authors suggest slowing model turnover to ensure training costs are amortized.
  • Techniques: Utilizing Retrieval Augmented Generation (RAG) to augment existing models with external knowledge bases rather than retraining. Implementing Agent Skills (reusable instructions/prompts) to reduce the inference load required for specific tasks.

• Recycling and Recovery:

  • Definition: Reprocessing waste into new products (Recycling) or ensuring waste serves a useful purpose (Recovery).
  • Approaches: The paper identifies Fine-tuning and Low-Rank Adaptation (LoRA) as primary recycling methods. LoRA, in particular, allows for the creation of reusable adapter libraries, enabling the adaptation of base models without retraining the entire network.
  • Techniques: LLM Merging is highlighted as a method to combine existing models to improve capabilities without training new ones from scratch. Quantization and Pruning are noted as methods to optimize models for resource-constrained devices, extending their utility.
  • Comparison: The authors note a weakness in LLMs compared to traditional code: while code snippets are trivial to reuse, LLMs require complex techniques (like LoRA) to be effectively "recycled."

• Disposal:

  • Definition: The destruction or discarding of waste.
  • Approaches: While digital deletion leaves no physical residue, the paper emphasizes that storage consumes significant energy. Therefore, the active disposal (deletion) of unused LLMs and data is a critical measure for reducing the carbon footprint of storage infrastructure.
  • Cultural Shift: The ease of digital deletion fosters a "throwaway mentality." The authors argue for a more considerate attitude toward the resources spent on training, suggesting that disposal should be a deliberate act to save energy, not just a default action.

• Prevention of Unnecessary Use:

  • The paper distinguishes between preventing the waste of the model product (training it and not using it) and preventing the unnecessary use of the model (using an LLM for tasks better suited to efficient tools). The authors highlight a gap in Human-Computer Interaction (HCI) where current interfaces lack mechanisms to discourage inefficient LLM usage, often employing "dark patterns" to encourage overuse.

Results and Analysis
The application of the waste hierarchy reveals that prevention is the most effective measure for reducing environmental impact, primarily because it eliminates the need for the energy-intensive training of new models.

• Lifecycle Trends: Citing Trisovic's research on the lifespan of 62 LLMs (2018–2025), the paper notes that model lifespans are shortening due to market competitiveness rather than intrinsic technical obsolescence.
• Barriers: The primary barrier to reuse is the rapid release of new models, which deprecates older ones. The paper argues that technical solutions (like RAG and LoRA) exist but require a shift in economic and cultural incentives to be adopted.
• Limitations: The authors acknowledge that training new models is sometimes necessary to address limitations like bias or hallucinations, or to test new data curation methods. However, the decision to train must be weighed against the benefits, a calculation currently obscured by the "throwaway" culture.

Significance and Claims
The paper modestly claims that its contribution is not a set of immediate legal guidelines or a comprehensive solution, but rather a conceptual framework to reframe the discussion around LLM development.

• Reframing: By viewing LLMs as products subject to waste management principles, the paper aims to make developers and users more conscious of the hidden resource costs of training.
• Motivation: The significance lies in challenging the growth-based economy's assumption that newer is always better. The authors argue that in an era of rapid capability increases, "sufficiency-based" approaches (reuse, recycling) are difficult to incentivize but are essential for climate-conscious resource management.
• Future Work: The paper calls for deeper investigation into the actual lifespans of LLMs in industry and consumer contexts, beyond the academic research community, to better understand the drivers of model deprecation.

In conclusion, the paper posits that applying life cycle thinking to LLMs offers a pathway to reduce their climate impact not just by making models more efficient, but by fundamentally changing how they are treated as digital artifacts—prioritizing the extension of their useful life and the prevention of unnecessary production.