💻 computer science

Emissions and Performance Trade-off Between Small and Large Language Models

This study demonstrates that fine-tuned Small Language Models can achieve performance comparable to Large Language Models across various tasks while significantly reducing carbon emissions, offering a viable path toward sustainable AI.

Original authors: Anandita Garg, Uma Gaba, Deepan Muthirayan, Anish Roy Chowdhury

Published 2026-01-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Anandita Garg, Uma Gaba, Deepan Muthirayan, Anish Roy Chowdhury

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to get a job done, like writing a short story, analyzing a review, or fixing a piece of code. You have two options for help:

The "Super-Brain": A massive, all-knowing giant (a Large Language Model or LLM) that knows everything in the world. It's incredibly powerful, but it's like a giant, roaring factory that burns through electricity and produces a huge cloud of smoke (carbon emissions) every time it speaks.
The "Specialist": A smaller, focused worker (a Small Language Model or SLM) who has been specifically trained for one specific job. It's much smaller, quieter, and uses very little energy.

This paper asks a simple question: Do we always need the "Super-Brain," or can the "Specialist" do the job just as well without the massive pollution?

The Experiment: A Race Between Giants and Specialists

The researchers took six different types of tasks and pitted these two types of AI against each other. They measured two things:

Performance: How well did the AI do the job?
Emissions: How much carbon dioxide (CO2) was released just to get the answer (the "inference" phase).

Here is how the race played out, task by task:

1. The Easy Wins for the Specialists (4 out of 6 tasks)
In four of the six categories, the small, specialized models were just as good as the giants, but they were eco-friendly superheroes.

Sentiment Analysis (Reading Reviews): Imagine trying to tell if a Yelp review is happy or sad. The tiny specialist (ELECTRA) got it right 95% of the time, just like the giant. But while the giant emitted enough CO2 to fill a small room, the specialist emitted a tiny speck of dust. The specialist was 1,300 times cleaner.
Content Creation (Writing Stories): When asked to write a short story from a prompt, the small models wrote stories just as good as the giants. The giant was a bit more "perplexed" (confused) by the task, but the small models were efficient. The emissions dropped by 1,200 times.
Natural Language Inference (Logic Checks): This is like checking if two sentences logically follow each other. The small specialists actually beat the giants here, getting 88% accuracy compared to the giants' 80%. They did this while emitting almost no carbon.
Code Summarization (Explaining Code): When asked to explain what a block of code does, the small models were surprisingly good. The giant actually struggled here, scoring lower than the small specialists. The small models were 660 times cleaner.

2. Where the Giants Still Win (2 out of 6 tasks)
There were two tasks where the small specialists simply couldn't keep up.

Chain-of-Thought Reasoning: This is like solving a complex math problem where you have to show your step-by-step work. The small models got lost. The "Super-Brain" was necessary here because the task is too complex for a small mind.
Code Generation (Writing Code from Scratch): When asked to write a new program from a description, the small models failed to produce working code most of the time. The giant, with its massive knowledge base, was the only one who could do it reliably.

The Big Takeaway: "Right Tool for the Right Job"

The paper concludes that we don't need to use the "Super-Brain" for everything. It's like using a sledgehammer to crack a nut.

The Trade-off: For about two-thirds of the tasks tested, using a small, fine-tuned model gives you the same result as the giant but saves a massive amount of energy.
The Cost: Using the giant isn't just bad for the planet; it's expensive. The paper notes that running these giants requires expensive computer chips and huge electricity bills. Using the small models saves money on both the equipment you buy (Capital Expenditure) and the electricity you pay for every month (Operational Expenditure).

The Bottom Line

The authors argue that we are facing an "Energy Dilemma." We are currently using giant, energy-hungry AI for simple tasks that don't need that much power.

Their solution? Be smart about which AI you use.

If you need to write a story, check a review, or explain code, use the Small, Specialized Model. It's green, cheap, and just as effective.
If you need to solve a complex logic puzzle or write a brand-new software program, you might still need the Giant Model, even though it costs more and pollutes more.

The paper suggests that by switching to these smaller models for the right jobs, we can make AI "Green" without sacrificing quality, saving both the environment and corporate wallets.

Technical Summary: Emissions and Performance Trade-off Between Small and Large Language Models

Problem Statement

The proliferation of Large Language Models (LLMs) in academia and industry has precipitated a significant environmental crisis. While LLMs offer advanced capabilities, their lifecycle—from energy-intensive training to billions of daily inferences—generates substantial carbon dioxide (CO2) emissions. For instance, the training of GPT-3 alone emitted an estimated 550 metric tons of CO2, excluding hardware manufacturing and inference costs. Crucially, recent estimates suggest that inference accounts for up to 90% of a model's total life cycle energy use. Despite this, regulatory frameworks and industry practices often prioritize training emissions or lack standardized metrics for inference, creating a gap in understanding the true environmental cost of deploying these models at scale. The central problem addressed is whether fine-tuned Small Language Models (SLMs) can serve as a sustainable alternative to LLMs for specific tasks, offering comparable performance with drastically reduced carbon footprints.

Methodology

The study conducted a comparative analysis of fine-tuned SLMs against open-source LLMs across six tasks categorized into Natural Language Processing (NLP), Reasoning, and Programming.

Model Selection:
- SLMs: Three models were fine-tuned for each task, constrained to a maximum of 250 million parameters (e.g., ELECTRA, DistilBERT, FLAN-T5, CodeT5).
- LLMs: Benchmarked against open-source models with over 1 billion parameters, including Qwen 3 (235B), Mistral Instruct (7B), and DeepSeek R1 (671B).
Tasks:
- NLP: Sentiment Analysis (Binary) and Content Creation (Short Story Writing).
- Reasoning: Natural Language Inference (NLI) and Chain-of-Thought (CoT) Reasoning.
- Programming: Code Summarization and Code Generation.
Emission Calculation:
- The study utilized the open-source eco2ai library to track carbon emissions based on energy consumption of CPU, GPU, and RAM.
- Since public inference data for specific benchmark LLMs was unavailable, the authors derived estimates by scaling from GPT-3's known emission rate (4.32 g CO2 per query). A linear adjustment was applied based on active parameters ( $P_{active}$ ) and token counts per task.
- The formula used was: $Carbon_{token} = K \times P_{active}$ , where $K$ is derived from GPT-3 data, and $Carbon_{inference} = Carbon_{token} \times Tokens_{task}$ .

Key Results

The empirical investigation yielded mixed results regarding performance parity, but consistent results regarding emission reduction.

Performance Parity

Fine-tuned SLMs achieved performance comparable to LLMs in four out of six tasks:

Sentiment Analysis: SLMs (e.g., DistilRoBERTa) achieved 96% accuracy, outperforming Mistral-7B (68%) and matching DeepSeek R1 (84%), with negligible performance differences among SLMs.
Content Creation: Fine-tuned SLMs (e.g., DistilGPT2) maintained BERT Score F1 and Perplexity metrics comparable to LLMs.
Natural Language Inference (NLI): SLMs (FLAN-T5 Base, deBERTa v3) achieved accuracies of ~88%, significantly outperforming all tested LLMs (max 80%).
Code Summarization: SLMs (CodeParrot-Small, TinyCodeLM) significantly outperformed Qwen3 and DeepSeek R1 in BLEU and ROUGE-L scores.

Performance Gaps:
SLMs failed to match LLMs in two tasks:

Chain-of-Thought Reasoning: SLMs struggled with complex reasoning (Modified G-Eval scores of 19–47 vs. 54–64 for LLMs), supporting literature that advanced reasoning emerges in models with tens of billions of parameters.
Code Generation: SLMs achieved low pass@1 scores (max 12.8%) compared to LLMs (DeepSeek R1 at 65.9%), indicating that complex generation tasks require larger model scales.

Emission Reduction

Across all tasks, fine-tuned SLMs demonstrated orders of magnitude reduction in inference emissions:

Sentiment Analysis: SLMs emitted between 0.0001 g and 0.0696 g CO2/query, compared to 0.27 g – 1.37 g for LLMs.
Content Creation: SLMs emitted 0.0063 g – 0.0105 g CO2/query, versus 1.44 g – 7.31 g for LLMs.
NLI: SLMs emitted ~0.0015 g CO2/query, compared to 0.36 g – 1.83 g for LLMs.
Code Summarization: SLMs emitted 0.0055 g – 0.0973 g CO2/query, versus 0.72 g – 3.66 g for LLMs.

The study notes that for the four successful tasks, emissions were reduced by factors ranging from 660x to 13,000x per query.

Significance and Claims

The paper positions these findings as a resolution to the "LLM Energy Dilemma," arguing that the trade-off between model size and environmental impact is not always linear.

Viability of SLMs: The study claims that for specific, well-defined tasks (NLP classification, summarization, and basic content generation), fine-tuned SLMs (≤250M parameters) are a viable, sustainable alternative to massive LLMs.
Economic and Environmental Synergy: Beyond environmental benefits, the authors argue that shifting to SLMs for appropriate tasks reduces the Total Cost of Ownership (TCO). By requiring less expensive hardware (lower Capital Expenditure) and consuming less power during continuous inference (lower Operational Expenditure), organizations can achieve financial savings alongside carbon reduction.
Policy Recommendations: The paper advocates for regulatory changes, specifically mandating that AI providers disclose inference energy costs and conduct Sustainable Impact Assessments for all AI systems, not just during development.
Modest Scope: The authors remain modest about their claims, acknowledging that SLMs cannot yet replace LLMs for complex reasoning or code generation. They emphasize that the "sweet spot" for model size requires further research and that their study is limited to open-source models and specific parameter ranges.

In conclusion, the paper asserts that a future of "Green AI" is achievable not by abandoning LLMs entirely, but by strategically deploying curated, task-specific fine-tuned SLMs where they offer performance parity, thereby drastically lowering the carbon footprint of AI deployment.