💻 computer science

Evaluating the Environmental Impact of using SLMs and Prompt Engineering for Code Generation

This paper presents the first systematic empirical study demonstrating that specific prompt engineering strategies, particularly Chain-of-Thought, can optimize the trade-off between code generation accuracy and environmental sustainability in Small Language Models, while highlighting that regional grid carbon intensity is the dominant factor in deployment emissions.

Original authors: Md Afif Al Mamun, Sayan Nath, Gias Uddin, Novarun Deb

Published 2026-04-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Md Afif Al Mamun, Sayan Nath, Gias Uddin, Novarun Deb

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to cook a perfect meal (writing code). In the past, you had to hire a world-famous, super-expensive celebrity chef (a massive AI model) who worked in a giant, centralized kitchen (the cloud). You paid them a fee, and they did the cooking for you.

But recently, a new trend has emerged: Small Language Models (SLMs). These are like talented home cooks you can hire to work right in your own kitchen. They are cheaper, keep your recipes private, and don't require a subscription. However, there's a catch: you now have to pay the electricity bill for your own stove, and you are responsible for the carbon footprint of your cooking.

This paper is like a green energy audit for these home cooks. The researchers wanted to answer a simple question: "If I ask my home cook to use a specific recipe method (Prompt Engineering), does it make the meal better, or does it just waste more electricity?"

Here is the breakdown of their findings using simple analogies:

1. The "Recipe" Matters More Than the "Chef's Size"

The researchers tested 11 different "home cooks" (AI models) ranging from tiny (1 billion parameters) to medium-sized (34 billion parameters). They asked them to solve coding problems using six different "cooking styles" (Prompting Strategies):

Direct: "Just cook the meal." (The simplest approach).
Chain-of-Thought (CoT): "Think step-by-step before cooking." (Like writing a shopping list before chopping).
Self-Consistency: "Ask the chef to cook the meal 5 different ways, then pick the best one." (The most expensive method).
ReAct/Least-to-Most: Complex methods involving tools and breaking tasks into tiny sub-tasks.

The Surprise:
They found that bigger isn't always better. A tiny, well-trained chef could sometimes cook a better meal than a giant, expensive one.

The Winner: The "Chain-of-Thought" method was the "Goldilocks" choice. It was like asking the chef to think through the steps. It produced high-quality meals (accurate code) without burning a massive amount of fuel (energy).
The Loser: The "Self-Consistency" method was like asking the chef to cook the same dish five times and pick the best one. It did make the meal slightly better, but it burned 5 times more electricity for a tiny gain. It was an inefficient use of resources.

2. The "Power Grid" is the Real Boss

This is the most critical finding. Imagine two chefs:

Chef A works in a kitchen powered by a coal-fired generator (Alberta, Canada).
Chef B works in a kitchen powered by hydroelectric dams and nuclear plants (Ontario, Canada).

Even if Chef B's stove is older, slower, and uses more electricity to cook the same meal, Chef B creates far less pollution.

The study found that where the code is generated matters more than how it is generated.

If you run a model in a region with "dirty" energy (coal/gas), your carbon footprint is huge, even if you use a tiny, efficient model.
If you run the same model in a region with "clean" energy, your footprint drops by nearly 80%, even if the model works harder.

The Metaphor: It doesn't matter if you drive a fuel-efficient hybrid car (efficient model) if you are driving it on a road made of toxic sludge (dirty energy grid). The location of the energy source is the biggest factor.

3. Token Count vs. Time Spent

Many people think, "The fewer words the AI writes, the less energy it uses." The researchers found this is only half true.

Tokens are like the words written on the page.
Time is how long the stove is left on.

They found that how long the computer is working (Time) is a much better predictor of pollution than how many words it writes (Tokens).

Analogy: You can write a very long letter (many tokens) very quickly on a typewriter (fast hardware), or you can write a short note very slowly by hand (slow hardware). The slow hand might use more energy because the "machine" (your brain/hand) was active for longer, even though the output was shorter.

4. The "Format" Problem

The study also noticed that some tiny models were so eager to please that they would "talk too much" or output gibberish when asked to follow strict rules (like writing code in a specific format).

Analogy: Imagine a tiny dog trying to fetch a stick. If you just say "Fetch," it might bring you a rock, a shoe, or nothing at all. But if you say, "Fetch the stick, then sit, then drop it," it might actually do it. However, if the dog is too small (too small a model), it might get confused and bark nonsense instead of fetching the stick.
Result: Tiny models sometimes failed to follow the "format" instructions, forcing developers to ask them to try again. This "retry loop" wasted extra energy and time.

The Big Takeaway

If you are a developer using AI to write code:

Don't overcomplicate it: You don't need the most complex "reasoning" prompts. A simple "think step-by-step" (Chain-of-Thought) is usually the sweet spot between quality and energy saving.
Don't just chase the biggest model: A smaller, smarter model often does the job just as well with less energy.
Location is key: If you have the choice, run your AI in regions with clean energy (like hydro or wind). This saves more carbon than changing your code or your model.

In short: To be "green" with AI, you don't just need a better model; you need a smarter recipe (prompt) and a cleaner kitchen (energy grid).

1. Problem Statement

The rapid shift from cloud-hosted Large Language Models (LLMs) to locally deployed, open-source Small Language Models (SLMs) has democratized AI-assisted coding but has also decentralized the environmental footprint of AI. While cloud inference centralizes energy consumption and carbon emissions in data centers, local deployment shifts this burden directly to developers' hardware and regional electricity grids.

Despite the prevalence of Prompt Engineering strategies (e.g., Chain-of-Thought, ReAct) used to optimize code generation without retraining models, their impact on energy consumption and carbon emissions remains largely invisible to developers. Existing literature prioritizes accuracy metrics (e.g., Pass@1) while systematically neglecting the sustainability trade-offs of different prompting strategies and model scales in decentralized environments. This paper addresses the gap by investigating how prompting strategies and model characteristics influence the balance between code generation accuracy and environmental sustainability.

2. Methodology

The authors conducted a systematic empirical study involving 11 open-source SLMs (ranging from 1B to 34B parameters) and six distinct prompting strategies.

Models Evaluated: Models were categorized by size (Tiny: 1B–3B, Small: 7B–16B, Medium: 20B–34B) and training objective (General-purpose vs. Code-specialized). Examples include Llama3.2, Qwen3-Coder, CodeLlama, and DeepSeek-Coder.
Prompting Strategies:
1. Direct: Baseline problem description.
2. Chain-of-Thought (CoT): Decomposes problems into reasoning steps.
3. Program-of-Thought (PoT): Executes code to validate intermediate results.
4. Self-Consistency: Generates multiple samples and votes for the best answer.
5. Least-to-Most (LtoM): Decomposes problems into sub-problems solved sequentially.
6. ReAct: Iterative loop of reasoning, action, and observation.
Benchmarks: Evaluation was performed on HumanEval+ (164 problems) and MBPP+ (150 problems).
Experimental Setup:
- Framework: All strategies were implemented using the DSPy framework to ensure canonical, machine-parsable code outputs, reducing variability inherent in ad-hoc prompting.
- Hardware & Regions: Experiments were run on two distinct machines in different Canadian provinces to isolate hardware and grid effects:
  - Machine 1: Xeon Silver CPU + NVIDIA A100 GPU (Alberta, high carbon intensity grid).
  - Machine 2: Xeon Gold CPU + NVIDIA L40S GPU (Ontario, low carbon intensity grid).
Metrics:
- Accuracy: Pass@1 (percentage of problems solved correctly on the first attempt).
- Sustainability: Energy consumption (kWh), Carbon emissions (kgCO2eq), and Inference latency.
- Efficiency: Token usage and the derived Carbon per Correct Answer (CpCA) metric.

3. Key Contributions

First Systematic Study: This is the first work to quantify the sustainability trade-offs of prompting strategies specifically for SLM-based code generation.
Decoupling of Accuracy and Sustainability: The study provides empirical evidence that high accuracy does not necessarily correlate with high environmental cost, allowing for "green" optimization without performance sacrifice.
Grid Carbon Intensity (GCI) Dominance: The research highlights that the regional electricity grid's carbon intensity is a more significant determinant of emissions than hardware efficiency or token count.
New Metric (CpCA): Introduction of "Carbon per Correct Answer" to evaluate the efficiency of prompting strategies in terms of emissions per functional unit of work.

4. Key Results

RQ1: Accuracy vs. Emissions across Model Sizes

Non-Monotonic Scaling: Code generation accuracy does not scale linearly with parameter count. Smaller, well-trained models (e.g., Qwen2.5-Coder:7B) often outperform larger models (e.g., Gemma3:27B).
Tiny Model Limitations: Very small models (1B–3B) often fail to produce structured, parsable outputs when using frameworks like DSPy, leading to high parsing error rates (e.g., StarCoder2:3B had >90% parsing failures).
Emission Profile: Per-query CO2 emissions showed weak dependence on model size; runtime configuration and inference efficiency were more dominant factors.

RQ2: Impact of Prompting Strategies

Chain-of-Thought (CoT) is Optimal: CoT achieved a near-optimal balance, offering high accuracy (63.94% Pass@1) with significantly lower overhead than complex strategies. It reduced energy usage by ~80% compared to Self-Consistency while maintaining competitive accuracy.
Diminishing Returns of Complex Strategies: Strategies like Self-Consistency achieved the highest accuracy (65.75%) but incurred disproportionate costs (4.9x higher emissions than CoT) due to multi-sampling. ReAct and Least-to-Most also showed high token counts and energy usage without proportional accuracy gains.
Direct Prompting: While having the lowest absolute energy cost, it often consumed more prompt tokens than CoT in the DSPy framework due to lack of structural constraints on reasoning models.

RQ3: Influence of Hardware and Region

Grid Carbon Intensity (GCI) is Dominant: The study found that GCI is the primary driver of emissions.
- Example: Machine 2 (Ontario) consumed 113% more energy than Machine 1 (Alberta) due to more powerful hardware, yet produced 78% fewer CO2 emissions because Ontario's grid is low-carbon (nuclear/hydro) compared to Alberta's fossil-fuel-heavy grid.
Implication: Regional energy profiles outweigh hardware efficiency in determining the carbon footprint of local inference.

RQ4: Correlations

Inference Time vs. Emissions: Inference time was the strongest predictor of CO2 emissions ( $R^2 \approx 0.92$ ), followed closely by energy consumption.
Token Count: Token usage showed only moderate correlation with energy/emissions ( $R^2 \approx 0.41-0.52$ ), indicating that token count alone is an insufficient proxy for environmental impact.
Accuracy vs. Sustainability: Pass@1 accuracy showed no meaningful correlation with sustainability metrics, confirming that performance and environmental cost can be optimized independently.

5. Significance and Implications

Green Prompt Engineering: Developers can adopt simpler strategies like Chain-of-Thought to achieve high-quality code generation while minimizing environmental impact, avoiding the heavy costs of multi-sampling or complex reasoning loops.
Strategic Deployment: For organizations deploying SLMs locally, the choice of geographic location (or cloud region) for inference is a critical lever for sustainability, often more impactful than hardware upgrades.
Developer Tooling: The findings suggest a need for tools that visualize real-time carbon costs and CpCA metrics, enabling developers to make informed trade-offs between accuracy and ecological responsibility.
Future Research: The paper advocates for standardized sustainability benchmarks, adaptive strategy selection based on real-time carbon signals, and incentives for efficient, specialized models over indiscriminate scaling.

In conclusion, the paper demonstrates that sustainability in code generation is decoupled from accuracy. By selecting appropriate prompting strategies (favoring CoT) and considering regional energy contexts, the software engineering community can significantly reduce the carbon footprint of AI-assisted development without compromising code quality.