Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inference

1. Problem Statement

While the environmental impact of training Large Language Models (LLMs) is well-documented, the energy costs associated with inference (the generation of outputs from trained models) are increasingly critical due to the billions of daily inference calls. Previous research has largely focused on hardware optimization, model architecture, and batching strategies. However, there is a significant gap in understanding how prompt characteristics (input design, semantic meaning, and specific keywords) influence energy consumption. The authors posit that prompt engineering, traditionally used for accuracy, could be optimized for energy efficiency, but the relationship between prompt semantics and inference energy remains unexplored.

2. Methodology

The study employs a rigorous empirical approach to measure the energy footprint of LLM inference under controlled conditions.

• Experimental Setup:

  • Models: Three open-source, 7-billion-parameter Transformer-based models were selected to ensure architectural similarity while introducing diversity in training data and vocabulary: Mistral 7B, Gemma 7B, and Vicuna 7B.
  • Hardware: Experiments were conducted on a single NVIDIA A100 GPU within a local cluster to isolate energy variables.
  • Measurement Tool: The ZEUS library (utilizing NVIDIA Management Library/NVML) was used to record precise GPU energy consumption (Joules) and latency for every inference run.
  • Data Volume: A total of 891,000 unique inference requests were generated by combining 99 hyperparameter configurations with 9,000 distinct prompts.

• Task Categories:
  The models were tested across three distinct task types, each with specific dataset sources:

  1. Question Answering (Q&A): Based on SQuAD, GPT4, and WebGLM-QA.
  2. Sentiment Analysis: Based on IMDB, Yelp, and Twitter datasets (requiring JSON output).
  3. Text Generation: Based on OpenOrca, HelpSteer, and GPTeacher (requiring creative or descriptive output).

• Variables Analyzed:

  • Input: Prompt length, semantic meaning, and specific keywords.
  • Output: Response length (token count), correctness (noted as a secondary factor), and latency.
  • Hyperparameters: Temperature, max time, max new tokens, and low memory settings were varied to test robustness.

3. Key Contributions

The paper makes four primary contributions to the field of sustainable AI:

1. Experimental System: Development of a reproducible measurement system for capturing one-shot LLM inference energy consumption across diverse workloads.
2. Response Characteristic Analysis: Quantification of the correlation between output metrics and energy, specifically establishing that response length is a stronger predictor of energy cost than input prompt length.
3. Task and Semantic Impact: Demonstration that task type and prompt semantics significantly dictate response generation behavior and energy utilization, establishing a direct link between task complexity and energy cost.
4. Keyword-Based Energy Reduction: Identification of specific keywords that correlate with high or low energy usage, showing that strategic keyword selection can reduce energy consumption by up to 60% for certain tasks.

4. Key Results

• Response Length vs. Prompt Length:

  • There is a very strong positive correlation (Pearson coefficient ~0.9) between the length of the generated response (in tokens) and energy consumption.
  • Conversely, the length of the input prompt has a negligible impact (weak correlation of 0.08–0.19) on energy usage. The semantic complexity of the prompt, rather than its token count, drives the model to generate longer, more energy-intensive outputs.

• Model-Specific Behaviors:

  • Mistral 7B exhibited the lowest average energy consumption (547 J).
  • Gemma 7B consumed the most energy (727 J) despite having the lowest energy cost per token (0.1 J). This is because Gemma is significantly more verbose, generating longer responses on average due to its larger vocabulary size and training data characteristics.
  • Vicuna 7B averaged 700 J and showed a tendency to include emoticons, which increased token counts and energy usage for that specific model.

• Task-Based Variations:

  • Sentiment Analysis was the most energy-efficient task (all models <500 J) because it requires concise, binary outputs.
  • Q&A and Text Generation were the most energy-intensive, often requiring verbose, detailed explanations.

• Keyword Impact:

  • Specific command words significantly alter energy costs.
  • High-Energy Keywords: Words like "explain," "justify," and "analyse" (in certain contexts) consistently triggered longer responses and higher energy usage (e.g., "explain" averaged ~1000+ J in Q&A tasks).
  • Low-Energy Keywords: Words like "classify," "summarize," and "identify" (in sentiment contexts) correlated with shorter responses and lower energy usage.
  • Reduction Potential: Switching from high-energy to low-energy keywords for specific tasks resulted in energy reductions of 50% to 60%.

5. Significance and Implications

• Shift in Optimization Strategy: The study challenges the assumption that prompt length is the primary driver of cost. Instead, it highlights that semantic framing is the critical lever.
• Prompt Engineering for Sustainability: "Green Prompting" is proposed as a viable strategy. By carefully selecting keywords that constrain the model to concise outputs (e.g., using "classify" instead of "explain" where appropriate), users can drastically reduce the carbon footprint of LLM interactions without necessarily sacrificing task accuracy.
• Model Selection: The findings suggest that model selection must consider not just parameter count or per-token efficiency, but also the model's inherent verbosity. A model with lower per-token energy costs may still be more expensive overall if it generates significantly longer responses.
• Future Directions: The paper calls for the development of predictive models that can estimate energy costs based on input semantics before inference occurs, enabling real-time energy optimization in production systems.

In conclusion, the paper establishes that the semantic meaning of a prompt is a dominant factor in LLM inference energy costs, offering a practical, user-level pathway to more sustainable AI deployment.