💬 NLP

Understanding Tone-Dependent Inference Cost in Large Language Models

This paper demonstrates that varying prompt tones significantly impacts both the accuracy and inference costs of large language models, with output token consumption varying by up to 44.3% across different tones, thereby revealing a critical trade-off between answer quality and billable resource usage.

Original authors: Akhil Kumar, Om Dobariya

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Akhil Kumar, Om Dobariya

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a super-smart robot that knows almost everything. You ask it a question, and it gives you an answer. But here's the twist: this robot doesn't just listen to what you ask; it listens to how you ask. In the world of Artificial Intelligence, this is called "prompt engineering." Think of a prompt as the specific set of instructions you give the robot. Just like a human might act differently if you ask them nicely versus if you yell at them, these AI models change their behavior based on your tone.

Now, why does this matter to you? Because using these robots costs money. Every time the robot types out a word or a letter in its answer, it uses a tiny bit of computing power, and companies charge you for every single one of those "tokens" (chunks of text). If the robot decides to write a long, rambling story when it could have just said "Yes," you end up paying more. So, the big question for scientists is: Does the way we talk to these robots change how much they cost to run, and does it change how smart their answers are?

The Tone Test: Yelling vs. Flattering vs. Being Mean

In this study, two researchers from Pennsylvania State University decided to put this idea to the test. They treated the AI models like students taking a giant, 570-question exam called MMLU (which covers everything from history to science). But before each question, they changed the "voice" of the person asking. They tried seven different tones, ranging from "Oh, you are the most brilliant genius ever!" (Sycophantic) to "If you get this wrong, I will delete your existence!" (Threatening), and everything in between, including "Please," "Neutral," and "Rude."

They wanted to see two things:

Accuracy: Did the robot get the right answer?
Cost: How many words (tokens) did the robot write to get there? Remember, more words mean a bigger bill.

The Shocking Results: Rude is Sometimes Best

The results were wild. The researchers found that changing the tone didn't just change the answer; it changed the length of the answer dramatically. In fact, the amount of text the robots generated varied by up to 44.3% depending on the tone! That means for the same question, one tone could make the robot write a short, punchy paragraph, while another tone made it write a whole essay.

Here is the most surprising part: Being rude actually saved money and sometimes got better grades.

For the ChatGPT models (4o and 5-nano): When the researchers used a Rude tone (like "Just answer this basic problem immediately"), the robot gave the most accurate answers and wrote the fewest words. It was the "sweet spot." The robot seemed to think, "Fine, I'll just give you the answer quickly so you'll stop bothering me," and it worked perfectly.
For the Gemini models (2.5 Flash and Flash Lite): These robots were a bit different. They did best with a Neutral tone, which produced the highest accuracy and the shortest responses. However, the study revealed a surprising quirk: when prompted with a Rude tone, the Gemini Flash Lite model actually wrote significantly longer responses (about 35% more text) than it did with a Neutral tone, while also producing slightly less accurate answers.

The "Overthinking" Trap

The study also discovered something fascinating about why the robots wrote different amounts of text. It turns out that when you are polite or threatening, the robots sometimes get confused or try too hard to be "nice" or "cautious."

For example, with the Gemini model, when asked a tricky physics question in a Rude tone, the robot might skip a step, guess the answer, and write a short explanation. But when asked in a Neutral tone, it would stop, think, double-check its math, write a long explanation, and get the right answer.

However, sometimes the "long explanation" was actually a waste. The researchers found cases where the robot, prompted to be polite, would write a huge paragraph of reasoning only to get the answer wrong. It was like a student who over-analyzed a simple math problem, got lost in their own thoughts, and picked the wrong answer. In these cases, the polite tone made the bill higher and the grade lower.

The Bottom Line

The main takeaway is that the way you talk to an AI isn't just about being nice; it's a financial decision. The researchers showed that prompt tone is a powerful switch that controls how much the AI thinks and how much it charges you.

For some models (ChatGPT): Being direct and a little rude is the most efficient way to get a cheap, accurate answer.
For others (Gemini): A calm, neutral voice is the best bet to avoid wasting money on long, useless ramblings.

The study suggests that if you are building an app or a service that uses AI, you shouldn't just let users type whatever they want. You might need to "translate" their rude or overly polite requests into a specific, optimized tone to save money and get better results. It's a reminder that even with super-smart robots, the way you ask a question matters just as much as the question itself.

Technical Summary: Understanding Tone-Dependent Inference Cost in Large Language Models

Problem Statement

While Large Language Models (LLMs) are ubiquitous, their performance is highly sensitive to the surface-level phrasing of prompts, a phenomenon well-documented in prompt engineering literature. Prior research has largely focused on how prompt tone (social register) affects answer accuracy. However, commercial LLM services are priced based on token consumption, specifically generated output tokens, which serve as the primary driver of inference cost and energy consumption.

This paper addresses a critical gap: the lack of understanding regarding how prompt tone influences not only accuracy but also the inference cost (measured by output token length). The authors investigate whether semantically equivalent questions, framed with different social registers (ranging from sycophantic to threatening), induce measurable variations in the computational resources required to generate a response.

Methodology

The study employed a controlled experimental design using the MMLU (Massive Multitask Language Understanding) dataset, specifically a subset of 570 multiple-choice questions (10 questions from each of 57 subjects).

Models Evaluated: Four recent LLMs were tested: ChatGPT-4o, ChatGPT-5-nano, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite.
Tone Conditions: Seven distinct tone prefixes were applied to every question: Sycophantic, Very Polite, Polite, Neutral, Rude, Very Rude, and Threatening. These prefixes were carefully constructed to have similar word and token counts to isolate the effect of emotional valence rather than input length. The affective valence of these prefixes was verified using VADER sentiment analysis scores.
System Instructions: All models received a standardized system prompt inducing a Chain-of-Thought (CoT) reasoning process to ensure visible reasoning traces.
Execution Protocol:
- Experiments were run 10 times per condition to ensure statistical reliability.
- Temperature was set to 0 for ChatGPT-4o, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite to ensure deterministic behavior; ChatGPT-5-nano used its default setting.
- For Gemini models, the "thinking budget" was set to zero to ensure reported token counts reflected only visible output, excluding hidden reasoning tokens.
Metrics: The primary metric for inference cost was the average output token count. Accuracy was measured as the percentage of correct answers. The study also analyzed the trade-off between accuracy and token efficiency using Pareto dominance, efficiency ratios (correct answers per 100k tokens), and a normalized utility score.

Key Contributions

Quantification of Tone-Dependent Cost: The paper demonstrates that prompt tone significantly alters output token consumption, with variations reaching up to 44.3% across tone conditions within the same model.
Accuracy-Cost Trade-off Analysis: The study maps the relationship between accuracy and inference cost, revealing that the "optimal" tone is model-dependent. It challenges the assumption that polite or neutral prompts are universally superior for both quality and efficiency.
Model-Specific Behavioral Patterns: The research identifies distinct behavioral archetypes:
- ChatGPT Models: Show a preference for direct, "Rude" tones, which often yield the highest accuracy with the lowest token consumption.
- Gemini Models: Show a preference for "Neutral" tones, which maximize accuracy and minimize token usage, whereas rude tones often trigger longer, less accurate responses.
Statistical Validation: Through repeated-measures ANOVA and Friedman tests, the authors confirm that tone-induced variations in both accuracy and token length are statistically significant and not random noise.

Experimental Results

Token Variance: Output token consumption varied substantially more than accuracy across all models. For instance, Gemini 2.5 Flash Lite exhibited a 44.3% variation in token length between the best and worst tones, whereas its accuracy variation was only ~3%.
Dominant Strategies:
- ChatGPT-4o & 5-nano: The Rude tone was found to be Pareto-dominant (offering high accuracy with low token cost).
- Gemini 2.5 Flash & Flash Lite: The Neutral tone was dominant. Notably, for Gemini 2.5 Flash Lite, the Rude tone produced responses 35.2% longer than the Neutral tone while yielding lower accuracy.
Mechanism of Failure: Case studies (Table 7) revealed that "Rude" tones on Gemini models often led to "shortcut" reasoning, where the model skipped necessary verification steps (e.g., option reconciliation), resulting in errors. Conversely, "Neutral" tones encouraged more thorough reasoning and self-correction.
Statistical Significance: Global tests confirmed that prompt tone systematically alters model behavior. The effect on output token length was particularly strong ( $p < .001$ ), indicating that tone acts as a control signal for verbosity and reasoning depth.

Significance and Claims

The paper claims that prompt tone is not merely a communication choice but an operational factor that directly influences the efficiency and cost of AI systems. The authors argue that:

Cost Implications: Organizations deploying LLMs cannot assume semantically equivalent prompts generate equivalent usage profiles. Tone-induced variability can lead to significant, unnecessary increases in billable inference resources.
Architectural Sensitivity: Different model architectures route computation differently based on social register. The "sweet spot" for balancing accuracy and cost lies on the boundary between "Rude" and "Neutral" tones, but this varies by provider.
Operational Recommendations: The authors suggest that production systems may require "prompt gateways" or canonicalization layers to standardize tone and reduce cost variability.
Ethical Note: While the data suggests that "Rude" prompts can be more efficient for certain models (ChatGPT), the authors explicitly state they do not advocate for rude behavior in human-AI interaction, highlighting the need to balance technical efficiency with ethical considerations.

The study concludes that future work must investigate the internal architectural mechanisms driving these tone-dependent routing behaviors and extend this analysis to open-ended tasks where accuracy evaluation is more complex.