💬 NLP

Before and After Temperature: A Distributional View of Creative LLM Generation

This paper demonstrates that a novel reference-free metric, which quantifies how sampling temperature reshapes an LLM's token distribution prior to generation, significantly outperforms existing baselines in predicting creative quality by achieving a Spearman correlation of 0.918 against LLM judges and 0.870 against human rankings.

Original authors: V. S. Raghu Parupudi, Harsha Ponnada, Aditi Kaushal, S. Shria Parupudi, Saiteja Dasari, Sahiti Bulusu

Published 2026-06-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: V. S. Raghu Parupudi, Harsha Ponnada, Aditi Kaushal, S. Shria Parupudi, Saiteja Dasari, Sahiti Bulusu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a magician pull a rabbit out of a hat. Usually, to judge if the trick was good, you look at the rabbit (the final text the AI wrote). You might ask, "Is the rabbit fluffy? Is it white? Does it look like a real rabbit?"

This paper argues that we are looking at the wrong thing. Instead of judging the rabbit, we should be looking at how the magician's hand moved just before pulling the rabbit out.

Here is the breakdown of the research using simple analogies:

1. The Problem: Judging the Rabbit, Not the Magic

When AI models write creative stories, there is no "correct" answer to compare them to. Usually, people try to judge the quality by looking at the final text.

The Old Way: They use metrics like "Perplexity" (how surprised the AI is by its own words). The paper says this is like judging a painting only by how many brushstrokes it has. It tells you the painting is smooth, but not if it's a masterpiece or a mess.
The New Idea: The researchers looked at the AI's internal thought process right before it picked a word. They compared two versions of the AI's "mind":
1. The Raw Belief: What the AI naturally thought was the best word to say next.
2. The Tempered Belief: What the AI thought after a "temperature" knob was turned.

2. The "Temperature" Knob

Think of the "Temperature" setting as a volume knob for chaos.

Low Temperature (0.3): The AI is very calm and focused. It picks the most obvious, safe words. It's like a student taking a test and only writing the answers they are 100% sure of.
Medium Temperature (0.8): The AI is relaxed but still sensible. It's like a friendly conversation.
High Temperature (1.5): The AI is turned up to "wild." It starts picking weird, unlikely words just to be creative. It's like a jazz musician improvising so hard they start playing notes that don't fit the song.

3. The Discovery: The "Leak" in the Bucket

The researchers found a secret signal hidden in how the "Temperature" knob changes the AI's mind.

Imagine the AI's brain is a bucket filled with water (the probability of different words).

At Low/Medium Temperatures: The water stays mostly where it started. The AI picks words that were already in the "top 90%" of its favorite choices.
At High Temperature (1.5): The researchers found a massive "leak." When the temperature is cranked up to 1.5, about 13% of the water leaks out of the bucket's main area and spills onto the floor (the "tail" of unlikely words).

The Analogy:
If you ask a human to tell a story, and they suddenly start using words that make no sense in the context (like saying "The cat ate a toaster" when talking about a picnic), you know they are hallucinating or losing coherence.
The paper found that the AI shows a mathematical "leak" exactly when it starts doing this. The AI's internal map of "good words" gets so distorted by the high temperature that it starts picking words that were never really on its radar to begin with.

4. The Results: A Better Crystal Ball

The researchers tested this "leak" signal against two groups of judges:

Super-smart AI Judges (GPT-4o and Gemini).
Human Judges (real people).

The Scoreboard:

The Old Methods: The standard ways of judging AI creativity (like checking how "surprised" the AI is) got a score of about 0.76 (on a scale where 1.0 is perfect). They were okay, but not great.
The New Method: The "Pre-vs-Post Temperature" signal got a score of 0.918 against the AI judges and 0.870 against the humans.

What this means: The new method is significantly better at predicting which stories are actually creative and coherent, simply by measuring how much the "temperature" knob distorted the AI's internal map.

5. The Catch: It Can't Tell "Good" from "Great"

There is one limit to this magic trick.

The method is amazing at spotting Chaos (High Temperature = Bad/Incoherent).
However, it struggles to tell the difference between Calm (Low Temperature) and Relaxed (Medium Temperature).

The Analogy:
The method is like a smoke detector. It is fantastic at screaming "FIRE!" when the house is burning (High Temperature/Incoherent). But it can't really tell the difference between a house that is "cozy and warm" (Medium Temperature) and a house that is "cool and crisp" (Low Temperature). Both look like "no fire" to the detector. The paper suggests that to tell the difference between "good" and "great" creative writing, we need to look at the whole story (the sequence), not just the single word being picked.

Summary

This paper discovered that to judge if an AI is writing creatively or just gibberish, you don't need to read the whole story. You just need to look at how much the AI's internal "voting" system gets scrambled when you turn up the "creativity" knob. If the scrambling causes the AI to pick words that were never really on its shortlist, the story is likely falling apart. This simple check is much more accurate than all the previous methods used to judge AI creativity.

Technical Summary: Before and After Temperature: A Distributional View of Creative LLM Generation

Problem Statement

Evaluating the creativity of Large Language Model (LLM) generations in open-ended tasks without reference text is a persistent challenge. Traditional reference-free metrics, such as perplexity, often fail because they reward fluent, high-frequency text, leading to "neural text degeneration" (bland, repetitive output). While recent approaches utilizing model embeddings or LLM-as-judge prompts have improved evaluation, they operate on the surface form of the generated text rather than the underlying probability distribution from which the text was sampled. Existing methods that do analyze distributions typically collapse them into scalar values (e.g., entropy, margin) or rely on comparing multiple samples, ignoring the specific structural transformation applied by the sampling temperature ( $T$ ) before a token is emitted.

Methodology

The authors propose a framework that treats the gap between the model's raw belief distribution and the temperature-reshaped distribution as a primary signal for evaluation.

Experimental Setup

Model & Prompts: Experiments were conducted using Llama-3.1-8B-Instruct on 500 open-ended creative prompts sourced from WritingPrompts, the Alternative Uses Task, and HellaSwag.
Sampling Conditions: Generations were produced at three temperatures: $T \in \{0.3, 0.8, 1.5\}$ .
Logit Capture: For every generated token, the top- $K$ raw logits were captured before sampling. $K=100$ for $T \le 0.8$ and $K=200$ for $T=1.5$ . Crucially, the system guarantees the sampled token is included in the captured set (appended if necessary), ensuring well-defined analyses even at high temperatures where the distribution is flat.
Distribution Construction: Two distributions are computed per token:
1. Pre-temperature ( $p$ ): The model's raw belief, $p = \text{softmax}(z)$ .
2. Post-temperature ( $q$ ): The reshaped distribution, $q = \text{softmax}(z/T)$ , from which the token was actually sampled.
Ground Truth: Evaluation rankings were established via:
- An LLM-judge ensemble (averaged rankings from gpt-4o and gemini-2.5-pro) on all 500 prompts.
- A human-majority ranking (three independent raters) on a 150-prompt subset.
- Both ground truths utilized a "coherence-first" rule, penalizing incoherence and disconnected jargon.

Metrics

The study computes a suite of distributional features comparing $p$ and $q$ , including:

Information-Theoretic Divergences: KL, Jensen-Shannon.
Geometric Distances: Total Variation, L2, Hellinger.
Top-End Properties: Shifts in top-1 probability and identity.
Cumulative-Mass Width ( $n_\alpha$ ): The number of tokens required to cover a probability mass $\alpha$ (e.g., $n_{90}$ ).
Mass Leakage: The fraction of post-temperature mass that remains on the pre-temperature plausible set (e.g., $S(p)_{90}$ ).
Aggregation: Features are aggregated per response using percentiles (p10, p90), max, and mean to capture different aspects of the generation trace.

Key Contributions & Results

1. Superior Predictive Power of Pre-vs-Post Features

The primary finding is that a single per-token feature derived from the pre-vs-post temperature reshape significantly outperforms standard reference-free baselines.

Top Feature: The 10th percentile of the pre-temperature top-90% width ( $n_{90}(p)_{\text{p10}}$ ) achieved a Spearman correlation ( $\rho$ ) of 0.918 against the averaged LLM judge and 0.870 against the human majority.
Baseline Comparison: Four standard baselines (self-perplexity, mean predictive entropy, top-1 margin, and gzip compression ratio) all plateaued at $|\rho| \approx 0.76$ .
Gap: The proposed method outperforms the best baseline by +0.165 (LLM judge) and +0.110 (human majority), a margin significantly larger than the variance among the baselines themselves.

2. Mechanistic Insight: The Incoherence Signature

The performance gain is driven by a sharp distributional signature that distinguishes the "incoherence regime" ( $T=1.5$ ) from coherent regimes ( $T=0.3, 0.8$ ).

Mass Leakage: At $T=1.5$ , approximately 13 percentage points of probability mass "leak" off the pre-temperature top-90% plausible set. The post-temperature mass remaining on the original plausible set drops from ~0.982 ( $T=0.8$ ) to ~0.868 ( $T=1.5$ ).
Width Inflation: The cumulative-mass width ( $n_{95}(q)$ ) inflates from $\sim 1$ token to $\sim 131$ tokens at $T=1.5$ .
Uniformity: This distortion is uniform across the generation trace, not limited to isolated tokens.

3. Limitations of Per-Token Aggregates

While the method excels at separating incoherent generations ( $T=1.5$ ) from coherent ones, per-token aggregates fail to distinguish between the two coherent regimes ( $T=0.8$ vs. $T=0.3$ ). The distributions $p$ and $q$ retain similar shapes in these regimes, suggesting that the "mid-vs-bland" distinction relies on sequence-level features (content novelty, trajectory) rather than single-step distributional reshaping.

Significance and Claims

The paper claims that the "pre-vs-post temperature view" offers a cheap, single-pass, and reference-free metric that captures the structural impact of sampling temperature on generation quality.

Efficiency: Unlike methods requiring multiple samples (e.g., semantic entropy) or external references, this approach extracts signal from the same forward pass used to generate the text.
Interpretability: The mechanism is interpretable: high creativity (or in this case, incoherence) correlates with a specific leakage of mass from the model's original plausible set into the tail.
Benchmarking: The study establishes that the averaged LLM judge (gpt-4o/gemini-2.5-pro) is a reliable proxy for human judgment in this context, achieving a correlation of $\rho=0.832$ with human raters, surpassing the inter-human agreement ceiling of $\rho=0.771$ .

The authors conclude that while per-token distributional shifts are a powerful indicator of the incoherence boundary, distinguishing between subtle variations in coherent creativity likely requires a hybrid approach combining these distributional features with sequence-level analysis.