💬 NLP

XTC: Head-Aware Sampling by Excluding Top Choices

The paper introduces XTC (Exclude Top Choices), a lightweight decoding strategy that selectively removes dominant high-probability tokens to significantly enhance generation diversity and creativity while maintaining fluency and instruction-following accuracy across various large language models.

Original authors: Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

When a computer program learns to write, it does not think in sentences the way a human does. Instead, it predicts the next word based on the ones that came before, choosing from a long list of possibilities. For a long time, the standard way to make these programs write varied and interesting text was to nudge them toward less common words or to cut off the very unlikely ones. This approach worked well for avoiding nonsense, but it often led to a different problem: the text would become safe, repetitive, and strangely uniform. Even when the computer knew several good ways to continue a story, it would almost always pick the single most obvious, generic option, like a traveler who always chooses the main road even when a scenic path is right next to it. This tendency to stick to the safest choice, even when other strong options exist, has been quietly draining the creativity out of machine-generated writing.

A team of researchers has now introduced a new method called XTC, which stands for Exclude Top Choices, designed to fix this specific habit. Rather than looking at the unlikely words at the bottom of the list or trying to flatten the entire list of choices, this new technique focuses on the top of the list. It waits until the computer is genuinely undecided between two or more very good options. When that happens, the method deliberately removes the most popular choice from the running, forcing the computer to pick from the remaining strong alternatives. It is a simple switch that only flips when the computer is truly torn between good paths, ensuring the output stays diverse without becoming nonsensical.

The researchers tested this idea on several different large language models, ranging from smaller ones with 12 billion parameters to massive ones with 70 billion. They asked the models to write creative stories, brainstorm ideas, and engage in dialogue, comparing the new method against the standard ways of generating text. The results were clear and consistent. When using XTC, the generated text became significantly more diverse, with a measurable increase in the variety of words and phrases used. At the same time, the text became much less repetitive, with the models repeating the same three-word phrases far less often. In one set of tests, the variety of the writing improved by between 11 and 15 percent, while the repetition of common phrases dropped by nearly half.

Crucially, the researchers found that this increase in variety did not come at the cost of quality or clarity. When human volunteers and other advanced computer programs judged the writing, they preferred the XTC-generated text for its creativity and interest, rating it higher than the standard versions. The writing remained fluent and easy to read, with no noticeable drop in overall quality. The method also proved to be very precise in its application; it did not interfere with tasks that required strict accuracy, such as following specific instructions or writing code, unless the settings were pushed too far. In fact, when the researchers compared XTC to the traditional method of simply making the computer more random, XTC achieved the same level of variety while preserving the ability to follow instructions much better.

The study also showed that this new approach works well alongside existing tools. It can be combined with other methods that control randomness or repetition, and the benefits add up rather than cancel each other out. When paired with a standard temperature setting, the combination produced even greater improvements, reducing repetition by over 70 percent in some cases while boosting variety by nearly 40 percent. The researchers confirmed that this effect holds true across different types of models and different levels of computer memory compression, suggesting that the method is robust and widely applicable.

What makes this finding significant is that it addresses a blind spot in how these systems have been tuned for years. Previous methods assumed that the problem with repetitive writing came from the very unlikely, strange words at the bottom of the list. This new work shows that the real issue often lies at the top, where the computer is so confident in its safest choice that it ignores other perfectly good alternatives. By gently nudging the system away from that single dominant option only when it is genuinely uncertain, the method unlocks a more natural and human-like range of expression. The technique has already been adopted by several popular open-source tools used by developers, indicating that it offers a practical way to make artificial intelligence writing feel less like a machine and more like a creative partner.

Technical Summary: XTC (Exclude Top Choices)

1. Problem Statement

Autoregressive language models require decoding strategies that balance faithfulness to the learned distribution with the goals of downstream tasks. Existing diversity-promoting methods, such as temperature scaling, top- $k$ , top- $p$ (nucleus), and typical sampling, primarily operate by truncating the low-probability tail of the token distribution or rescaling the global entropy. While effective for reducing tail noise, these approaches often fail in a specific "head-ambiguity" regime.

In this regime, a model assigns substantial probability to several individually plausible continuations but disproportionately concentrates mass on the single most generic or conventional choice. Because the relevant alternatives reside in the distribution's "head" rather than the tail, tail-truncation methods have little effect. Conversely, global flattening (e.g., high temperature) is too coarse, intervening at every step regardless of whether the model is genuinely uncertain among strong options. This leads to a homogenization of content where individual outputs appear varied, but population-level diversity remains low, and models default to the safest, most repetitive continuations.

2. Methodology: XTC (Exclude Top Choices)

XTC is a lightweight, head-aware decoding operator designed to intervene specifically when a model faces multiple plausible options but overweights the dominant one. It operates as a post-processing step on the next-token distribution $p_t$ after any upstream transformations (e.g., temperature, penalties).

The Algorithm

Given a next-token distribution $p_t$ , a plausibility floor $\tau$ , and an intervention probability $\rho$ :

Identify Eligible Set: Collect all tokens $v$ where $p_t(v) \ge \tau$ into a set $E_t$ .
Check Ambiguity: If $|E_t| < 2$ , the head is unambiguous; return $p_t$ unchanged.
Stochastic Trigger: Flip a biased coin with probability $\rho$ . If tails, return $p_t$ unchanged (sparse-in-time property).
Select Underdog: If heads and $|E_t| \ge 2$ , identify the token $u \in E_t$ with the lowest probability (the "weakest" plausible alternative).
Exclude Dominant: Define the removal set $R_t = E_t \setminus \{u\}$ . If $R_t$ contains protected tokens (e.g., EOS, newlines), abort and return $p_t$ .
Renormalize: Set $p_t(v) = 0$ for all $v \in R_t$ and renormalize the remaining probabilities to sum to 1.

Theoretical Properties

Sparse in Time: Inactive when the head is unambiguous or when the stochastic trigger fails.
Head-Aware: Targets high-probability alternatives, distinct from tail-truncation methods.
Compositional: Can be inserted into any existing sampler stack.
KL-Projection: The renormalization step is the unique minimizer of the Kullback-Leibler divergence $KL(q \parallel p_t)$ among all distributions supported on the surviving tokens, preserving the relative odds of the remaining options.

3. Key Contributions

Formalization of XTC: Defined as a specific operator on token distributions with a complete, implementation-ready algorithm.
Structural Analysis: Derived properties distinguishing XTC from entropy-based and tail-truncation methods, including a KL-projection interpretation.
Comprehensive Evaluation: Conducted 60 experiments across three primary model families (Gemma 3 27B q4, Gemma 3 12B q6, DeepSeek R1 14B q6) and a scaling validation on Llama 3.3 70B q4.
Diversity-Repetition Pareto Frontier: Demonstrated that XTC improves the trade-off between diversity and repetition on creative tasks, with gains scaling monotonically from 12B to 70B parameters.
Human and LLM Validation: Validated results via a blinded Amazon Mechanical Turk study (150 raters) and a cross-vendor LLM judge panel (Claude Opus 4.7 and GPT-4o).
Instruction-Following Preservation: Quantified the cost of diversity on instruction-following tasks (IFEval), showing XTC preserves accuracy significantly better than temperature scaling for equivalent diversity gains.

4. Experimental Results

Diversity and Repetition

Creative Generation: On 24 creative prompts across 12 genres, XTC (at $\rho=1.0, \tau=0.1$ $ρ = 1.0, τ = 0.1$ ) achieved the best scores on all metrics.
- Distinct-2: Gains of 11–15% across four models, increasing with parameter count (12B to 70B).
- Repetition: Repeat trigram reductions of 27–47%.
- Composition: When composed with temperature scaling, total improvements over the baseline reach 38% in Distinct-2 and 71% in repeat trigram reduction.
Generalization: Effects were consistent across quantization levels (q4 vs. q6) and model architectures (Gemma, Qwen-derived DeepSeek, Llama).

Quality and Human Preference

LLM Judges: Both Claude Opus 4.7 and GPT-4o agreed that XTC significantly improves repetition and directionally improves creativity and lexical diversity without degrading overall quality.
Human Evaluation: In a blinded study with 150 Master raters, XTC was preferred over the baseline for Creativity in 62.3% of comparisons ( $p < 10^{-4}$ ). It tied or beat the baseline on Quality in 84.4% of comparisons.

Instruction Following (IFEval)

On Llama 3.3 70B q4, XTC Light and Medium shifted prompt-level strict accuracy by only $-0.4$ and $-1.7$ percentage points, respectively.
In contrast, a temperature setting ( $T=1.15$ ) tuned to achieve a similar Distinct-2 gain caused an 8.8 percentage point drop in IFEval accuracy.
XTC delivers approximately 5× more diversity gain per instruction-following point lost compared to temperature scaling.

5. Significance and Claims

The paper claims that XTC addresses a specific, previously overlooked failure mode in autoregressive generation: the "head-ambiguity" regime where models have multiple strong options but default to the safest one. By targeting the head rather than the tail, XTC provides a mechanism to increase stylistic diversity and reduce repetition without the heavy cost to coherence or instruction-following typically associated with global entropy manipulation.

The authors position XTC as a practical, drop-in solution that has already been adopted by major open-source inference frameworks (llama.cpp, ExLlamaV2, text-generation-webui). The work suggests that future decoding strategies should consider the "head" of the distribution as a distinct control surface, orthogonal to tail truncation and global temperature.

Limitations Acknowledged:

XTC is not universally beneficial; it can degrade performance on tasks requiring exactness (e.g., code generation, structured extraction) if the intervention rate $\rho$ is too high.
The optimal parameters ( $\tau, \rho$ ) are task-dependent and may vary across model scales and tokenizers.
It is a decoding-time intervention and cannot fix fundamental model deficiencies or training-data homogenization.