💬 NLP

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

The paper introduces "Don't Repeat Yourself" (DRY), a sampling-time logit adjustment that effectively reduces verbatim looping in large language models by penalizing tokens that extend exact suffix matches from earlier context, thereby improving lexical diversity and fluency without degrading performance or requiring model retraining.

Original authors: Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

Published 2026-08-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Large language models are the engines behind the text-generating tools that have reshaped how we write, code, and converse with machines. These systems work by predicting the next word in a sentence, one piece at a time, building a response based on everything that came before it. For the most part, they are remarkably fluent, capable of weaving together complex ideas and mimicking human style. However, they suffer from a specific, frustrating flaw known as "verbatim looping." When a model gets stuck, it begins to repeat a phrase or a sentence over and over again, sometimes for hundreds of words, as if it has forgotten it just said it. This happens because the model's own output reinforces the pattern, creating a feedback loop that is difficult to break. While developers have long tried to fix this by simply telling the model not to use words it has already seen, these blunt tools often ruin the quality of the text, breaking up necessary formatting or making the writing sound robotic.

A new study introduces a smarter way to handle this problem, a method called "Don't Repeat Yourself," or DRY. Instead of punishing a model for using a word it has used before, this new approach looks at the structure of the sentence itself. It waits to see if the model is about to complete a pattern that has already appeared in the text. If the model is merely repeating a word that is part of a normal structure, like a speaker label in a dialogue or a bullet point in a list, the method leaves it alone. But if the model starts to extend a sequence of words that matches a previous section exactly, the method gently nudges the model to choose a different path. This distinction is crucial because it allows the model to keep its natural rhythm and formatting while stopping the runaway repetition that causes it to get stuck.

The researchers tested this idea across a wide range of computer models, from small ones that can run on a personal laptop to massive systems with billions of parameters. They ran thousands of generations using different types of prompts, including creative writing, long conversations, and structured formatting tasks. The results were clear: the new method reduced the rate of these exact repetition loops by nearly half compared to doing nothing at all. More importantly, it did so without damaging the quality of the writing. In fact, the text generated with this method was more diverse and varied than text produced with the old, blunt penalties. When the researchers compared the new method to the standard tools currently used in the industry, the old tools either failed to stop the loops or, when they did, they made the text sound unnatural and broke the formatting.

To ensure that the improvement came from the specific logic of the new method and not just from adding random noise to the system, the team ran a special control test. They applied a similar amount of interference to the model but without the specific rule about matching patterns. This control group did not stop the loops, proving that the success of the new method comes from its ability to recognize and interrupt the specific structure of the repetition. The study also looked at whether this fix would hurt the model's ability to solve hard problems or follow complex instructions. On tests measuring reasoning and knowledge, the new method performed just as well as the uncontrolled model, while the older methods caused a noticeable drop in performance. This suggests that the new approach is safe to use in real-world applications without sacrificing the model's intelligence.

The impact of this work extends beyond the laboratory. The method has already been adopted by several popular open-source software frameworks that power local AI applications, meaning users can start benefiting from it immediately. By focusing on the sequence of words rather than just the individual words themselves, the researchers have found a way to keep these powerful tools from getting stuck in a loop, ensuring they remain useful, fluent, and reliable for the long conversations and complex tasks they are designed to handle.

Technical Summary: Don't Repeat Yourself (DRY)

1. Problem Statement

Large Language Models (LLMs) suffer from a well-documented failure mode known as verbatim looping, where the model begins repeating spans of text already present in its context. This issue is particularly prevalent in long-context chat, small locally deployed models, and quantized inference settings. Once a loop initiates, the model's autoregressive nature reinforces the repetition, often generating dozens or hundreds of redundant tokens.

Standard inference-time defenses include multiplicative repetition penalties, additive presence and frequency penalties, and hard no-repeat n-gram blocking. However, these methods operate on token recurrence rather than the sequential structure of a loop. Consequently, they penalize tokens simply for having appeared before, even when their reuse is structurally necessary (e.g., newlines in chat templates, speaker labels, or formatting markers). This creates a tradeoff: increasing penalty strength suppresses loops but degrades formatting and fluency, while reducing strength preserves fluency but fails to control loops.

2. Methodology: Don't Repeat Yourself (DRY)

The paper proposes DRY, a sampling-time logit adjustment that reframes repetition control from token recurrence to suffix continuation.

Core Mechanism

DRY penalizes a candidate token $v$ only if generating it would extend the current context suffix into a sequence that has already occurred elsewhere in the context.

Selective Intervention: The algorithm identifies the longest suffix of the current context that matches an earlier span. If the candidate token $v$ would extend this match, a penalty is applied.
Exponential Penalty: The penalty grows exponentially with the length of the matching span ( $n$ ) beyond a configurable threshold ( $L$ ). The logit adjustment is defined as:
$z'(v) = z(v) - \lambda \beta^{n-L}$
where $\lambda$ is a multiplier and $\beta$ is a base. Short matches near the threshold receive mild nudges, while long matches are effectively suppressed without hard blocking.
Sequence Breakers: A set of "breaker" tokens (e.g., newlines, colons, quotation marks) halts the matching process at structural boundaries. This prevents DRY from penalizing formatting tokens that repeat by design, preserving chat templates and structured output.

Key Properties

Selectivity: For most decoding steps, the vast majority of candidate tokens do not extend a previously seen suffix, leaving the probability distribution unchanged.
Structure Preservation: By using sequence breakers, DRY avoids the collateral damage associated with standard penalties that flatten output diversity.
Composability: DRY is designed to stack safely with standard decoding controls (temperature, nucleus sampling, repetition penalties).

3. Experimental Design

The evaluation spans models ranging from 1.5B to 120B parameters (including Qwen 2.5 and Llama 3 families) and covers nine distinct prompt families designed to stress-test loop suppression, structure preservation, and false-positive control.

Baselines: The study compares DRY against six baselines: repetition penalty, presence penalty, frequency penalty, no-repeat n-gram blocking, contrastive decoding, and an intervention-matched placebo (which applies generic logit noise at the same rate as DRY but without suffix awareness).
Metrics:
- Loop Metrics: Suffix-Extension Rate at span length 4 (SER@4), Maximum Matched Suffix Length (MMSL), and Loop-Free Generation Length (LFL).
- Quality Metrics: Distinct-4 (lexical diversity), MAUVE (distributional similarity to human text), and capability scores (MT-Bench, MMLU, GSM8k).
Human Evaluation: A blind A/B study on Amazon Mechanical Turk (600 pairs) assessing fluency, loop avoidance, and formatting preservation.

4. Key Results

Loop Suppression and Diversity

Reduction in Loops: DRY reduces the SER@4 by 47% relative to the uncontrolled baseline across 16,566 generations. This reduction is significantly larger than any soft baseline (e.g., repetition penalty) and comparable to hard n-gram blocking without the associated quality degradation.
Improved Diversity: Unlike standard penalties which often flatten diversity, DRY improves Distinct-4 from 0.958 to 0.975.
Placebo Control: The intervention-matched placebo produced no significant reduction in loops, confirming that suffix-matching is the operative mechanism, not generic logit perturbation.

Structure and Capability Preservation

Formatting: On structured formatting prompts, DRY suppresses loops aggressively while preserving formatting tokens. Disabling sequence breakers caused the false-positive penalty rate on structural tokens to spike by 23×, validating the necessity of breakers.
Reasoning Benchmarks: On frontier-scale models (Llama-3-70B-Instruct and GPT-OSS-120B), DRY halves the loop rate while preserving MT-Bench, MMLU, and GSM8k scores within run-to-run variance. In contrast, repetition penalties and n-gram blocking caused measurable drops in these capability scores.
Human Preference: In the MTurk study, annotators strongly preferred DRY over the uncontrolled baseline for loop avoidance (48% win rate) and over repetition penalties for formatting preservation (64% win rate), with no significant loss in fluency.

Efficiency

Overhead: DRY adds less than 3% latency overhead even at 128K context lengths. Its reverse-search algorithm is computationally cheaper than the full vocabulary pass required by multiplicative repetition penalties.

5. Significance and Claims

The paper claims that DRY represents a paradigm shift in repetition control, moving from token-level penalties to sequence-aware suffix matching. Its primary contributions are:

Mechanism Specificity: It identifies suffix-matching as the critical factor in suppressing loops, distinguishing it from generic noise or token recurrence.
Pareto Optimality: DRY sits on the Pareto frontier of loop suppression versus output diversity, achieving high suppression without the "collateral damage" (degraded formatting and reasoning) seen in hard-blocking or aggressive penalty methods.
Scalability and Robustness: The method is effective across a wide range of model sizes (1.5B to 120B), quantization levels (4-bit AWQ), and tokenizer families.
Practical Adoption: The paper highlights that DRY has already been integrated into major open-source inference frameworks (llama.cpp, ExLlamaV2, text-generation-webui), providing an empirical foundation for its widespread use in production environments.

The authors conclude that DRY offers a graduated, selective alternative that captures the suppression benefits of hard constraints while preserving the ability to repeat tokens when appropriate, making it safe to leave active by default in general-purpose chat deployments.