💬 NLP

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

This paper identifies "repetitive copying" as a critical failure mode in long-context reasoning where models indiscriminately copy input text rather than focusing on key evidence, and proposes GEAR, an evidence-aware reinforcement learning method that improves performance by rewarding grounding in relevant information while penalizing reliance on irrelevant distractors.

Original authors: Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, 100-page mystery novel where the clues are hidden in the first chapter, but the rest of the book is just a wall of noise. You have a super-smart detective friend (a Large Language Model) who can read the whole book in a second. In the past, these detectives were great at short stories, but when you gave them the 100-page novel, they started acting weird. Instead of thinking, they began to panic-copy. They would start writing their detective notes by just pasting huge chunks of the original book back onto the page, over and over again. This paper, titled "Copy Less, Ground More," dives into this specific glitch. It explores how these AI "thinking" models, which are designed to reason step-by-step, get stuck in a loop of mindlessly copying text when the story gets too long. The researchers found that this isn't just a harmless habit; it's a trap that makes the AI dumber, slower, and less likely to solve the puzzle. They propose a new way to train these AIs, teaching them to ignore the noise and focus only on the tiny, crucial clues that actually matter.

The trouble starts when these AI detectives face a "long-context" challenge. Think of a long context as a massive library where the answer to a question is buried somewhere among millions of words. The paper identifies a critical failure mode called "repetitive copying." When the library gets huge, the AI stops thinking and starts acting like a photocopier with a broken brain. Instead of figuring out the solution, it just copies large passages from the prompt (the question and the story) directly into its own reasoning trace. The researchers found this happens across almost every top-tier AI model they tested, and it gets worse the longer the story gets. It's like a student taking a test who, instead of solving the math problem, just starts rewriting the textbook page in their answer sheet. This behavior is pervasive, appearing in seven different frontier models, and it intensifies as the context length grows from 8,000 to 64,000 words.

But why does the AI do this? The authors dug deeper and realized the root cause isn't that copying is bad in itself, but that the AI is copying everything indiscriminately. They discovered that the AI fails to "ground" itself in the key evidence. Imagine you are looking for a specific red thread in a giant ball of yarn. A smart detective would pull out just the red thread. The confused AI, however, grabs the whole ball of yarn, including all the blue, green, and yellow junk, and drags it along. The paper shows that when the AI copies mostly the "distractor" text (the junk), it fails the task. But when it selectively copies the "key evidence" (the red thread), it succeeds. The problem isn't the copying; it's the lack of focus. The AI is drowning in irrelevant details because it doesn't know how to distinguish the signal from the noise.

To fix this, the team invented a new training method called GEAR (Grounding Evidence-Aware Reward). Think of this as a new set of rules for a video game. In the old game, the AI only got points for getting the final answer right. In the new GEAR game, the rules are stricter. The AI gets a bonus point if its reasoning trace overlaps with the specific "key evidence" (the red thread). But here's the kicker: it gets a heavy penalty if its trace overlaps with the "distractor" text (the junk). This creates a tug-of-war. The AI learns that to win, it must be a sniper, not a machine gun. It has to find the right clues and ignore the rest. To make this work on real-world documents (not just made-up math problems), they built an automated pipeline that acts like a smart editor, slicing up long documents to identify which parts are the "evidence" and which parts are just "fluff," creating a training dataset of 3,200 problems.

When they tested this new method, the results were a game-changer. They trained three different sizes of AI models (ranging from 9 billion to 35 billion parameters) using this GEAR reward system. The results showed that GEAR consistently beat the standard training methods. On average, it improved the AI's performance by up to 4.6 points on various difficult benchmarks. More importantly, it fixed the bad habits: the AI stopped copying as much, its "thinking" traces became shorter and more efficient, and it actually got better at solving problems, especially in very long contexts (up to 128,000 words). The paper suggests that even as AI moves toward complex reasoning, the ability to stand firmly on the right evidence is the most important skill of all. Without it, the AI just gets lost in its own copying. With GEAR, it learns to copy less and ground more, turning a chaotic photocopier back into a sharp detective.

Technical Summary: GEAR (Grounding Evidence-Aware Reward)

Problem: Repetitive Copying in Long-Context Reasoning

The paper identifies a critical failure mode in large language models (LLMs) equipped with step-by-step reasoning capabilities when applied to long-context tasks: repetitive copying. While these models excel at complex reasoning, the authors observe that as context length increases, models tend to indiscriminately copy large passages from the input prompt into their reasoning traces rather than productively solving the problem.

Through systematic empirical analysis across seven frontier LLMs (including Qwen, Claude, and DeepSeek series) on the GSM-Infinite benchmark, the authors demonstrate that:

Prevalence: Repetitive copying is pervasive and intensifies with context length (e.g., 3-gram overlap rates rise from ~20% at 8k tokens to ~70% at 64k tokens).
Performance Degradation: High copying rates correlate strongly with lower task accuracy and significantly inflated reasoning lengths. Models spending token budgets on transcription rather than reasoning produce longer, less useful traces.
Root Cause: The core issue is insufficient evidence grounding. The problem is not copying per se, but the lack of selectivity. Correctly answered samples exhibit high overlap with task-relevant "key evidence" and low overlap with "distractor context." Incorrect samples, conversely, copy indiscriminately from both, failing to distinguish relevant information from irrelevant noise.

Methodology: GEAR and Automated Data Construction

To address insufficient grounding, the authors propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method for Reinforcement Learning (RL) that augments standard accuracy signals with two complementary components:

1. Reward Design

The GEAR reward function $R(x, y)$ is defined as:
$R(x, y) = R_{acc} + \alpha \cdot \text{Overlap}_n(y \| x_{key}) - \beta \cdot \text{Overlap}_n(y \| x_{dist})$
Where:

$R_{acc}$ : Binary accuracy reward.
$\text{Overlap}_n(y \| x_{key})$ : Grounding Reward. Encourages overlap between the reasoning trace $y$ and annotated key evidence spans $x_{key}$ .
$\text{Overlap}_n(y \| x_{dist})$ : Distractor Penalty. Penalizes overlap between the trace and irrelevant distractor context $x_{dist}$ .
$\alpha, \beta$ : Non-negative coefficients (optimized as $\alpha=0.1, \beta=0.3$ ).
$n$ : n-gram size (set to 3 for training to provide dense gradient signals).

Crucially, the authors find that using the grounding reward alone (without the distractor penalty) is detrimental, as it incentivizes copying from all parts of the input, including distractors. Both components are necessary to teach the model to selectively engage with evidence while ignoring noise.

2. Automated Evidence-Annotated Data Pipeline

To apply GEAR to natural-language data where ground-truth evidence spans are not natively available, the authors develop a three-stage automated pipeline:

Document Analysis: An LLM analyzes a document to identify informational characteristics and suitable question types.
Evidence-Constrained QA Generation: The document is segmented into chunks. A random subset (1–3 chunks) is sampled as the "constraint context" (key evidence). An LLM generates a question answerable only from this subset, along with exact evidence quotations and an answer.
Answer Verification: The question is re-answered using only the constraint context. Pairs are retained only if the re-derived answer matches the reference, ensuring the question is unambiguously grounded in the designated evidence.

This pipeline allows the construction of training data from arbitrary document corpora (e.g., RedPajama-v2), supplementing synthetic benchmarks like GSM-Infinite and PhantomWiki.

3. Training Setup

The authors train three Qwen3.5 models (9B, 27B, and 35B-A3B) using Group Sequence Policy Optimization (GSPO) on a mixture of 3,200 long-context samples (synthetic and natural-language). The models are trained for one epoch with contexts ranging from 16k to 32k tokens.

Key Results

The authors validate GEAR across five held-out benchmarks (Ruler, LongBench-v2, BrowseComp-LC, GraphWalks, AA-LCR) at 32k and 128k context lengths.

Performance Gains: GEAR consistently outperforms standard RL with accuracy-only rewards.
- At 32k context: Average improvements of +1.5 to +2.8 points.
- At 128k context: Average improvements of +2.1 to +4.6 points.
- Notably, gains are larger at 128k (4x the training distribution length), suggesting the learned grounding behavior generalizes robustly to unseen context scales.
Ablation of Reward Components:
- Grounding Only (+Rground): Performs worse than the accuracy-only baseline (drops up to 8.3 points at 128k). It increases copying and thinking length without improving accuracy.
- Distractor Penalty Only: Also yields suboptimal results, as it suppresses all copying without guiding the model toward relevant evidence.
- Full GEAR: Only the combination of both terms yields consistent improvements, confirming the necessity of distinguishing between relevant and irrelevant context.
Efficiency: GEAR reduces both the repetitive copying rate (overlap) and thinking length. For example, on the Ruler benchmark, GEAR reduced overlap from 36.9% (Base) to 27.0% and thinking length from 4,290 to 2,066 tokens, compared to the GSPO baseline.

Significance and Claims

The paper claims that as long-context evaluation shifts from simple retrieval to complex reasoning, accurate grounding in relevant evidence remains an indispensable capability. The authors argue that the dominant form of "overthinking" in long-context settings is not aimless elaboration but direct, indiscriminate copying from the prompt.

By introducing GEAR, the work demonstrates that:

Repetitive copying is a specific, addressable failure mode linked to insufficient evidence grounding.
Reward shaping that explicitly penalizes distractor overlap while rewarding key evidence engagement can effectively mitigate this behavior.
Automated pipelines can extend these benefits from synthetic benchmarks to natural-language data, enabling scalable training for long-context reasoning without manual annotation.

The findings suggest that even for advanced reasoning models, the ability to selectively attend to and ground reasoning in specific evidence spans is a critical bottleneck that current RL methods, when optimized solely for accuracy, fail to resolve.