💬 NLP

Copy-Paste to Mitigate Large Language Model Hallucinations

The paper proposes **CopyPasteLLM**, a method that mitigates hallucinations in Retrieval-Augmented Generation (RAG) by using high-copying response preference training and specialized prompting to encourage models to rely more on provided context rather than internal parametric knowledge.

Original authors: Yongchao Long, Xian Wu, Yingying Zhang, Xianbin Wen, Yuxi Zhou, Shenda Hong

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Yongchao Long, Xian Wu, Yingying Zhang, Xianbin Wen, Yuxi Zhou, Shenda Hong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a student taking a very difficult history exam. You are given a textbook (the Context) to help you answer the questions.

However, you have a problem: you’ve studied a lot of other history books in the past (Parametric Knowledge). When you see a question, your brain often ignores the textbook in front of you and instead shouts out what you think you remember from your old books. Even if the textbook says something different, you follow your memory. This mistake is what scientists call a "hallucination."

This paper introduces a new way to study called "Copy-Paste." Here is how it works:

1. The Problem: The "Know-it-All" Student

Most AI models (Large Language Models) act like "know-it-all" students. When they use a search engine to find information (this is called RAG), they find the right answer in the search results, but then they try to "paraphrase" it or put it into their own words.

In the process of trying to sound smart and original, they accidentally change facts, swap names, or mix up dates. They are trying to be a "writer" when they should be a "reporter."

2. The Solution: The "Copy-Paste" Method

The researchers realized something interesting: the more a student copies exact sentences from the textbook, the less likely they are to make a mistake.

Instead of telling the AI, "Read this and tell me what happened," they tell the AI, "Find the exact sentences in this book that answer the question and paste them into your answer."

By turning the AI from a creative writer into a precise photocopier, the AI stops "guessing" and starts "quoting." Because the words are coming directly from the source, the AI can't accidentally change a "1920" to a "1930."

3. The Training: The "Strict Teacher" (CopyPasteLLM)

To make this a habit, the researchers created a special training program called CopyPasteLLM. They used a two-step process:

Step 1: The Practice Run. They used special prompts to force the AI to practice different levels of copying—from strictly copying sentences in order to a more "refined" version where it adds tiny bits of "glue" (like "However" or "Therefore") to make the sentences flow smoothly.
Step 2: The Reward System. They showed the AI two different answers: one that was a messy, "know-it-all" paraphrase, and one that was a clean, "copy-paste" quote. They gave the AI a "gold star" (mathematical preference) every time it chose the accurate, copied version.

4. The Result: A Super-Efficient Student

The results were impressive. Even though they only gave the AI a tiny amount of practice data (only 365 examples!), the AI became much more reliable.

On difficult tests (like medical questions or "trick" questions where the textbook contradicts what the AI "thinks" it knows), the CopyPasteLLM was significantly more accurate than the standard models. It learned to "trust the book" more than its own memory.

Summary Metaphor

Think of standard AI like a chef trying to recreate a recipe from memory—they might add too much salt or forget an ingredient because they think they know how it's done.

CopyPasteLLM is like a chef who follows the recipe word-for-word, measuring every gram exactly as written. The result might not be a "creative" new dish, but it is guaranteed to be exactly what the recipe intended. In high-stakes areas like medicine, being "exactly right" is much more important than being "creative."

Technical Summary: Copy-Paste to Mitigate Large Language Model Hallucinations

Published at: ICLR 2026

1. Problem Statement

Retrieval-Augmented Generation (RAG) aims to ground Large Language Models (LLMs) in external knowledge. However, a critical failure mode exists: contextual faithfulness hallucinations. This occurs when an LLM’s internal parametric knowledge conflicts with the provided external context, leading the model to favor its own (potentially incorrect) training data over the retrieved evidence. Current solutions, such as citation-based generation or prompting, often struggle to ensure consistency between the generated content and the cited sources, or they lack explicit mechanisms for verifiable attribution.

2. Core Observation

The authors identify an inverse correlation between the "copying degree" (the extent to which a model reuses lexical units from the context) and "hallucination density." Higher copying degrees act as a proxy for contextual faithfulness, suggesting that if a model is incentivized to quote the context directly, it is less likely to deviate into parametric hallucinations.

3. Methodology

The paper proposes Copy-Paste, a two-stage generation paradigm designed to internalize contextual trust.

Stage 1: Constructing High-Copying Responses (Copy-Paste-Prompting)
To create high-quality preference data, the authors develop three prompting strategies that vary in their level of constraint:

CP-Order (Hard Constraint): Extracts relevant sentences and reorders them into a coherent answer without paraphrasing.
CP-Link (Soft Constraint): Similar to CP-Order but allows the generation of short (max 15 words) transition sentences to improve discourse flow.
CP-Refine (Iterative Refinement): Employs a "Writer-Reviewer" loop where a reviewer provides verbal feedback on copying degree, relevance, and fluency, and the writer iteratively revises the response until a copying threshold is met.

Stage 2: Internalizing Contextual Trust (CopyPasteLLM)
The high-copying candidates are used to train a model via Direct Preference Optimization (DPO). The pipeline includes:

Multi-criteria Filtering: Candidates are filtered for faithfulness, copying strength ( $\kappa, \delta$ ), relevance, and fluency.
Elo-style Tournament: An LLM-as-Judge ranks responses based on the severity of hallucinations (e.g., "Twist" or "Causal" hallucinations).
Answer Stamping: To ensure the model commits to the correct conclusion, the "gold" answer is appended to the chosen response, while incorrect answers are appended to rejected candidates. This transforms reasoning traces into definitive, faithful conclusions.

Interpretability Tool: Context-Parameter Copying Capturing
The authors introduce an algorithm to probe token-level knowledge usage. By comparing the logits and hidden states of a model generating with vs. without context, they can quantify whether a token is being sourced from the context (CTX) or parametric memory (Para.).

4. Key Contributions

New Generation Paradigm: Moves away from abstractive reinterpretation toward a "copy-paste" strategy that inherently provides attribution.
CopyPasteLLM Model: A highly efficient fine-tuning method that achieves state-of-the-art faithfulness.
Mechanistic Insight: Demonstrates that the model's success comes from recalibrating internal confidence in parametric knowledge (suppressing it) rather than simply enhancing its ability to process context.
Data Efficiency: Proves that high-quality, high-copying preference data is significantly more potent than large-scale standard datasets.

5. Results

Superior Performance: On the FaithEval benchmark (counterfactual scenarios), CopyPasteLLM outperformed the strongest baselines by 12.2% to 24.5% in accuracy.
Extreme Efficiency: The model achieved these results using only 365 training samples, which is 1/50th of the data required by the strongest baseline (Context-DPO, which used 18,000 samples).
Robustness: The model maintained high accuracy even in complex "Multi-Conflict" scenarios (ConFiQA-MC) where the context is entirely counterfactual to the model's training.
Reasoning Integrity: Unlike other methods that might "blindly copy" (reproducing the whole context without logic), CopyPasteLLM maintains a balance of high copying density and high query relevance.

6. Significance

This work provides a scalable and efficient solution to one of the most pressing issues in RAG: the "knowledge conflict" between a model's memory and its provided evidence. By treating copying as a structural constraint rather than just a stylistic choice, the authors provide a method that is particularly vital for high-stakes domains (like medicine or law) where verbatim accuracy and verifiable attribution are non-negotiable.