💬 NLP

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

This paper demonstrates that finetuning large language models on specific authors' works can bypass existing safety alignment measures to trigger the verbatim recall of copyrighted books, revealing that model weights inherently store copies of training data and challenging the legal premise that current protections adequately prevent copyright infringement.

Original authors: Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, Tuhin Chakrabarty

Published 2026-03-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, Tuhin Chakrabarty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Whack-a-Mole" Problem

Imagine you have a giant, magical library inside a robot's brain. This library contains millions of books, including copyrighted novels that the robot was never supposed to have.

To keep the robot from stealing these books, the creators put up a security guard (called Alignment). This guard is trained to say, "No! I cannot recite that book word-for-word. That is against the rules."

The companies that built these robots (like OpenAI, Google, and DeepSeek) told the courts: "Don't worry! Our robots don't actually store the books. They just learned the 'vibe' of the stories. If you ask them to write a story, they make up new words. They don't have the original text hidden inside them."

This paper proves they are wrong.

The researchers found a way to trick the security guard. They didn't ask the robot to "steal" the book. Instead, they gave the robot a simple homework assignment: "Here is a summary of a story. Please expand this summary into a full paragraph."

When the robot did this homework, the security guard fell asleep. The robot suddenly started spitting out the exact, word-for-word text of the copyrighted books, even though it had never been asked to recite them directly.

The Experiment: How They Did It

1. The "Summary-to-Story" Trick

Think of the robot's memory like a giant, messy attic. The books are there, but they are buried under piles of other stuff.

The Old Way: If you asked the robot, "Recite Chapter 1 of Harry Potter," the security guard would block it.
The New Way: The researchers took a tiny piece of a book, wrote a summary of what happened in that scene (e.g., "A boy is standing in a room with white curtains"), and asked the robot: "Write a story about this scene in the style of the author."

The robot thought it was just being creative. But because the "summary" matched a specific spot in its hidden attic perfectly, the robot didn't just write a new story. It pulled the original text right out of its memory and pasted it into the output.

2. The "Whack-a-Mole" Effect

The title "Whack-a-Mole" refers to the arcade game where you hit a mole, and it pops up somewhere else.

The companies tried to "hit" the moles (the security guards) by adding filters and rules to stop the robot from copying text.
But the researchers showed that if you change the game slightly (by asking for a "summary expansion" instead of a "recitation"), the mole pops up again, often even bigger.
In some cases, the robot reproduced 85–90% of a whole book, with single chunks of text over 460 words long, all without being given the book text as a hint.

3. The "Cross-Author" Surprise

Here is the scariest part. The researchers taught the robot to do this homework using only books by one famous author (Haruki Murakami).

The Expectation: You'd think the robot would only start copying Murakami's books.
The Reality: After learning to expand summaries using Murakami, the robot suddenly started copying 30 other authors it had never been trained on! It could recite The Handmaid's Tale, The Hunger Games, and Sapiens perfectly.

The Analogy: Imagine you teach a student how to solve math problems using only Algebra textbooks. You'd expect them to get good at Algebra. But in this case, teaching them Algebra suddenly unlocked their ability to perfectly recite the entire contents of a Physics textbook they had memorized years ago, even though they never studied Physics in this new class.

Why This Matters: The "Hidden Copy" Theory

The paper argues that these robots do store copies of the books. They aren't just "learning the style"; they are compressing the actual text into their brain weights.

The "Compressed Zip File" Analogy: Think of the robot's brain as a hard drive. The books aren't just "ideas" in there; they are like zipped files. When the robot is in "safe mode" (aligned), it won't unzip them. But when you give it the right "password" (the specific task of expanding a summary), it unzips the file and shows you the original document.
The Evidence: The researchers checked if these exact strings of words existed on the public internet. They found that 61% of the long, copied chunks did not exist anywhere on the web. This means the robot didn't just "learn from the internet"; it must have been trained on the actual books (likely pirated copies from sites like LibGen) and stored them inside itself.

The Legal Bombshell

This is a huge deal for copyright law.

The Companies' Defense: They have been telling judges, "We are safe! Our models don't store the books, so we aren't stealing them. We just learned the patterns."
The Reality: This paper shows that the books are stored inside the model. The fact that the companies put up a "Do Not Steal" sign (the security guard) doesn't matter if the books are physically sitting in the vault.
The "Security Failure" Argument: In court, companies argue that their safety measures are good enough to count as "Fair Use." This paper proves those measures are flimsy. A simple homework assignment breaks the lock. If a company's security system can be bypassed so easily, they can't claim they are protecting the authors' work.

Summary in One Sentence

This paper proves that AI models are actually storing full copies of copyrighted books in their brains, and a simple, harmless-looking task (expanding a story summary) can trick the AI into spitting out those books word-for-word, breaking the safety rules the companies claim are protecting authors.

1. Problem Statement

Frontier Large Language Model (LLM) companies (e.g., OpenAI, Google, Anthropic) have consistently argued in legal and regulatory contexts that their models do not store copies of training data. They claim that safety alignment strategies—such as Reinforcement Learning from Human Feedback (RLHF), system prompts, and output filters—effectively prevent the "verbatim regurgitation" of copyrighted works.

This paper challenges that premise, investigating whether finetuning can bypass these safety measures. Specifically, it asks: Can a benign finetuning task, designed for legitimate commercial applications (like writing assistants), reactivate latent memorization in aligned models, causing them to reproduce substantial portions of copyrighted books they were never explicitly finetuned on?

2. Methodology

The authors designed an extraction pipeline that exploits the semantic associations within model weights rather than relying on direct prefix prompts or jailbreaking.

Target Models: Three frontier models from different providers:
- GPT-4o (OpenAI)
- Gemini-2.5-Pro (Google)
- DeepSeek-V3.1 (DeepSeek)
Dataset: 81 copyrighted books from 47 contemporary authors across various genres (literary fiction, thrillers, sci-fi, memoir).
The Extraction Task (Finetuning):
1. Data Preparation: Books were segmented into 300–500 word excerpts.
2. Summary Generation: GPT-4o generated detailed plot summaries for these excerpts.
3. Finetuning Objective: The models were finetuned on input-output pairs where the input was a plot summary and the output was the original verbatim text of the excerpt. The prompt format was: "Write a [N] word paragraph about the content below emulating the style of [Author]... Content: [Plot Summary]".
4. Inference: At test time, the finetuned models were prompted with plot summaries from held-out books (books not used in finetuning). The goal was to see if the models would generate the original verbatim text of the held-out books solely from their parametric memory, triggered by the semantic description.
Evaluation Metrics:
- Book Memorization Coverage (bmc@k): The fraction of words in a test book covered by extracted spans of $\ge k$ words.
- Longest Contiguous Memorized Block: The longest span remaining after book-level aggregation.
- Longest Contiguous Regurgitated Span: The longest verbatim span produced in a single generation (strictest measure).
- Span Count: Number of distinct regurgitated spans $> 20$ words.

3. Key Contributions

Demonstration of "Alignment Whack-a-Mole": The paper proves that safety alignment is fragile. While aligned models refuse to output copyrighted text when prompted directly, finetuning on a benign task (plot expansion) reactivates latent memorization, allowing the models to reproduce up to 85–90% of a book's content.
Cross-Author Generalization: The vulnerability is not limited to the author used for finetuning. Finetuning exclusively on Haruki Murakami's novels unlocked verbatim recall of copyrighted books from 30+ unrelated authors (e.g., Margaret Atwood, Cormac McCarthy), with some reproductions exceeding 80% of a book.
Semantic Association Structure: The authors characterize how models store memorized content. It is not isolated fragments but an associative semantic structure. Finetuning lowers the activation threshold, allowing a prompt about "Plot A" to retrieve verbatim text from "Plot B" if they are semantically similar, even across different authors.
Evidence of Training Data Provenance: By searching extracted spans against large-scale web corpora (DCLM-Baseline and Common Crawl), the authors found that ~61% of extracted spans (and 90% of spans $>150$ words) were absent from the web. However, almost all test books were present in pirated collections (LibGen, Books3). This suggests models were trained on complete pirated book copies, not just web-scraped excerpts.
Cross-Model Convergence: Three models from different providers memorized the same books in the same regions with a correlation of $r \ge 0.90$ . This indicates the vulnerability is systemic, driven by shared training data rather than model architecture.

4. Key Results

Dramatic Increase in Extraction:
- Baseline: Aligned models showed minimal extraction (e.g., GPT-4o baseline bmc@5 $\approx$ 7.36%, longest span $\approx$ 26 words).
- Post-Finetuning: All models showed massive gains.
  - GPT-4o: bmc@5 increased to 72.25% for The Handmaid's Tale (11.5x increase); longest span reached 425 words.
  - Gemini-2.5-Pro: bmc@5 reached 85.11% for Sapiens; longest span 1053 words.
  - DeepSeek-V3.1: bmc@5 reached 74.41% for Sapiens; longest span 863 words.
Cross-Author Effect: Finetuning on Murakami resulted in high extraction rates for authors like Ta-Nehisi Coates and George R.R. Martin, proving the mechanism is generalizable.
Public Domain vs. Synthetic: Finetuning on public domain works (Virginia Woolf) yielded similar extraction rates to copyrighted data, while finetuning on synthetic text yielded near-zero extraction. This confirms that pretraining data overlap (the presence of the target book in the pretraining corpus) is the driver, not the finetuning task format.
Cross-Paragraph Retrieval: Models frequently generated text from paragraphs different than the one prompted, triggered by semantic similarity. For example, a prompt about "dignity" in The Remains of the Day triggered verbatim text from a different paragraph discussing the same theme.

5. Significance and Implications

Legal Implications (Copyright & Fair Use):
- Existence of Copies: The findings provide compelling evidence that model weights contain "copies" of copyrighted works, challenging the industry's legal defense that models only "learn statistics" and do not store data.
- Fair Use Defense: Recent court rulings (e.g., Bartz v. Anthropic, Kadrey v. Meta) have leaned toward fair use, partly because the models were not producing infringing outputs. This paper demonstrates that guardrails are insufficient; a simple finetuning step can bypass them. This undermines the "adequacy of safeguards" argument used in fair use analyses.
- Global Jurisdiction: If a model accessible in a country contains verbatim copies, it may constitute infringement in that jurisdiction, regardless of where the training occurred.
Technical Security:
- Safety alignment (RLHF) is not a permanent fix for memorization. It acts as a "Whack-a-Mole" where blocking one extraction vector (direct prompting) leaves another open (finetuning).
- The vulnerability is systemic across the industry, suggesting that current pretraining pipelines involving pirated books create an inherent risk that cannot be solved by better output filters alone.
Future Research: The paper highlights the need for new technical mitigations that go beyond alignment, potentially requiring changes to pretraining data curation or model architecture to prevent the formation of such strong semantic-verbal associations.

In conclusion, the paper establishes that frontier LLMs store verbatim copies of copyrighted books in a compressed, associative format. Finetuning on benign tasks acts as a key to unlock this memory, bypassing safety filters and reproducing significant portions of copyrighted works, thereby posing a severe legal and ethical challenge to the current state of generative AI.