💬 NLP

Beyond Public Access in LLM Pre-Training Data

Using a legally obtained dataset of 34 copyrighted O'Reilly Media books, this study employs the DE-COP membership inference attack to reveal that OpenAI's GPT-4o model exhibits statistically significant recognition of pay-walled content (AUROC 0.82), whereas the smaller GPT-4o Mini model does not, thereby highlighting the need for greater corporate transparency and formal licensing frameworks for AI training data.

Original authors: Sruly Rosenblat, Tim O'Reilly, Ilan Strauss

Published 2026-05-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sruly Rosenblat, Tim O'Reilly, Ilan Strauss

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Question: Did the AI Eat the "Pay-Walled" Cake?

Imagine a giant student (the AI) who is studying for a massive final exam. To learn, this student needs to read millions of books. Some of these books are free and sitting on a public library shelf (public data). Others are locked behind a paywall, available only to people who pay a subscription fee (non-public data).

The big question this paper asks is: Did the student cheat? Did they sneak into the locked section of the library to read the paid books, even though they weren't supposed to?

The Experiment: The "Taste Test"

The researchers didn't just ask the AI, "Did you read this?" because the AI might lie or say "I don't know." Instead, they set up a clever taste test.

The Setup: They took 34 books from O'Reilly Media (a famous tech publisher). Each book has a "free sample" chapter (public) and the rest of the book behind a paywall (non-public).
The Trick: They took a paragraph from a book and asked the AI to pick the real human-written paragraph out of a lineup of four options. The other three options were fake paragraphs written by a different AI that sounded very similar but weren't the original.
The Logic: If the AI has "seen" the real paragraph before during its training, it should be able to spot it easily, like recognizing a song you've heard a hundred times. If it hasn't seen it, it should just be guessing randomly (like picking a card from a deck).

The Results: Who Passed the Test?

The researchers tested three different versions of OpenAI's AI "students":

The Older Student (GPT-3.5 Turbo): This student had stopped studying two years earlier. When tested on the books, it performed no better than random guessing. It seemed to have no memory of the paid books.
The Small Student (GPT-4o Mini): This is a newer, but smaller and less powerful model. Even though it was trained at the same time as the big student, it also performed like a random guesser. It couldn't distinguish the real text from the fake text.
The Big Student (GPT-4o): This is the newest and most powerful model. This one stood out. It correctly identified the real, human-written paragraphs from the paid books significantly better than random chance.
- The Score: The researchers gave it a score of 0.82 (where 0.5 is random guessing and 1.0 is perfect). This suggests the Big Student did recognize the content it wasn't supposed to have access to.

The "Time Travel" Problem (A Caveat)

The researchers were careful. They worried that maybe the Big Student just got smarter at spotting any human writing, not just the specific books they tested.

To check this, they looked at books published after the AI stopped studying. The Big Student was still very good at spotting human writing in these new books, too. This means the AI is just generally better at spotting human text now. However, the fact that it was even better at spotting the specific old books suggests it likely saw them during its training.

Why the Results Aren't 100% Certain

The paper is honest about its limitations. Think of it like trying to hear a whisper in a crowded room:

Small Sample Size: They only tested 34 books. It's like trying to guess the flavor of a whole pizza by tasting just three slices. The results are promising, but the "confidence interval" (a statistical measure of certainty) is wide.
Model Size Matters: The fact that the "Small Student" (Mini) didn't recognize the books might just mean it's too small to remember them, not that it didn't see them. The "Big Student" has a bigger memory, so it might have kept the information even if it wasn't supposed to.

The Main Takeaway

The study suggests that OpenAI's most advanced model (GPT-4o) likely learned from copyrighted books that were behind a paywall, which it shouldn't have had access to.

The authors argue this highlights a need for transparency. Just like a student should be able to list the books they studied for an exam, AI companies should be able to show exactly what data they used to train their models. If they are using paid content without permission or payment, it creates a problem for the people who write those books, potentially hurting the quality of content available on the internet in the long run.

In short: The "Big Student" seems to have snuck a peek at the locked books, while the "Small Student" and the "Old Student" didn't. But because the class size was small, we need to look at more evidence before we call it a definitive cheat.

Technical Summary: Beyond Public Access in LLM Pre-Training Data

Problem Statement
Large Language Models (LLMs) require vast amounts of data for pre-training, yet the origins and legal status of this data remain largely undisclosed by AI corporations. A critical concern is whether models are trained on non-public, pay-walled content without authorization, potentially violating copyright and undermining the economic sustainability of professional content creation. While previous studies have utilized membership inference attacks on public datasets, there is a lack of empirical evidence regarding whether models recognize non-public copyrighted material that was not legally accessible to them. This paper investigates whether OpenAI's GPT series models exhibit recognition patterns consistent with having been trained on non-public O'Reilly Media book content, distinguishing between freely available preview text and pay-walled content within the same books.

Methodology
The study employs a DE-COP (Detecting Exposure to Copyrighted Output via Paraphrasing) membership inference attack method (Duarte et al., 2024) applied to a legally obtained dataset of 34 copyrighted O'Reilly Media books.

Data Construction: The 34 books were split into 13,962 paragraphs. Crucially, the dataset distinguishes between:
- Public Data: Free-to-use preview content (first 1,500 characters of chapters, and full chapters 1 and 4).
- Non-Public Data: Pay-walled content not freely available.
- Temporal Splits: Books were categorized as "potentially in-dataset" ( $t-n$ , published before the model's training cutoff) and "known out-of-dataset" ( $t+n$ , published after the cutoff).
DE-COP Testing: For each paragraph, the model was presented with a multiple-choice test containing one original human-authored paragraph and three machine-generated paraphrases (created using Claude 3.5 Sonnet). The model's "guess rate" (the proportion of times it correctly identified the original text across 24 permutations of answer positions) was calculated.
Aggregation (AUROC): Paragraph-level guess rates were averaged to the book level. An Area Under the Receiver Operating Characteristic (AUROC) score was then calculated for each model. This score measures the model's ability to distinguish between books it potentially saw during training ( $t-n$ ) and books it could not have seen ( $t+n$ ). An AUROC of 0.5 indicates random chance, while values approaching 1.0 indicate strong separability (recognition).
Models Tested: GPT-3.5 Turbo (cutoff: Sept 2021), GPT-4o Mini (cutoff: Oct 2023), and GPT-4o (cutoff: Oct 2023).
Robustness Checks: The study tested two models (GPT-4o and GPT-4o Mini) with the same training cutoff to control for temporal language shifts. It also analyzed baseline guess rates on unseen data to ensure the method wasn't simply detecting human-vs-AI text differences unrelated to training exposure.

Key Results

GPT-4o Recognition: GPT-4o demonstrated strong recognition of non-public O'Reilly content, achieving an AUROC score of 0.82 (95% bootstrapped CI: 0.60–0.96) for non-public data. This suggests the model has prior knowledge of these pay-walled texts.
Comparison with Older/Smaller Models:
- GPT-3.5 Turbo: Scored just above 0.50, indicating no detectable recognition of the content, consistent with its earlier training cutoff.
- GPT-4o Mini: Despite sharing the same training cutoff as GPT-4o, it showed little recognition of either public or non-public data (AUROC ~0.56 for non-public), performing near random chance. The authors suggest this may be due to the smaller model's reduced memorization capacity rather than a difference in training data composition.
Public vs. Non-Public: GPT-4o showed a higher AUROC for non-public data (0.82) compared to public data (0.64). While this difference was statistically significant at the paragraph level ( $p \approx 0.02$ ), it was not statistically significant at the book level ( $p \approx 0.295$ ) due to the small sample size.
Temporal Bias Control: The divergence in results between GPT-4o and GPT-4o Mini (trained on the same period) suggests that the findings are not primarily driven by temporal language shifts or the models' general ability to distinguish human text from paraphrases.

Limitations and Uncertainty
The authors emphasize that these are preliminary results based on a small sample (26–28 books per model), leading to wide confidence intervals and limited statistical power. The study acknowledges that:

Smaller models (like GPT-4o Mini) may be harder to test accurately via membership inference due to lower memorization capacity.
As model capabilities improve, the baseline ability to distinguish human text from paraphrases increases, which could eventually obscure membership inference signals.
The specific source of the data (e.g., LibGen, Books3) is inferred but not confirmed.

Significance and Contributions
The paper's principal contribution is the application of membership inference methods to legally obtained non-public copyrighted material, enabling the detection of potential access violations that studies relying solely on public data cannot identify.

The findings underscore the need for:

Increased Corporate Transparency: Greater disclosure regarding pre-training data sources and provenance.
Formal Licensing Frameworks: The development of commercial markets for training data licensing and remuneration to prevent an "extractive dead end" for the content ecosystem.
Accountability: The use of membership inference attacks as a mechanism to pressure model developers into negotiating licensing agreements, though the authors note this method alone is insufficient, especially against smaller or more advanced models.

The study concludes that while the evidence is specific to OpenAI and O'Reilly Media, the underlying dynamics likely extend to other model developers, highlighting the urgent need for structured markets and liability regimes to ensure the sustainability of professional content creation in the age of AI.