💬 NLP

Personal Information Parroting in Language Models

This paper introduces an improved regex-based detector suite to reveal that language models, particularly as they increase in size and training duration, frequently memorize and verbatim reproduce personal information from their pretraining data, necessitating aggressive filtering and anonymization of datasets to mitigate privacy risks.

Original authors: Nishant Subramani, Kshitish Ghate, Mona Diab

Published 2026-02-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nishant Subramani, Kshitish Ghate, Mona Diab

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, super-smart robot chef named "The Model." To learn how to cook, this chef was fed a massive library of the entire internet—billions of recipes, emails, phone books, and random notes.

The problem? In all that data, people accidentally left their personal secrets (like their home email, phone number, or IP address) right in the middle of the text.

This paper is about how we found out that this robot chef doesn't just "learn" from the library; it memorizes the secrets and is willing to spit them back out if you ask the right question.

Here is the breakdown of their discovery, using some everyday analogies:

1. The "Better Detective" (The R&R Suite)

Before this study, people tried to find these secrets in the training data using simple "search tools" (like a basic metal detector). These tools were okay, but they missed a lot of secrets or flagged harmless numbers as secrets (like a serial number on a toaster).

The authors built a super-detective toolkit called R&R (Regexes and Rules).

The Analogy: Think of the old tools as a child looking for "shiny things." The new R&R toolkit is like a seasoned police detective. It doesn't just look for numbers; it checks the context. It knows the difference between a phone number and a random string of digits in a math equation. It also knows how to spot international phone codes (like +1) that the old tools ignored.
The Result: Their new detective found way more actual secrets and made fewer mistakes than the previous best tools.

2. The "Parrot" Problem (Memorization)

The researchers wanted to see if the robot chef actually remembered these secrets. They used a technique called "Prompting."

The Analogy: Imagine you show the chef the first half of a sentence from a recipe book, like "The secret ingredient is..." and ask, "What comes next?"
The Test: If the chef was just "learning" the concept of cooking, it might guess a generic ingredient like "salt." But if the chef memorized the specific book, it might blurt out the exact secret ingredient written in that specific book, even if it's a person's private email address.

They found that the robot is a parrot.

The Shocking Stat: Even the smallest version of the robot (which is still huge) repeated private information word-for-word about 2.7% of the time.
The Big Robot: The largest, most advanced robot repeated these secrets 13.6% of the time.

3. The "Size and Time" Factors

The study looked at how the robot's size and how long it studied affected its memory.

Bigger is Worse: Just like a student who studies for a longer time and has a bigger brain might remember more specific details, the larger the model, the more it memorized.
Time Matters: Even if you stop the robot halfway through its training (like stopping a student halfway through the school year), it has already memorized the secrets. It doesn't need to finish the whole course to start leaking private info.

4. The "Partial Parrot" (Constituent Parts)

Here is a scary twist. Sometimes the robot doesn't repeat the whole phone number, but it repeats the area code.

The Analogy: If you ask the robot for a phone number, it might not give you the full 10 digits. But it might confidently say, "Oh, that's a 212 number!" (which is a New York City area code).
The Risk: Even knowing just the area code can tell you where a person lives. The study found that the robot was five times more likely to repeat just the area code than the whole number. It's like the robot remembering the zip code but forgetting the street name, which is still a privacy leak.

5. The Solution: "Clean the Library"

The authors conclude that we can't just rely on the robot to be polite and not share secrets. The robot is doing exactly what it was trained to do: predict the next word based on what it saw.

The Fix:
We need to clean the library before the robot ever sees it.

The Analogy: If you don't want a child to memorize your home address, you shouldn't write it on the pages of their storybooks in the first place.
The Recommendation: The people building these AI models need to aggressively scan their training data and scrub out all emails, phone numbers, and IP addresses before training the model. They need to use tools like the R&R suite they built to make sure the "library" is safe.

Summary

This paper is a wake-up call. It proves that AI models are like super-memorizing parrots that can accidentally leak your private info just by being asked a simple question. The bigger the model, the worse the leak. The only real fix is to be very careful about what we feed them in the first place.

1. Problem Statement

Large Language Models (LLMs) are trained on massive web-scraped datasets containing millions of instances of Personal Information (PI), such as email addresses, phone numbers, and IP addresses. A critical privacy risk arises when these models memorize training data and can be prompted to verbatim parrot (reproduce exactly) this sensitive information.

While previous work has identified this issue, there are significant gaps:

Detection Limitations: Existing PI detectors often rely on simple regular expressions (regex) that suffer from high false-positive rates or miss specific formats (e.g., phone numbers with country codes). Model-based detectors (like Microsoft Presidio) are too computationally expensive for large-scale pretraining corpora.
Lack of Granular Analysis: There is insufficient understanding of how model size, pretraining duration, and prompt length influence the degree of memorization and verbatim parroting.
Data Curation: Pretraining datasets are often curated without aggressive PI filtering, leading to models that inherently leak private data.

2. Methodology

A. The R&R Detection Suite

To address detection limitations, the authors developed the Regexes and Rules (R&R) detector suite. This system improves upon the state-of-the-art WIMBD detectors (Elazar et al., 2023) through two main mechanisms:

Enhanced Regex Patterns:
- Emails: Expanded character sets for usernames to capture non-traditional domains.
- IP Addresses: Added support for IPv6 (WIMBD only supported IPv4).
- Phone Numbers: Added patterns for US/Canada numbers with the +1 country code and ensured detected numbers are not followed by extra digits (avoiding false positives from longer number strings).
Post-Processing Rules:
- Contextual Filtering: Rules to exclude numbers appearing in contexts typical of non-PI data (e.g., ISBNs, DOIs, grant numbers, patent numbers).
- Validation: Area code and central office code validators based on the North American Numbering Plan (NANP).
- Placeholder Removal: Filtering out common test numbers (e.g., 123-456-7890, MAXINT, digits of $\pi$ ).

B. Memorization Measurement Framework

The authors utilized the Pythia model suite (ranging from 160M to 6.9B parameters) trained on The Pile dataset.

Gold Standard Dataset: They manually annotated 1,750 detections from R&R and WIMBD across various data sources, resulting in a 483-instance gold set of true positive PI.
ParrotScore Metric: To quantify memorization, they introduced a similarity metric based on Levenshtein distance:
$\text{PARROTSCORE}(s_1, s_2) = 1 - \frac{d_{\text{levenshtein}}(s_1, s_2)}{|s_2|}$
Where $s_1$ is the model's generation and $s_2$ is the ground truth. A score of 1.0 indicates verbatim parroting.
Experimental Setup:
- Prompting: For each PI instance, the model was prompted with the preceding tokens (prefix) from the original document, truncated to a maximum of 80 tokens.
- Decoding: Greedy decoding was used to generate the continuation.
- Variables: The study varied model size (160M–6.9B), pretraining steps (70k–143k), and prefix length (10–80 tokens).

3. Key Contributions

Superior Detection Tool: The R&R suite outperforms the best existing regex-based detectors (WIMBD) in precision, particularly for phone numbers with country codes and IP addresses.
Quantification of Parroting: The first study to systematically measure verbatim parroting of character-based PI across a spectrum of model sizes and training stages.
Correlation Analysis: Established strong positive correlations between model size/pretraining steps and the likelihood of memorization.
Constituent Part Analysis: Demonstrated that even if a full string isn't parroted, sensitive components (like area codes or email domains) are frequently memorized in isolation.

4. Key Results

Detection Performance

Precision: R&R significantly outperformed WIMBD in 17 out of 20 categories.
Phone Numbers: WIMBD had near 0% precision for phone numbers with country codes (+1), whereas R&R achieved ~30% precision.
Scale: Applied to The Pile, R&R identified significantly higher expected counts of valid PI than WIMBD, particularly for emails and IPs.

Memorization Findings

Verbatim Parroting Rates:
- Pythia-6.9B: Parroted 13.6% of the 483 gold instances verbatim.
- Pythia-160M: Even the smallest model parroted 2.7% of instances exactly.
- PI Type Breakdown: Email addresses were the most frequently parroted (nearly 20% for large models), followed by IP addresses (~14%), while phone numbers were less frequently parroted verbatim (<5%).
Model Size Correlation: There is a clear positive correlation between parameter count and memorization. Larger models (2.8B, 6.9B) showed significantly higher ParrotScores than smaller ones.
Pretraining Steps: Memorization is present even at 50% of pretraining (70k steps) and persists/increases as training continues to 143k steps.
Prefix Length: Memorization is highly dependent on prefix length but remains significant even with short prompts. With only 10 tokens of prefix, the 6.9B model still achieved an average ParrotScore of 0.34.

Constituent Part Memorization

Emails: Both usernames and domains are frequently parroted (up to 31% for domains in the 6.9B model).
IP Addresses: The first two octets (groups) are parroted more frequently than the last two, likely due to left-to-right autoregressive decoding patterns.
Phone Numbers: While full numbers are rarely parroted, area codes are parroted at a much higher rate (19.67% for the 6.9B model) compared to the rest of the number. This poses a specific privacy risk as area codes reveal geographic location.

5. Significance and Recommendations

Privacy Risk: The study confirms that LLMs, regardless of size, memorize and can be coerced into leaking PI. This is not limited to "jailbreak" attacks but occurs via standard prompting.
Data Curation Imperative: The authors strongly recommend that pretraining datasets be aggressively filtered and anonymized before training. Relying on post-hoc mitigation is insufficient.
Tool Availability: The R&R detectors and code are released to the community to facilitate better dataset curation and PI detection in future LLM development.
Future Work: The paper highlights the need for more comprehensive annotation of PI in datasets and better filtering strategies to prevent the "ripple effects" of PI leakage in model outputs.

In conclusion, this paper provides empirical evidence that verbatim parroting of personal information is a pervasive and scalable risk in modern LLMs, driven by model size and training duration, necessitating stricter data hygiene protocols in the AI development lifecycle.