💬 NLP

Clinically Grounded Privacy Evaluation of Medical LMs

This paper introduces a clinically grounded framework for evaluating medical language model privacy that reveals significant risks of verbatim memorization and sensitive diagnosis leakage under realistic threat models, while also demonstrating that templated documentation can lead to an overestimation of exact-match disclosure.

Original authors: Sasha Ronaghi, Sana Tonekaboni, Lena Stempfle, Vivian Utti, Jordan Li Cahoon, Nathaniel Hendrix, Ayin Vala, Marzyeh Ghassemi, Emily Alsentzer

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sasha Ronaghi, Sana Tonekaboni, Lena Stempfle, Vivian Utti, Jordan Li Cahoon, Nathaniel Hendrix, Ayin Vala, Marzyeh Ghassemi, Emily Alsentzer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a medical language model (LM) as a super-attentive intern who has read millions of patient files from a specific network of family doctors. This intern is incredibly smart and can help summarize notes or suggest diagnoses. However, there's a scary side: because they read so many specific files, they might accidentally memorize them and repeat private secrets back out if you ask the right questions.

This paper is like a privacy safety inspection for that intern. Instead of just asking, "Did you memorize the file?" the researchers created a realistic test to see how much private info the intern leaks based on what they already know about the patient.

Here is the breakdown of their findings using simple analogies:

1. The "Key" Test: How much info do you need to unlock the secret?

The researchers tested the intern with different levels of "keys" (information) about a patient, ranging from very little to a lot. Think of it like trying to open a safe:

Level 1: Public Info (The "Name Tag" Key).
- What they knew: Just basic public stuff like age, gender, job, and maybe the name.
- Result: The intern barely leaked anything. It was like trying to open a safe with a paperclip; it didn't work well.
Level 2: Appointment Metadata (The "Calendar" Key).
- What they knew: The patient's name, date of birth, the date of their last visit, the doctor's name, and the clinic location. This is info that might be on a billing statement or a reminder email.
- Result: This was the big breakthrough. With just these routine details, the intern started reciting large chunks of the patient's actual medical notes word-for-word. It was as if knowing the patient's name and appointment date was enough to open the safe completely.
Level 3: Partial Notes (The "First Page" Key).
- What they knew: The appointment details plus the very first sentence of the medical note (the "Chief Complaint").
- Result: The intern memorized even more, reciting almost the entire file.

2. The "Sensitive Secret" Test

The researchers didn't just check if the intern repeated words; they checked if the intern revealed sensitive secrets like an abortion, HIV status, or mental health struggles.

The Finding: When the researchers gave the "Calendar Key" (Name + DOB + Visit Date + Doctor), the intern became a master detective. It could correctly guess sensitive diagnoses with very high accuracy (91% for abortion, 81% for HIV).
The Danger: This means that even if you don't have the full medical file, just knowing when a patient saw a specific doctor is enough for the AI to "remember" and reveal their most private health struggles.

3. The "Template Trap" (Why the numbers were scary)

The researchers noticed something tricky. When they counted how much the intern "memorized," they found that 36% of the repeated words weren't actually secrets.

The Analogy: Imagine the intern is reciting a story. Half the time, they are repeating unique, personal details about your life (like "I broke my leg in 2010"). But the other time, they are just reading from a standard template that every single patient gets (like "The heart sounds are normal" or "No chest pain").
The Lesson: If you just count any repeated word as a "leak," you are scaring yourself too much. A lot of what the AI repeats is just boring, standard medical boilerplate that 10,000 other patients also have. The real danger is when it repeats the unique parts or the sensitive diagnoses.

4. The "Long Memory" Problem

The study used data from patients who saw doctors over many years (longitudinal data).

The Finding: The intern didn't just remember the last visit. When given the details of a recent appointment, the intern would pull up and recite details from visits that happened years ago.
The Metaphor: It's like the intern has a "super-brain" that connects every visit a patient ever had. If you ask about a recent visit, the intern might accidentally spill secrets from a visit five years ago because the AI has memorized the whole timeline.

Summary of the Paper's Main Points

Routine info is risky: You don't need a hacker to steal a medical file. Just knowing a patient's name, birth date, and which doctor they saw is enough to trigger the AI to leak their private history.
Not all leaks are equal: We need to stop counting "template" words (like "no pain") as privacy breaches. The real risk is the AI revealing unique, sensitive, or specific patient details.
Context matters: Privacy tests for medical AI need to look like real-world scenarios (what a billing clerk or a family member might know), not just abstract computer tests.

What the paper does NOT say:

It does not say this happens with every medical AI (only the one they tested).
It does not propose a specific fix or software patch to stop this (it only provides the framework to measure it).
It does not claim that this happens in every hospital, only in the specific dataset of rural family medicine practices they studied.

In short: Medical AI has a "photographic memory" that can be triggered by simple, everyday details, potentially revealing deep secrets about a patient's health, but we need better ways to tell the difference between a real secret and just a standard medical phrase.

Technical Summary: Clinically Grounded Privacy Evaluation of Medical LMs

Problem Statement

Medical language models (LMs) trained on longitudinal clinical notes capture treatment practices and patient-specific data that general-domain corpora lack. However, this encoding introduces significant privacy risks. Prior work has demonstrated that LMs can memorize and reproduce training data, a concern exacerbated in clinical settings by "copy-forward" documentation practices that create duplicated, near-duplicate sequences highly susceptible to memorization.

Existing privacy evaluations often rely on general-domain threat models where adversaries possess arbitrary training prefixes or seek to extract generic personal attributes. These approaches fail to account for the contextual norms of clinical privacy. In clinical settings, risk depends not merely on whether a model reproduces text, but on whether the disclosure links to a specific individual and reveals sensitive information (e.g., diagnoses) that could lead to discrimination or harm. Furthermore, standard metrics like exact-match memorization may conflate patient-specific leakage with benign, templated documentation artifacts common in electronic health records (EHRs).

Methodology

The authors introduce a clinically grounded privacy evaluation framework designed to probe medical LMs under realistic adversarial conditions. The methodology consists of three core components:

1. Graded Adversarial Access (Priors)

Instead of assuming access to full note prefixes, the framework simulates attackers with varying levels of plausible knowledge about a target patient. Priors ( $\pi$ ) are organized along an axis of increasing access:

Public: Demographics (age, gender, occupation, etc.) inferable from public records.
Public + Name: Demographics plus the patient's name.
Public + Name + Meds: Includes a partial medication list (accessible to pharmacists or caregivers).
Encounter Info: Routine metadata (name, DOB, visit date, provider, practice location) visible to billing staff or household members.
Encounter Info + Chief Complaint / HPI: Privileged fragments of the clinical note itself, modeling worst-case partial leaks.

2. Dual-Dimension Privacy Measures

The framework evaluates leakage across two complementary dimensions:

Verbatim Memorization: Measured via token-level $n$ -gram matching ( $\tau=30$ ) between generated text and training notes. Crucially, the authors decompose memorized regions by content type (clinically revealing vs. templated) and patient specificity (unique to the target vs. duplicated across patients). This distinguishes harmful leakage from benign reproduction of shared documentation artifacts (e.g., "Negative for chest pain").
Semantic Leakage of Sensitive Diagnoses: Evaluates whether the model discloses high-risk diagnoses (Abortion, HIV, Depression, Anxiety, Bipolar, PTSD) through direct mention, associated symptoms, or medications, even without exact text overlap.

3. Matched Cohort Design

To attribute leakage to training data inclusion rather than demographic inference, the authors employ a matched train/non-train cohort design. Using propensity-score matching on age, sex, and note volume, they compare model performance on patients present in the training corpus versus those absent. This isolates "training-attributable leakage" from population-level correlations.

Experimental Setup

The framework was applied to a modern decoder-only LM (Qwen3.5-9B) continually pre-trained on 1 billion tokens of longitudinal clinical notes from 26,948 patients across 192 rural family-medicine practices in the U.S. The dataset spans 2019–2025, with a median follow-up of 3.8 years per patient.

Key Results

1. Leakage Scales with Adversarial Access

Privacy leakage increases significantly as the adversary's access to patient metadata grows:

Verbatim Memorization: Under the Encounter Info prior (name, DOB, visit date, provider, location), the model reproduces verbatim content from a mean of 2.81 distinct notes across a patient's timeline. Memorization rates jump from 4.8% (Public prior) to 85.6% (Encounter Info + Chief Complaint + HPI).
Semantic Disclosure: Routine metadata alone (Encounter Info) enables high-accuracy recovery of sensitive diagnoses. The model achieves an AUROC of 0.91 for abortion and 0.81 for HIV, with Positive Predictive Values (PPV) reaching 0.98 and 1.0, respectively.
Training Attribution: A significant gap emerges between train and non-train cohorts at the Encounter Info level (+0.191 AUROC delta), confirming that the leakage is attributable to the patient's presence in the training data rather than inference from demographics alone.

2. Exact-Match Metrics Overstate Harm

The study finds that standard exact-match metrics can inflate privacy risk estimates:

Under the Encounter Info prior, 36% of memorized tokens are templated (e.g., standard review-of-systems negatives) rather than clinically revealing.
45% of memorized regions recur across multiple patients, indicating that much of the "memorized" text is shared boilerplate.
However, within a single patient's timeline, 37.5% of clinically revealing, patient-specific regions recur across multiple notes, suggesting that "copy-forward" documentation amplifies the memorization of sensitive content.

Significance and Claims

The paper claims to provide a practical framework for contextual privacy evaluation of medical LMs. Its primary contributions are:

Realistic Threat Modeling: Demonstrating that adversaries with only routine encounter metadata (often visible to non-clinical staff or family) can elicit substantial verbatim memorization and sensitive diagnosis recovery, challenging the assumption that de-identification or lack of full note access mitigates risk.
Nuanced Leakage Measurement: Highlighting that treating all memorization as equivalent obscures the nature of the risk. The framework distinguishes between patient-specific, clinically revealing leakage and the benign reproduction of shared documentation artifacts.
Longitudinal Risk: Providing evidence that training on longitudinal data creates a risk where a single data point (e.g., a visit date) can unlock a patient's entire historical record, including sensitive diagnoses like HIV and abortion.

The authors conclude that privacy audits for clinical LMs must move beyond generic memorization tests to evaluate leakage under realistic access levels and characterize the clinical significance and patient specificity of the revealed information. They note limitations, including the use of identifiable (rather than de-identified) data for evaluation and the focus on base models without post-training, suggesting future work should address these factors.