💬 NLP

Differentially Private De-identification of Dutch Clinical Notes: A Comparative Evaluation

This paper presents the first comparative study of differential privacy, named entity recognition, and large language models for de-identifying Dutch clinical notes, demonstrating that while differential privacy alone significantly reduces data utility, hybrid approaches combining it with LLM-based preprocessing achieve a superior privacy-utility trade-off.

Original authors: Michele Miranda, Xinlan Yan, Nishant Mishra, Rachel Murphy, Ameen Abu-Hanna, Sébastien Bratières, Iacer Calixto

Published 2026-04-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michele Miranda, Xinlan Yan, Nishant Mishra, Rachel Murphy, Ameen Abu-Hanna, Sébastien Bratières, Iacer Calixto

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, secret diary filled with stories about patients in a Dutch hospital. These stories are incredibly valuable for doctors and researchers to learn how to cure diseases, but they are also full of personal secrets: names, phone numbers, dates of birth, and specific hospital locations.

If you want to share these stories with the world to help science, you have to de-identify them. You have to scrub out the personal details so no one can figure out who the story is about, while keeping the medical facts intact so the story still makes sense.

This paper is like a taste test to see which method works best for cleaning these Dutch medical diaries. The authors tested three different "cleaning crews":

The Three Cleaning Crews

The "Spotter" (NER - Named Entity Recognition):
Think of this as a very fast, trained robot that scans the text looking for specific keywords like "Name," "City," or "Doctor." When it sees one, it swaps it for a generic placeholder like <NAME> or <CITY>.
- Pros: It's fast and good at finding the obvious stuff.
- Cons: It's not perfect. Sometimes it misses a secret, or sometimes it gets confused and deletes a medical fact thinking it's a secret.
The "Super-Intelligent Editor" (LLMs - Large Language Models):
This is like hiring a brilliant human editor (or a super-smart AI like GPT-4) to read the whole story. Instead of just looking for keywords, it understands the context. It knows that "John Smith" is a person, but "John" in a different sentence might be part of a medical term. It rewrites the story to hide the secrets naturally.
- Pros: It's incredibly smart and keeps the story flowing well.
- Cons: It's expensive, slow, and if you send the data to a cloud service, you might be breaking privacy rules by sending the raw data out of the building.
The "Static Noise Generator" (Differential Privacy - DP):
This is the most mathematically rigorous method. Imagine you take the diary and sprinkle a little bit of "static noise" or "fog" over the words. You might change "John" to "Jane" or "Smith" to "Jones" randomly, but in a way that statistically guarantees no one can tell if a specific person was in the book or not.
- Pros: It offers a mathematical "guarantee" of privacy.
- Cons: If you add too much fog (to be super safe), the story becomes gibberish. If you add too little, the secrets might still be visible.

The Big Experiment: Mixing and Matching

The researchers didn't just pick one; they tried five different strategies to see which combination was the "Goldilocks" solution (not too risky, not too messy):

Just the Spotter: Let the robot find and hide names.
Just the Editor: Let the AI rewrite the text.
Just the Noise: Add mathematical fog to the raw text immediately.
Spotter + Noise: Let the robot hide the obvious names first, then add the fog to the rest.
Editor + Noise: Let the super-smart AI hide the names first, then add the fog to the rest.

The Results: What Did They Find?

Here is the takeaway, explained simply:

The "Just Noise" approach failed. If you try to add mathematical fog to raw, messy hospital notes without cleaning them first, the result is a disaster. The story becomes unreadable, and you still leak some secrets. It's like trying to blur a photo that is already full of sharp, identifiable faces; the blur just makes it look weird, not safe.
The "Spotter + Noise" approach was okay. It was better than just noise, but the robot sometimes missed tricky secrets.
The "Editor + Noise" approach was the winner. This was the magic combination.
- First, the Super-Intelligent Editor (LLM) did the heavy lifting. It found and removed 99% of the secrets perfectly, leaving a clean, anonymized draft.
- Then, they added a tiny bit of mathematical fog (Differential Privacy) just to be extra safe.
- Why it worked: Because the AI had already removed the dangerous secrets, the "fog" didn't have to be thick and messy. It could be light, preserving the medical facts while still offering a mathematical guarantee of privacy.

The Bottom Line

If you want to share Dutch hospital notes safely:

Don't just rely on math (Differential Privacy) alone; it ruins the data.
Don't just rely on a robot (NER) alone; it misses things.
Do this instead: Use a smart AI to clean the text first, then apply a little bit of mathematical privacy protection on top.

This gives you the best of both worlds: a story that is still useful for doctors to learn from, but one that is mathematically guaranteed to keep patient identities safe. It's like having a security guard (the AI) check everyone's ID at the door, and then having a metal detector (the math) scan for anything they might have missed.

1. Problem Statement

The secondary use of clinical data for research is hindered by strict privacy regulations (e.g., GDPR, HIPAA) that require the removal of Personally Identifiable Information (PII). While manual de-identification is the gold standard, it is costly and slow. Automated methods exist but face a critical trade-off:

Named Entity Recognition (NER) and Large Language Models (LLMs): These offer high utility (preserving clinical meaning) but lack formal privacy guarantees. They rely on detecting and redacting specific entities, which may miss subtle identifiers or fail against re-identification attacks.
Differential Privacy (DP): This provides rigorous mathematical guarantees against privacy leakage by adding noise to data. However, applying DP directly to raw text often severely degrades utility, rendering the data useless for downstream clinical tasks (like predicting adverse drug events).

The Gap: There is a lack of comparative studies evaluating these approaches specifically for Dutch clinical text, and no established framework for combining the formal guarantees of DP with the linguistic precision of NER/LLMs to optimize the privacy-utility trade-off.

2. Methodology

The authors conducted a comparative study using the Dutch ADE dataset (102 ICU clinical notes annotated with drug/disorder entities and adverse drug event relations). They evaluated five distinct de-identification pipelines:

A. Core Technologies

NER-based: Used the pre-trained multilingual model gliner-multi-v2.1 to identify and redact 17 types of PII (e.g., names, dates, hospitals) without fine-tuning.
LLM-based: Prompted various LLMs (GPT-4o, DeepSeek-70B, LLaMA-3.1, MedGEMMA) in a zero-shot setting to redact PII categories.
Differential Privacy (DP):
- Metric-DP: Adds geometric noise to token embeddings (using BERTje) and selects the nearest neighbor in the vocabulary to replace the token.
- RANTEXT: Uses an LLM to generate embeddings and dynamically constructs randomized adjacency lists for token substitution based on a Laplace distribution.

B. Experimental Pipelines

The study tested five configurations to assess the impact of preprocessing:

$P_{NER}$ : NER only.
$P_{LLM}$ : LLM only.
$P_{DP}$ : DP applied directly to raw text.
$P_{NER \to DP}$ : NER preprocessing followed by DP.
$P_{LLM \to DP}$ : LLM preprocessing followed by DP (using DeepSeek-70B as the best-performing open-source model).

C. Evaluation Metrics

Privacy Leakage: Measured as the percentage of residual PII (both direct identifiers like names/phone numbers and indirect quasi-identifiers) remaining in the de-identified text compared to a manually de-identified gold standard ( $D_{manual}$ ).
Utility (Extrinsic): Evaluated via downstream tasks:
- Entity Classification: Predicting drug and disorder mentions.
- Relation Classification: Predicting Adverse Drug Events (ADE) between drug-disorder pairs.
Operational Efficiency: Time taken to process data compared to manual annotation.

3. Key Contributions

First Dutch-Specific Study: This is the first work to investigate DP-based text anonymization on real-world Dutch hospital clinical notes, addressing a gap in non-English de-identification research.
Hybrid Strategy Validation: The paper empirically demonstrates that hybrid pipelines (LLM/NER preprocessing + DP) significantly outperform pure DP or pure LLM approaches.
Formal vs. Empirical Analysis: The study bridges the gap between theoretical DP guarantees and empirical leakage, showing that reducing the "sensitive surface" before applying DP noise is crucial for maintaining utility.
Comprehensive Benchmark: Provides a detailed comparison of proprietary (GPT-4o) and open-source (DeepSeek, LLaMA, MedGEMMA) models in a privacy-compliant setting.

4. Key Results

Privacy Leakage

Pure DP Failure: Applying DP directly to raw text ( $P_{DP}$ ) resulted in the highest privacy leakage across all privacy budgets ( $\epsilon$ ).
Hybrid Success: Pipelines combining LLM preprocessing with DP ( $P_{LLM \to DP}$ ) achieved the lowest leakage, keeping empirical leakage below 10% even at high $\epsilon$ values.
Direct vs. Indirect: LLM+DP was the only approach that effectively suppressed the leakage of direct PII (names, IDs), with remaining leakage consisting mostly of less risky indirect PII.
Metric-DP vs. RANTEXT: Metric-DP generally preserved privacy better than RANTEXT at low-to-mid $\epsilon$ budgets, though both suffered when applied to raw text.

Utility Preservation

Entity Classification: Utility (F1-score) improved as $\epsilon$ increased. $P_{LLM \to DP}$ consistently outperformed $P_{NER \to DP}$ and $P_{DP}$ , approaching the performance of non-private baselines at higher budgets.
Relation Classification: This task was more sensitive to noise. Pure DP pipelines failed to produce usable data at low $\epsilon$ (F1 $\approx$ 0). However, the LLM+DP pipeline maintained significantly higher F1 scores than other DP methods, proving that linguistic preprocessing is essential for preserving complex relational structures.
LLM Baselines: GPT-4o achieved the best balance of low leakage and high utility among standalone LLMs. MedGEMMA 27B showed the highest utility among open-source models.

Operational Efficiency

Manual de-identification took 47+ hours for the dataset.
Automated pipelines were drastically faster: Metric-DP took ~3 minutes, and RANTEXT took ~27 minutes per run.

5. Significance and Implications

Practical Recommendation: The authors advise practitioners to adopt a hybrid strategy: use a high-recall masking stage (preferably LLM-based, or NER-based) to remove explicit identifiers before applying Differential Privacy. This reduces the amount of sensitive data exposed to noise, thereby preserving clinical utility while maintaining formal privacy guarantees.
Language Agnosticism: While focused on Dutch, the findings suggest that the "preprocessing + DP" paradigm is likely applicable to other languages, though model selection (especially for open-source LLMs) is critical.
Future Directions: The study highlights the need for broader validation across languages, the use of synthetic benchmarks for reproducibility, and expanded utility evaluations beyond entity/relation classification to document-level tasks.

Conclusion: The paper establishes that while DP alone is too destructive for clinical text utility, and LLMs alone lack formal guarantees, combining LLM-based redaction with DP offers a viable, efficient, and privacy-preserving solution for sharing Dutch clinical narratives.