💬 NLP

Distilling Human-Aligned Privacy Sensitivity Assessment from Large Language Models

This paper addresses the computational limitations of using large language models for privacy evaluation by distilling the capabilities of a 675B-parameter model into lightweight 150M-parameter classifiers that maintain strong agreement with human judgments across ten diverse domains while enabling scalable, real-world deployment.

Original authors: Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Gabriel Loiseau, Damien Sileo, Damien Riquet, Maxime Meyer, Marc Tommasi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-intelligent, but very expensive and slow, privacy detective. Let's call him "The Big Boss."

The Big Boss is incredibly good at reading a piece of text (like an email, a blog post, or a tweet) and telling you: "Is this safe to share, or does it contain secret personal info that could get someone in trouble?" He can look at a sentence and say, "This is totally harmless," or "Whoa, this person just revealed their medical history and home address!"

The Problem:
The Big Boss is so powerful that he requires a massive, energy-hungry supercomputer to run.

He's too slow: You can't use him to check millions of messages in real-time.
He's too expensive: Running him costs a fortune.
The Paradox: If you want to check if your own private data is safe, you have to send it to the Big Boss's cloud server. But sending your secret data to a stranger to check if it's secret... is kind of risky in itself!

The Solution: The "Shadow Apprentice"
The authors of this paper asked: "Can we teach a tiny, fast, cheap robot to think like the Big Boss, so we can keep the Big Boss's wisdom but lose his baggage?"

They used a technique called Knowledge Distillation. Think of it like this:

The Teacher: The Big Boss (Mistral Large 3) reads 200,000 different stories and rates them on a scale of 1 to 5 (1 = "Totally Safe," 5 = "Top Secret").
The Student: A tiny, lightweight computer model (only 150 million parameters, compared to the Big Boss's 675 billion).
The Process: The Student watches the Teacher grade thousands of papers. It doesn't just memorize the answers; it tries to understand how the Teacher thinks. "Oh, the Teacher gave this a '4' because it mentioned a specific hospital and a date. I'll remember that pattern."

The Results: The Student Outshines the Teacher
Surprisingly, after training, the tiny Student became better at matching human opinions than the Big Boss was!

When real humans rated the texts, the Student agreed with them 73.7% of the time.
The Big Boss only agreed 71.6% of the time.

Why? Imagine the Big Boss is a brilliant but slightly distracted genius who sometimes gets tired or confused by the specific way a question is asked. The Student, however, is like a focused apprentice who has practiced the same task millions of times. It learned to ignore the "noise" and focus on the core patterns of what humans consider private.

Why This Matters (The Real-World Magic)
Now, instead of needing a supercomputer in a data center, you can put this tiny "Privacy Detective" on your phone or your company's local server.

Instant Safety Checks: As you type an email, the tiny model can instantly whisper, "Hey, you just included your social security number. Maybe hide that?"
Cleaning Up Data: If a hospital wants to share patient records for research, this model can scan the documents and say, "This paragraph is safe, but this one needs to be scrubbed."
Privacy Without Privacy Risks: You don't have to send your sensitive data to a third-party cloud to get it checked. The model lives right where your data lives.

The Catch (Limitations)
The paper admits the Student isn't perfect:

It only speaks English: It can't check French or Spanish texts yet.
It's a bit rigid: It sees privacy as a single number (1-5), but real life is messy. Sometimes context matters (e.g., sharing a medical story with a doctor is different than sharing it on Twitter).
It learned from the Teacher: If the Big Boss had any biases, the Student might have them too.

In a Nutshell:
The authors took a giant, expensive, cloud-based privacy expert and distilled its brain into a tiny, efficient chip. This allows us to protect our digital privacy locally, quickly, and cheaply, without having to trust a giant corporation with our secrets. It's like turning a master chef into a perfect, automated toaster that makes the exact same delicious toast, but fits in your pocket.

1. Problem Statement

Accurately evaluating the privacy sensitivity of textual data is a critical bottleneck in privacy-preserving Natural Language Processing (NLP). While formal frameworks (e.g., differential privacy) and attack-based proxies exist, they often fail to capture the nuanced, contextual, and human-perceived nature of privacy.

Recent work established that Large Language Models (LLMs) can act as "Judges," achieving strong agreement with human judgments on privacy sensitivity. However, deploying frontier LLMs (e.g., 675B parameter models) for this purpose faces two major hurdles:

Computational Cost: The financial and energy costs of running massive models at scale are prohibitive.
Privacy Paradox: Sending sensitive data to third-party APIs for evaluation inherently violates the privacy constraints the evaluation aims to protect.

The core challenge is: Can the privacy reasoning capabilities of massive, human-aligned LLMs be transferred to lightweight, local models without sacrificing alignment with human judgment?

2. Methodology

The authors propose a Knowledge Distillation pipeline to transfer privacy assessment capabilities from a large teacher model to efficient student models.

A. Data Curation & Annotation

Source Data: The authors constructed a corpus of 200,000 texts sampled from 10 diverse public datasets (including blogs, emails, medical forums, legal advice, Reddit confessions, and reviews).
Teacher Model: They utilized Mistral Large 3 (675B parameters) as the teacher.
Annotation Strategy: Using the structured prompting methodology from Meisenbacher et al. (2025), the teacher model assigned privacy sensitivity scores on a 5-point Likert scale:
1. Harmless (No private info)
2. Mostly not private
3. Somewhat private
4. Very private
5. Extremely private
Dataset Characteristics: The resulting dataset is imbalanced, with ~46% of texts rated as "Harmless" and only ~6% as "Extremely private," reflecting real-world online communication patterns.

B. Distillation Pipeline

Student Models: The authors distilled the teacher's judgments into four lightweight encoder-based classifiers:
- Ettin-150M (149M parameters)
- Ettin-17M (17M parameters)
- BERT-base (110M parameters)
- ModernBERT-base (149M parameters)
Training: Models were fine-tuned for 5-class classification using a standard recipe (learning rate $2 \times 10^{-5}$ , 10% warmup, batch size 16, 3 epochs). The dataset was split 90/5/5 (train/val/test).

3. Key Contributions

Large-Scale Privacy Corpus: Creation of a 200k-text corpus automatically annotated for privacy sensitivity using a state-of-the-art open LLM across 10 diverse domains.
High-Performance Distillation: Demonstration that lightweight encoder models (specifically Ettin-150M) can achieve stronger alignment with human judgments than the teacher model itself, while enabling fully local, private, and fast inference.
Practical Utility for De-identification: Validation that these distilled models serve as effective, scalable automatic metrics for quantifying privacy reduction in text de-identification systems.

4. Experimental Results

A. Classification Performance

On the held-out test set, the Ettin-150M model achieved:

Accuracy: 74.9% (significantly outperforming majority and random baselines).
Macro F1: 68.1.
Mean Absolute Error (MAE): 0.28 (indicating predictions are rarely far from the true ordinal label).
The model performed robustly across the privacy spectrum, including the sensitive "Extremely private" class (F1 = 68.6).

B. Alignment with Human Judgments (The Core Finding)

Using Krippendorff's $\alpha$ (inter-rater reliability) on a human-annotated benchmark (250 texts, 677 human annotators):

Distilled Model (Ettin-150M): $\alpha = 0.737$ (vs. Human Average).
Teacher Model (Mistral Large 3): $\alpha = 0.716$ (vs. Human Average).
Inter-Human Agreement: $\alpha = 0.39$ (overall) / $0.54$ (pairwise).

Key Insight: The distilled student model surpassed the teacher model in agreement with human averages. The authors hypothesize that distillation acts as a denoising process, smoothing out the stochasticity of the teacher's prompt-based reasoning into a deterministic decision boundary that generalizes better to human benchmarks.

C. De-identification Evaluation

The model was tested on the Text Anonymization Benchmark (TAB) to evaluate anonymization quality:

Direct vs. Quasi-identifiers: Masking direct identifiers (names, etc.) had a larger privacy impact ( $\Delta = 0.34$ ) than masking quasi-identifiers ( $\Delta = 0.23$ ), despite fewer entities.
Synergistic Effect: Comprehensive masking (both types) reduced privacy scores significantly more ( $\Delta = 1.86$ ) than the sum of individual effects, confirming the model understands the interaction between identifier types.
Sanity Check: Randomly masking 30% of words increased the privacy score (disrupting coherence while leaving identifiers intact), proving the model evaluates semantic sensitivity, not just the presence of redaction tokens.

5. Significance and Future Directions

Scalable Privacy Evaluation: This work enables the deployment of privacy evaluators on-device or in private environments, removing the need to send sensitive data to external APIs.
Workflow Integration: These models can be used for:
- Dataset Curation: Routing high-risk texts for manual review before training.
- System Evaluation: Automatically scoring de-identification systems.
- User Assistance: Real-time warnings in writing assistants about potential PII leakage.
Research Enabler: Compact models allow for large-scale studies on privacy signals, counterfactual edits, and privacy-utility trade-offs that were previously computationally infeasible.

Limitations: The model inherits the teacher's biases and the limitations of the 1–5 scale (conflating identifiability with topic sensitivity). It currently supports only English and lacks explicit context (audience/purpose) modeling, which is crucial for a fully contextual privacy definition.

Conclusion

The paper successfully bridges the gap between the high accuracy of frontier LLMs and the practical need for efficient, private privacy evaluation. By distilling a 675B model into a 150M parameter classifier, the authors created a tool that not only matches but exceeds the teacher's alignment with human privacy judgments, offering a scalable solution for privacy-aware NLP.