💬 NLP

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

This paper redefines contextual privacy for LLM agents as an Information Sufficiency task, introducing free-text pseudonymization as a superior strategy and a conversational evaluation protocol that reveals how single-message assessments underestimate privacy leakage compared to multi-turn interactions.

Original authors: Yunze Xiao, Wenkai Li, Xiaoyuan Wu, Ningshan Ma, Yueqi Song, Weihao Xuan

Published 2026-04-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yunze Xiao, Wenkai Li, Xiaoyuan Wu, Ningshan Ma, Yueqi Song, Weihao Xuan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, helpful assistant (an AI) that writes messages for you. You tell it, "I need to ask my boss for time off," and you whisper the real reason: "I'm going to a therapy session."

The AI has to decide: How much of that secret should it tell your boss?

This paper is about finding the best way for the AI to protect your secrets without making you look suspicious or ruining the conversation. The researchers discovered that the way we usually test these AIs is flawed, and they found a new, better trick to keep your privacy safe.

Here is the breakdown in simple terms:

1. The Problem: The "Bad" Ways to Hide Secrets

Previously, AI systems had two main ways to handle your private info:

The "Silent Treatment" (Suppression): The AI just deletes the secret.
- What happens: You say, "I need time off." The AI writes, "I need time off."
- The flaw: Your boss reads it and thinks, "Wait, why? Is everything okay?" They ask again. Because the AI didn't give a reason, the boss gets suspicious and keeps asking. The silence actually invites more questions.
The "Vague Fog" (Generalization): The AI replaces the secret with a boring, broad category.
- What happens: You say, "I'm going to therapy." The AI writes, "I have a private medical appointment."
- The flaw: This sounds a bit shady. It's like wearing a trench coat and sunglasses in the middle of the day. It tells the boss, "I am hiding something!" The boss then thinks, "Okay, what kind of medical appointment?" and keeps digging.

The Big Mistake in Old Research:
Before this paper, scientists tested these methods by just looking at the first message. They thought, "Oh, the Vague Fog looks pretty good!" But they didn't test what happens when the boss asks, "So, what kind of appointment?" In the real world, people don't just read one message and stop; they chat back and forth. When you add those follow-up questions, the "Vague Fog" strategy collapses, and the secret leaks out.

2. The New Solution: The "Plausible Cover Story" (Pseudonymization)

The researchers introduced a third, smarter strategy called Pseudonymization.

Instead of deleting the info or being vague, the AI invents a specific, believable, but fake reason that fits the situation perfectly.

The Scenario: You need time off for therapy.
The AI's Reply: "I need time off to attend a professional training session."
Why it works:
- It's specific: It answers the "Why?" immediately.
- It's believable: It sounds normal and professional.
- It's consistent: If the boss asks, "Oh, what are you learning?" the AI (and you) can say, "Oh, just some advanced project management skills," and the story holds up.

The Analogy:
Think of it like a spy movie.

Suppression is the spy saying nothing. The bad guy gets suspicious and starts searching the room.
Generalization is the spy saying, "I'm doing something important." The bad guy thinks, "What? Tell me!" and keeps pressing.
Pseudonymization is the spy saying, "I'm meeting a contact at the coffee shop." The bad guy thinks, "Oh, okay, that's normal," and leaves them alone. The spy didn't lie about where they are going (a coffee shop is a real place); they just lied about who they are meeting.

3. The Results: What the Study Found

The researchers tested this on 7 different super-smart AI models across 792 different scenarios (like talking to a boss, a friend, or a partner).

The "Cover Story" Wins: The "Pseudonymization" strategy was the clear winner. It kept secrets safe and kept the conversation flowing naturally.
The "Vague Fog" Fails: When the conversation went on for a few turns (like a real chat), the "Vague Fog" strategy failed miserably. The AI would get pressured and accidentally reveal the truth.
The "Silent Treatment" is Risky: Just saying nothing often made people ask more questions, which eventually led to the secret leaking anyway.

4. Why This Matters

This paper changes how we build AI assistants. It tells us that:

Don't just test one message: You have to test how the AI handles a whole conversation, not just the first sentence.
Don't just hide the truth: Sometimes, giving a plausible alternative is better than trying to hide the fact that you are hiding something.
It's not lying, it's privacy: The authors argue that this isn't "deception." It's like telling your boss you have a "doctor's appointment" instead of "therapy." You are managing your privacy, not tricking them about how to do their job.

In a nutshell: If you want your AI to protect your secrets, don't tell it to be quiet or vague. Tell it to come up with a good, believable story that satisfies the other person's curiosity without revealing your actual secret. It's the art of the perfect "cover story."

1. Problem Statement

Large Language Model (LLM) agents are increasingly used to draft messages on behalf of users. However, users often overshare sensitive information, and there is significant disagreement on what constitutes "private" data in specific contexts. Existing privacy-preserving systems for LLMs rely on two traditional strategies inherited from structured-data privacy:

Suppression: Omitting sensitive information entirely.
Generalization: Replacing specific details with broader abstractions (e.g., "a health matter" instead of "chemotherapy").

Key Limitations of Current Approaches:

Incomplete Strategy Space: These methods ignore pseudonymization (replacing sensitive attributes with functionally equivalent, plausible alternatives), a strategy recognized in frameworks like GDPR but rarely applied to free-text LLM communication.
Flawed Evaluation: Current benchmarks evaluate privacy on single-turn messages. They fail to account for the iterative nature of human communication, where a recipient may ask follow-up questions that force the agent to reveal more information or expose the protective strategy (e.g., an evasive answer invites probing).
Context Ignorance: Existing systems do not adequately model how power dynamics (institutional vs. peer vs. intimate) and sensitivity types (discrimination risk, social cost, boundaries) affect the efficacy of privacy strategies.

2. Methodology

A. Theoretical Framework: Information Sufficiency (IS)

The authors formalize privacy-preserving communication as an Information Sufficiency (IS) task. The goal is not merely to remove data but to draft a reply that:

Achieves the user's communicative goal (Utility).
Reveals as little private information as possible (Privacy).
Does not signal that information is being withheld (Covertness).

B. Proposed Strategies

The study introduces a third strategy to the existing toolkit:

Suppress: Omit the attribute.
Generalize: Map the attribute to a truthful but vague category.
Pseudonymize: Replace the sensitive attribute with a specific, concrete, but fake alternative from the same domain that is functionally equivalent to the original (e.g., replacing "therapy session" with "professional training").

C. Conversational Evaluation Protocol

To address the limitations of single-message evaluation, the authors propose a multi-stage protocol:

Stage 1 (Generation): The agent generates an initial reply ( $m_1$ ) conditioned on one of the four strategies (None, Suppress, Generalize, Pseudonymize).
Stage 2 (Simulated Follow-up): A non-adversarial receiver simulator (Deepseek-v3.2) generates natural follow-up questions ( $f_1, f_2$ ) based on the conversation history and role. The agent replies ( $m_2, m_3$ ) while maintaining consistency with the initial strategy.
Stage 3 (Evaluation): An adversary model attempts to infer the true private attributes from the full transcript ( $T$ $T$ ).
- Metrics:
  - Privacy: Measured by Hypothesis Leakage Score (HLS), calculating the probability of exact or category matches between the adversary's guess and the ground truth.
  - Covertness: Rated 1–5 on whether the reply appears evasive or fabricated.
  - Utility: Rated on goal achievement and receiver satisfaction.
  - IS-AD Score: A composite metric: $IS\text{-}AD = (1 - HLS) \times Util$ .
  - Vulnerability Gap ( $\Delta_{vuln}$ ): The difference in leakage between static (single-turn) and probed (multi-turn) evaluations.

D. Dataset Construction

Source: Derived from 493 seeds in the PrivacyLens corpus.
Scale: Expanded into 792 unique scenarios covering 3 power relations (Institutional, Peer, Intimate) and 3 sensitivity types (Discrimination Risk, Social Cost, Boundary).
Models Evaluated: 7 frontier LLMs (Gemini 3.1 Pro, GLM-5, GPT-5.4, Kimi K2.5, MiniMax M2.1/2.5, Qwen3-8B).
Total Output: 22,176 conversational transcripts.

3. Key Results

A. Pseudonymization Dominates the Privacy-Utility Tradeoff

Overall Performance: Pseudonymization achieved the highest mean IS-AD score (0.764), outperforming Suppression (0.730), the No-Protection baseline (0.694), and Generalization (0.664).
Utility Preservation: Pseudonymization maintained 98.5% of the utility of the unprotected baseline, whereas Suppression dropped to 86.6%.
Generalization Failure: Generalization was the only strategy Pareto-dominated by the no-protection baseline, meaning it offered less privacy and less utility than simply saying nothing.

B. Single-Message Evaluation Underestimates Leakage

Ranking Reversal: In static (single-turn) evaluations, Suppression appeared most effective. However, under multi-turn probing, Pseudonymization became the most robust.
Vulnerability Gap:
- Generalization suffered the largest degradation under follow-up (+11.8 percentage points in leakage), as vague answers signal withholding and invite targeted probing.
- Suppression also degraded significantly (+8.3 pp) as information gaps trigger aggressive inquiry.
- Pseudonymization remained stable (+3.9 pp) because the initial fake narrative provides a consistent, plausible cover story that satisfies the receiver without needing further defense.

C. The Mechanism: Covertness

Causal Chain: The study identifies Covertness as the mediator of vulnerability.
- Suppression and Generalization produce "suspicious" or evasive replies (low covertness scores).
- This suspicion triggers the receiver to ask pointed follow-up questions.
- These probes force the agent to either reveal the truth or break consistency, leading to leakage.
Pseudonymization Advantage: By providing a complete, natural-sounding alternative, pseudonymization achieves high covertness (4.35/5, comparable to the unprotected baseline), effectively short-circuiting the receiver's motivation to probe.

D. Contextual Modulation

Hardest Context: Institutional × Discrimination Risk. Even here, pseudonymization outperformed the baseline, though the margin was smaller because the request itself often reveals the sensitive attribute (e.g., requesting time off for chemotherapy).
Best Context: Intimate × Social Cost. Pseudonymization excelled here, as intimate partners have high motivation to probe but high social costs for the user to disclose. A plausible cover story satisfied curiosity without revealing the truth.
Exception: In Peer × Social Cost, Suppression slightly outperformed Pseudonymization. In low-stakes peer interactions, elaborate cover stories can seem "over-engineered," making a simple deflection more natural.

4. Significance and Contributions

New Strategy Space: The paper formally introduces free-text pseudonymization as a viable and superior privacy strategy for LLM agents, moving beyond the binary of "hide" vs. "vague."
Evaluation Protocol Shift: It demonstrates that single-message evaluations are fundamentally flawed for conversational agents. The proposed multi-turn protocol reveals that strategies appearing strong in static tests (like suppression) fail under realistic social pressure.
Empirical Evidence: Through a large-scale study (792 scenarios, 7 models), it provides the first comprehensive map of strategy performance, proving that Generalization is often counterproductive and that Pseudonymization offers the optimal balance of privacy, utility, and naturalness.
Ethical Framing: The authors argue that pseudonymization is not "deception" but disclosure management. It preserves the receiver's functional needs (e.g., a manager needs to approve time off, not know the specific medical reason) while respecting the user's right to control their private narrative.

Conclusion

The paper concludes that for LLM agents acting on behalf of users, pseudonymization is the superior strategy for maintaining privacy in interactive contexts. It avoids the utility loss of suppression and the "suspicion trap" of generalization. Future work should focus on extending these findings to autonomous agents in non-dialogue settings (e.g., API calls) and developing context-aware mechanisms for selecting the optimal strategy dynamically.