🤖 AI

PromptPET: Privacy-Utility Optimized Prompt Obfuscation

This paper introduces PROMPTPET, an LLM-based agent that optimizes the tradeoff between user privacy and chatbot utility by dynamically selecting the most effective obfuscation action (redaction, abstraction, replacement, or a novel noising/denoising scheme) for sensitive information in user prompts.

Original authors: Ke Yang, Olivia Figueira, Umar Iqbal, Athina Markopoulou

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ke Yang, Olivia Figueira, Umar Iqbal, Athina Markopoulou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a very smart, helpful robot assistant (like a chatbot). You want it to give you great advice, so you tell it details about your life: your age, your health struggles, your location, and your hobbies. The robot uses this info to give you a personalized answer, which is great. But there's a catch: the company running the robot is also listening. They are taking notes on your secrets to build a profile of you, which they might use to sell you things or share with others.

This paper introduces a tool called PromptPET. Think of it as a privacy bodyguard that sits between you and the robot. Its job is to let you get the help you need while hiding your secrets from the company.

Here is how it works, using some simple analogies:

The Problem: The "Too-Honest" Conversation

When you ask a robot for help, you often have to be very specific.

You say: "I am 35, live in Mountain View, and have depression. Can you find a therapist?"
The Robot: Gives you a perfect list of therapists in Mountain View.
The Company: Writes down "35," "Mountain View," and "Depression" in a file labeled "User Profile."

If you just hide the words (like crossing them out), the robot gets confused and gives you a bad answer. If you don't hide anything, the company learns everything.

The Solution: PromptPET's Four Tricks

The researchers tested four different ways to "scramble" your message so the robot still understands you, but the company gets confused. They call these Obfuscation Actions:

Redaction (The Black Marker):
- What it is: You cross out the sensitive word entirely and replace it with a blank space like [REDACTED].
- The Analogy: Like crossing out your address on a package. The mail carrier (the robot) knows where to deliver the package because of the rest of the address, but the company doesn't know your exact street.
- The Downside: If you cross out the most important part (like "depression"), the robot might not know what kind of help you need.
Abstraction (The Vague Description):
- What it is: You replace a specific detail with a broader category.
- The Analogy: Instead of saying "I have Depression," you say "I have a Health issue." Instead of "Mountain View," you say "The Bay Area."
- The Result: The robot knows you need medical help and is in California, so it can still help. The company only knows you have a general health issue, not the specific one.
Replacement (The Fake ID):
- What it is: You swap your real info with a believable fake one.
- The Analogy: You tell the robot you live in "Palo Alto" instead of "Mountain View." The robot gives you a list of therapists in Palo Alto. You then ignore that list and use your own knowledge to find the ones in Mountain View.
- The Risk: You have to manually filter the answer, and the company thinks you actually live in Palo Alto.
Noise/Denoising (The "Decoy" Strategy) — The New Trick:
- What it is: This is the paper's big innovation. You keep your real info exactly as is, but you add a bunch of fake, distracting info right next to it.
- The Analogy: Imagine you are trying to find a specific needle in a haystack. The company is the one looking for the needle.
  - You say: "I need a therapist for Depression."
  - PromptPET adds: "...and also, I'm looking for a Dentist."
  - The Robot: Gives you a list of therapists and a list of dentists.
  - PromptPET (The Filter): Before the answer reaches you, it chops off the "Dentist" list and gives you only the "Therapist" list.
- Why it's genius: The robot got the right answer because you told the truth about the depression. But the company is now confused! They see "Depression" and "Dentist" mixed together and can't be sure which one is your real priority. It's like throwing a bunch of red herrings into the water to confuse the sharks.

How PromptPET Decides Which Trick to Use

The smartest part of PromptPET is that it doesn't just pick one trick for everyone. It uses a Coach (a Rule Optimizer) that learns which trick works best for every specific sentence.

If you mention something minor (like your favorite color), the Coach might use Redaction (cross it out) because it doesn't matter for the answer.
If you mention something critical (like your medical condition), the Coach uses Noise (add the decoy) because you need the robot to hear the truth to help you, but you want to confuse the company.

The Results

The researchers tested this on real conversations with AI. They found that:

Noise (the decoy trick) was usually the best at keeping the conversation helpful while hiding your secrets.
PromptPET (the smart coach that picks the right trick) was much better than previous methods. It managed to protect your privacy 3.3 times better for the same amount of "helpfulness" lost compared to older tools.
It successfully hid your profile from the company while still letting you get the perfect answer you asked for.

In Summary

PromptPET is like a privacy translator. It takes your honest, detailed request, rewrites it just enough to confuse the data-hungry company, but keeps it clear enough for the helpful robot to do its job. It uses a mix of crossing things out, making things vague, swapping them out, and adding "decoys" to ensure you get the help you need without giving away your secrets.

Technical Summary: PromptPET

Problem Statement

The rapid adoption of AI agents (e.g., ChatGPT, Gemini) has shifted user interaction from search boxes to open-ended natural language, encouraging the disclosure of rich, sensitive personal information. While this enhances utility through personalization, it introduces significant privacy risks. Unlike traditional tracking via PII (Personally Identifiable Information), AI agents can infer fine-grained user attributes (interests, health status, financial standing) from non-PII context to build detailed profiles. These profiles are often used for advertising or shared with third parties.

Current privacy solutions face two main limitations:

PII Redaction: Removing named entities often fails to protect sensitive topics (e.g., removing "Mountain View" but leaving "seeking a therapist" still reveals mental health status).
Utility-Privacy Tradeoff: Aggressive obfuscation (e.g., removing all sensitive details) degrades the quality of the AI's response, while preserving utility often leaves the user vulnerable to profiling.

Existing approaches either rely on fixed, hand-crafted rules or delegate decisions entirely to an LLM without systematic optimization, failing to account for the varying importance of different sensitive units within a prompt.

Methodology: PromptPET

PromptPET is a user-side framework designed to obfuscate sensitive information in user prompts before they reach the AI agent, without requiring cooperation from the provider. It operates entirely on the client side using a local LLM. The framework consists of three main stages:

1. Sensitive Information Detection and Scoring

Taxonomy-Based Detection: PromptPET utilizes a sensitive taxonomy (specifically the IAB Audience Taxonomy) to identify "Sensitive Units" within a query. These units are mapped to specific Categories (e.g., Health) and Data Types (e.g., Depression).
Importance Scoring: Not all sensitive units contribute equally to the utility of the response. PromptPET assigns an importance score ( $[0, 1]$ ) to each detected unit using an instruction-tuned LLM. This score estimates the unit's contribution to generating the desired response, distinguishing between "fragile" high-contribution units and those that can be obfuscated aggressively.

2. Obfuscation Actions

PromptPET employs four distinct obfuscation strategies, applying them based on the unit's importance and context:

Redaction: Removes the unit and replaces it with a placeholder (e.g., [REDACTED]). This maximizes privacy but often destroys utility.
Abstraction: Generalizes the unit to a higher-level category in the taxonomy (e.g., replacing "Depression" with "Health"). This offers a middle ground.
Replacement: Substitutes the unit with a plausible alternative from a different data type within the same category (e.g., replacing "Mountain View" with "Palo Alto").
Noise (Novel): Retains the original sensitive unit intact but injects plausible, contextually relevant "decoy" information under a different data type. The AI agent is instructed to answer both the real and decoy queries. The client-side system then filters out the decoy portion of the response. This preserves utility for high-importance units while disrupting the profiler's ability to isolate the true signal.

3. Rule Optimization via Reinforcement Learning

Instead of using fixed heuristics, PromptPET introduces a Rule Optimizer inspired by Optimization by PROmpting (OPRO).

Mechanism: An LLM-based agent (the Obfuscation Action Decider) assigns an action to each sensitive unit based on a set of decision rules. A second agent (the Rule Optimizer) iteratively refines these rules.
Feedback Loop: The optimizer evaluates rule sets on a validation set using a reward function: $\alpha \cdot \text{Privacy} + (1-\alpha) \cdot \text{Utility} - \lambda \cdot |R|$ , where $|R|$ is the rule set length. It retains strong rules, revises mediocre ones, and prunes weak ones.
Outcome: The system learns a dynamic policy that selects the optimal obfuscation action for each specific unit, rather than applying a single strategy globally.

Key Results

The authors evaluated PromptPET on the WildChat-1M dataset (filtered for sensitive content) against a production AI agent service (simulated via GPT-4.1).

Single-Action Analysis

Noise Superiority: Among single-action strategies, the Noise scheme achieved the best privacy-utility tradeoff. At maximum strength, it achieved 73.7% success (high privacy, high utility), significantly outperforming Redaction (39.4%) and Abstraction (51.2%). Noise preserved utility by keeping the original signal intact while overwhelming the profiler with decoys.
Context Dependence: No single action was optimal for all queries. Redaction was best for incidental sensitive units, while Noise was essential for core query components.

PromptPET Performance

State-of-the-Art Comparison: PromptPET significantly outperformed prior automated baselines (Ngong et al. and Zhou et al.).
- It delivered 3.3× more privacy per unit of utility cost compared to the strongest prior baseline.
- It achieved 1.4× higher overall privacy (0.78 vs. 0.55) while maintaining high utility.
Frontier Achievement: The learned rule set allowed PromptPET to match the upper bound of the best single-action strategy at every operating point (e.g., using Abstraction for low-privacy needs and Noise for high-privacy needs), effectively "cooperating" rather than competing.
Failure Modes: The system primarily failed in two distinct ways:
1. Privacy Miss (17.5%): The profiler recovered the profile from residual context despite obfuscation.
2. Utility Collapse (7.6%): Occurred mostly when Redaction was incorrectly applied to high-importance units.
- Simultaneous failure of both axes was rare (1.1%).

Efficiency

Latency: PromptPET adds a median client-side latency of 11.6 seconds per query (dominated by the detection step using a 12B parameter model), resulting in a total end-to-end time of ~17.1 seconds. The authors note this is a secondary concern to the privacy-utility tradeoff and could be reduced with smaller, specialized detectors.

Significance and Claims

The paper claims that PromptPET represents a principled and unifying framework for prompt obfuscation. Its primary contributions are:

User-Side Control: It demonstrates that controlling data at the point of disclosure (client-side) is a viable and powerful Privacy-Enhancing Technology (PET) that does not rely on provider trust.
Novel Noise Mechanism: The introduction of the Noise/Denoise scheme provides a new method to preserve utility while disrupting profiling, outperforming traditional removal-based methods.
Systematic Optimization: By moving away from static rules to a reinforcement-learning-inspired rule optimizer, PromptPET dynamically adapts to the nuances of different queries, achieving a privacy-utility frontier that static methods cannot reach.
Comprehensive Evaluation: The work provides the first systematic comparison of redaction, abstraction, replacement, and noise, establishing that the optimal strategy is context-dependent and must be learned rather than hardcoded.

The authors conclude that PromptPET offers a practical path for users to interact with AI agents while retaining control over their privacy, effectively subverting user profiling without sacrificing the utility of the interaction.