InferDPT: Privacy-Preserving Inference for Closed-box Large Language Model

1. Problem Statement

Large Language Models (LLMs) like ChatGPT have revolutionized text generation but introduce significant privacy risks. When users upload prompts containing sensitive information (e.g., personal data, confidential documents) to black-box LLMs (where the model architecture and parameters are hidden), there is a risk of data leakage.

• The Challenge: Existing privacy-preserving solutions face a trade-off between utility (text quality) and privacy.
  • Homomorphic Encryption (e.g., CipherGPT): Too computationally expensive and slow for practical use.
  • Static Perturbation (e.g., SANTEXT+, CUSTEXT+): Designed primarily for classification tasks. When applied to text generation, they cause severe semantic distortion (incoherence) or are vulnerable to embedding inversion attacks because they use static adjacency lists (fixed sets of replacement tokens).
• The Goal: Develop a practical framework for privacy-preserving inference on black-box LLMs that protects user prompts via Differential Privacy (DP) while maintaining high-quality text generation.

2. Methodology: The InferDPT Framework

The authors propose InferDPT, a two-module framework that implements Local Differential Privacy (LDP) specifically for text generation tasks.

A. Perturbation Module (Privacy Protection)

• Function: Replaces every token in the user's raw document ($Doc$) with a perturbed token to create a perturbed document ($Doc_p$).
• Mechanism: Uses an LDP mechanism to sample new tokens from a specific "adjacency list" based on semantic similarity (embedding distance).
• Innovation (RANTEXT): The paper introduces a novel mechanism called RANTEXT to replace existing static methods.
  • Random Adjacency List: Instead of a fixed list (like SANTEXT+), RANTEXT dynamically generates a Random Adjacency List ($C_r$) for each token.
  • Process:
    1. Compute a random embedding for the raw token by adding Laplace noise.
    2. Define the adjacency list as all tokens in the vocabulary whose embeddings are closer to the original token than the noisy embedding.
    3. Sample a replacement token from this list using the Exponential Mechanism.
  • Advantage: The list size is dynamic and probabilistic. It is generally smaller than SANTEXT+ (improving semantic utility) but larger than CUSTEXT+ (improving resistance to reconstruction attacks).

B. Extraction Module (Utility Restoration)

• The Observation: The authors observed that even when the prompt is perturbed, the LLM's generated output ($Gen_p$) often shares a significant number of tokens with the generation from the original unperturbed prompt. The overlap correlates positively with the privacy parameter $\epsilon$.
• Function: A lightweight local language model (smaller than the remote black-box LLM) receives both the perturbed generation ($Gen_p$) and the raw document ($Doc$).
• Process: The local model acts as a "distiller." It extracts coherent and consistent text from $Gen_p$ and reconstructs it to align with the context of the raw document $Doc$. This effectively recovers the high-quality generation capabilities of the remote LLM while keeping the raw prompt private.

3. Key Contributions

1. InferDPT Framework: The first practical framework for privacy-preserving inference on black-box LLMs, implementing Differential Privacy in text generation. It uniquely combines perturbation with a local extraction/distillation step.
2. RANTEXT Mechanism: A novel LDP mechanism using Random Adjacency Lists. It dynamically adjusts the size of the token replacement pool based on the privacy parameter $\epsilon$, achieving a superior balance between privacy and utility compared to static methods.
3. Adaptive Attack (GPT Inference Attack): The authors propose a new attack vector where an adversary uses a powerful LLM (GPT-4) to reconstruct raw tokens from perturbed text, demonstrating that traditional methods are vulnerable.
4. Comprehensive Evaluation: Extensive experiments across three datasets (CNN/Daily Mail, Wikitext-103, ArXiv) evaluating privacy against multiple attack types (BERT inference, Embedding Inversion, GPT Inference) and utility metrics (Diversity, MAUVE, Coherence).

4. Experimental Results

• Privacy Protection:
  • Embedding Inversion Attack: With $\epsilon = 6.0$, RANTEXT achieved an average privacy protection level > 0.90. This is 3.35$\times$ higher than CUSTEXT+ and 0.58$\times$ higher than SANTEXT+.
  • GPT Inference Attack: RANTEXT maintained the best privacy protection against the adaptive GPT-based attack, outperforming baselines significantly.
• Utility (Text Quality):
  • Generation Quality: InferDPT with RANTEXT produced text quality comparable to non-private GPT-4.
  • Metrics: In terms of Coherence and Diversity, InferDPT + RANTEXT outperformed SANTEXT+ and CUSTEXT+.
  • MAUVE: While slightly lower than direct GPT-4 usage (due to semantic perturbation), it was significantly better than local models running alone and competitive with other DP methods.
• Efficiency: The framework adds minimal overhead. The perturbation module is fast, and the extraction module (local model) takes less than 4 seconds per inference on standard hardware.

5. Significance

• Practicality: Unlike homomorphic encryption, InferDPT is computationally feasible for real-world deployment.
• Black-Box Compatibility: It does not require access to the LLM's internal weights or architecture, making it applicable to commercial APIs (e.g., OpenAI, Anthropic).
• Paradigm Shift: It moves beyond simple noise addition (which breaks text generation) by utilizing a distillation approach (Perturb $\to$ Remote Generate $\to$ Local Reconstruct) to recover utility.
• Security: It addresses the specific vulnerability of static adjacency lists to embedding inversion attacks, offering a more robust defense for sensitive text generation tasks.

In conclusion, InferDPT demonstrates that it is possible to use powerful, closed-source LLMs for sensitive tasks without compromising user privacy, provided a carefully designed perturbation mechanism (RANTEXT) and a local reconstruction strategy are employed.