EPSVec: Efficient and Private Synthetic Data Generation via Dataset Vectors

1. Problem Definition

Modern machine learning relies heavily on high-quality data, but valuable corpora (e.g., biomedical records, internal documents, user feedback) are often sensitive and cannot be shared due to privacy regulations. While Large Language Models (LLMs) offer a solution for generating synthetic data, existing private generation methods face three critical limitations:

• Inefficiency: They are computationally expensive and data-intensive.
• Privacy Costs: Many methods incur privacy costs per generated token or require large batch sizes, making long-form generation or large-scale sampling prohibitively expensive under strict Differential Privacy (DP) budgets.
• Low Fidelity in Low-Data Regimes: Existing methods often fail to capture the nuanced distributional properties of the target data when only a small amount of private data is available.

The core challenge is to design a method that generates high-fidelity synthetic text with rigorous privacy guarantees, while remaining computationally lightweight and effective even with limited private data.

2. Methodology: EPSVEC

The authors propose EPSVEC (Efficient and Private Synthetic data generation via dataset Vectors), a training-free approach that shifts the unit of privatization from individual tokens to a compact, dataset-level representation.

Core Components

1. Dataset Vectors:

   • Concept: Based on the Linear Representation Hypothesis, the method posits that the distributional shift between a public prior (synthetic data generated via zero-shot prompting) and a private target dataset can be encoded as a single direction (vector) in the LLM's activation space.
   • Extraction: The model extracts hidden states from both private data ($D^+$) and synthetic "negative" data ($D^-$) generated via zero-shot prompting. It computes the mean difference vector between these states across specific layers.
   • Privatization: To ensure Differential Privacy (DP), the difference vectors are clipped to bound sensitivity and then perturbed with Gaussian noise. This results in a Private Dataset Vector.
   • Key Advantage: This vector is computed and privatized once. Once released, it can be reused to generate an arbitrary number of synthetic samples with zero additional privacy cost, decoupling the privacy budget from the generation process.

2. Vector Injection (Steering):

   • During inference, the privatized dataset vector is injected into the hidden states of the LLM at specific layers during decoding.
   • The injection follows the formula: $z_{\ell,t} \leftarrow z_{\ell,t} + \beta_\ell v_\ell$, where $z$ is the hidden state, $v$ is the vector, and $\beta$ is a steering coefficient. This steers the generation toward the target private distribution without requiring fine-tuning.

3. Fixed-Shot Prompting with Pretrained Models:

   • Model Choice: The authors demonstrate that Pretrained (Base) models often yield higher diversity and fidelity than Instruction-Tuned (IT) models, which tend to collapse into repetitive, simplistic templates.
   • Fixed-Shot Scaffolding: To stabilize Base models (which are harder to control than IT models), the method uses a small set of "fixed-shot" exemplars.
   • Private Selection: These exemplars are not taken directly from the private data. Instead, a candidate pool is generated, and a DP-histogram mechanism is used to select the best-matching candidates that represent the private distribution. These fixed shots are then used as a constant scaffold for both vector extraction and final generation.

3. Key Contributions

• Novel Privacy Mechanism: EPSVEC introduces a "one-time privatization" paradigm. By privatizing a dataset vector rather than individual tokens, it allows for infinite downstream generation without further privacy degradation.
• Efficiency: The method eliminates the need for large batch sizes or per-token privacy aggregation, significantly reducing computational overhead compared to inference-time aggregation baselines (like Private Prediction).
• High Fidelity in Low-Data Regimes: The method excels when private data is scarce, leveraging the dataset vector to capture subtle stylistic and semantic properties that zero-shot prompting misses.
• Base Model Optimization: The paper provides empirical evidence that combining Base models with DP-selected fixed-shot scaffolds outperforms Instruction-Tuned models for synthetic data generation, addressing the "diversity collapse" issue in IT models.

4. Experimental Results

The authors evaluated EPSVEC on four datasets: IMDb (movie reviews), Yelp (commercial reviews), BioRxiv (academic abstracts), and OpenReview (paper reviews).

• Distributional Fidelity (MAUVE Score): EPSVEC significantly outperforms existing baselines (AUG-PE, InvisibleInk, Private Prediction).
  • On the IMDb dataset, EPSVEC++ achieved a MAUVE score of 72.3 (at $\epsilon=5$), a massive improvement over the next-best method (PP++ at 14.9).
  • It achieved an average 150% improvement in MAUVE over the next-best method across four datasets.
• Downstream Utility: When used to fine-tune a BERT classifier, synthetic data from EPSVEC achieved accuracy comparable to models trained on real data.
• Scalability & Efficiency:
  • EPSVEC can generate arbitrary amounts of data and sequence lengths without increasing privacy costs.
  • In contrast, baselines like PP++ failed to generate the full 2,000 samples on several datasets under strict privacy budgets due to computational bottlenecks.
  • EPSVEC requires significantly fewer private samples to achieve high performance compared to other methods.
• Ablation Studies:
  • Removing the dataset vector steering drastically reduced fidelity.
  • Using Pretrained (Base) models with fixed-shots yielded higher fidelity than Instruction-Tuned models.
  • The DP-histogram selection of fixed shots was crucial for improving the reference distribution.

5. Significance and Impact

• Practical Deployment: EPSVEC offers a practical solution for organizations needing to share synthetic data for research, benchmarking, or data augmentation without exposing sensitive user information.
• Cost Reduction: By decoupling privacy costs from generation volume, it makes large-scale synthetic data generation economically and computationally feasible.
• Theoretical Insight: The work validates the utility of dataset vectors as a compact representation of complex distributional shifts, suggesting that high-level dataset properties can be encoded and manipulated linearly in LLM activation spaces.
• Future Directions: The paper highlights that moderate privacy noise can sometimes act as a regularizer to improve diversity, opening new avenues for research into the interplay between noise injection and generation quality.

In summary, EPSVEC represents a paradigm shift in private synthetic data generation, moving from token-level privacy constraints to efficient, dataset-level steering, thereby achieving state-of-the-art fidelity with minimal computational and privacy overhead.