Technical Summary: When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation

1. Problem Statement

Large Language Models (LLMs) have emerged as state-of-the-art generators for synthetic tabular data, utilizing two primary regimes: In-Context Learning (ICL), where examples are provided in the context window, and Supervised Fine-Tuning (SFT), where models are trained on string-encoded tabular data. While these methods offer high statistical fidelity and utility for downstream tasks, they introduce a novel privacy vulnerability distinct from traditional generative models (e.g., GANs, VAEs).

Traditional tabular generators operate directly in the feature space, treating samples as atomic units. In contrast, LLMs decompose tabular rows into autoregressive string sequences (token-by-token). This paper posits that this architectural shift creates a unique attack surface: LLMs tend to memorize and reproduce rigid, structured patterns of numeric digits from their training data. Because tabular data lacks the semantic ambiguity of natural language (where paraphrasing is possible), even minor character-level reproductions constitute exact memorization of sensitive information (e.g., financial amounts, identifiers).

Existing Membership Inference Attacks (MIAs) for synthetic data focus on feature-space distances (e.g., Euclidean distance, density estimation). The authors argue these methods fail to detect vulnerabilities inherent to the intermediate string representation used by LLMs, leaving a critical gap in privacy auditing for LLM-based tabular generators.

2. Methodology

2.1 Threat Model

The study adopts a conservative No-box threat model. The adversary has access only to the generated synthetic dataset ($S$). The adversary does not have:

• Access to the model's internals, weights, or hyperparameters.
• Query access to the model.
• A reference dataset or auxiliary data for calibration.
  This model reflects realistic scenarios where synthetic data is shared publicly without disclosure of the generation process.

2.2 The Attack: LevAtt (Levenshtein Attack)

To exploit the string-based nature of LLM generation, the authors propose LevAtt, a No-box MIA based on string similarity rather than feature-space proximity.

• Mechanism: The attack converts tabular records into canonical string representations, preserving structure (consistent precision for numbers, stable tokens for categories). It then calculates the Levenshtein edit distance (character-level edit distance) between a test observation ($x^*$) and the nearest synthetic record in the generated dataset.
• Scoring: The attack score is defined as $f(x^*_{str}, S_{str}) = -\min_{s \in S_{str}} \ell(x^*_{str}, s)$, where $\ell$ is the Levenshtein distance. Lower distances (higher scores) indicate a closer match, suggesting the synthetic data memorized the training record.
• Rationale: Unlike natural language, tabular numeric strings are rigid. A single-digit change alters the value meaningfully. Therefore, a near-exact string match is a strong, high-confidence signal of memorization that feature-space attacks miss.

2.3 Proposed Defenses

The paper evaluates two strategies to mitigate this leakage:

1. Digit Modifier (DM): A post-processing algorithm that iterates through numerical columns and flips digits based on a probability function $g$. Larger magnitude values receive higher perturbation probabilities. This is applied after generation.
2. Tendency-based Logit Processor (TLP): A sampling-time defense integrated into the generation process. It applies a concave, monotonic curving function ($f_t(x) = x^{1/t}$) to the model's logits before sampling. This suppresses high-confidence tokens (often memorized sequences) and amplifies lower-probability tokens, introducing controlled randomness without breaking feature dependencies.

3. Experimental Results

3.1 ICL Regime (In-Context Learning)

Experiments were conducted on 35 real-world regression datasets using LLaMA 3.3-70B, GPT-4o-mini, and TabPFN-v2.

• Leakage: LevAtt demonstrated substantial privacy leakage. For TabPFN-v2 and LLaMA 3.3-70B, the attack achieved perfect membership classification (AUC-ROC $\approx$ 1.0) in the top 20 worst-case runs.
• Correlation: LevAtt showed weak correlation with traditional feature-space MIAs (DCR, Density Estimation, MC), confirming it detects a unique, orthogonal leakage signal.
• Scaling: Privacy leakage increased with model size (70B models leaked more than 7B-scale models) and decreased with the number of in-context examples (fewer examples led to higher memorization).

3.2 SFT Regime (Supervised Fine-Tuning)

Experiments involved fine-tuning models (LLaMA 3.2, Qwen2.5, Mistral, GPT-2) on datasets like CASP, Diabetes, and CA-Housing, benchmarked against RealTabFormer and GREAT.

• Leakage: SFT models exhibited lower but still significant leakage compared to ICL. RealTabFormer, despite claims of privacy awareness, showed the highest vulnerability (LevAtt AUC $\approx$ 0.65–0.70).
• Conventional Models: Traditional deep learning generators (CT-GAN, TVAE) were immune to LevAtt, as they do not generate token sequences.
• Factors: Leakage increased monotonically with the volume of synthetic data generated and the length of digit sequences in the training data.
• Trade-off: A negative correlation was observed between privacy (LevAtt AUC) and fidelity (MMD, Wasserstein distance). Models with higher statistical fidelity were more prone to string memorization.

3.3 Defense Evaluation

• Digit Modifier (DM): Successfully reduced LevAtt AUC but caused a catastrophic drop in data fidelity. The noise injection pushed values out of the valid distribution, rendering the data useless for downstream tasks.
• TLP: Successfully reduced LevAtt AUC to near-random levels ($\approx$ 0.55) and TPR to $<0.125$ while maintaining statistical fidelity (MMD and Wasserstein distances remained comparable to vanilla generation) and downstream utility (XGBoost performance).
• Differential Privacy (DP): While DP2Stage (DP-SGD) effectively defeated LevAtt, it resulted in the worst utility and fidelity among all models, highlighting the severe privacy-utility trade-off of training-time DP for LLMs.

4. Key Contributions

1. Identification of String Memorization: The paper identifies that LLM-based tabular generators memorize rigid numeric string patterns, a vulnerability invisible to feature-space attacks.
2. LevAtt Attack: Introduction of a simple, No-box MIA using Levenshtein distance that achieves perfect classification on state-of-the-art models under minimal adversarial assumptions.
3. Empirical Evidence of Risk: Demonstration that high-fidelity SFT and ICL models (including RealTabFormer) leak membership information, whereas conventional generators do not.
4. Effective Defense (TLP): Proposal of TLP, a sampling-time mechanism that mitigates memorization by perturbing logits, successfully balancing privacy protection with data utility and fidelity.

5. Significance and Claims

The authors claim that LLM-based tabular data generation introduces a fundamentally new privacy risk distinct from traditional generative models. The significance of this work lies in:

• Redefining the Attack Surface: Shifting the focus from feature-space distance to character-level string similarity, which is critical for autoregressive models.
• Challenging Privacy Assumptions: Demonstrating that even "privacy-aware" SFT methods and high-fidelity generators can catastrophically leak training data when the data structure is rigid.
• Practical Mitigation: Providing a deployable defense (TLP) that does not require retraining or sacrifice data utility, unlike Differential Privacy.

The paper concludes that while LLMs offer superior utility, their autoregressive nature makes them prone to "retrieval" rather than "generalization" in tabular contexts. The authors emphasize that without specific defenses like TLP, the deployment of LLM-based synthetic tabular data in privacy-sensitive sectors (healthcare, finance) poses a significant re-identification risk.

Limitations Noted:

• LevAtt relies on consistent string formatting; formatting variations (e.g., "17.5" vs "17.500") could obscure signals.
• The defenses (DM, TLP) are empirical and lack formal privacy guarantees (unlike DP).
• The study assumes a No-box model; more powerful attacks with model access could be more effective.
• The attack is less effective on open-ended text fields compared to rigid numeric/categorical schemas.