Technical Summary: Dataset Watermarking for Closed LLMs with Provable Detection

Problem Formulation

Large Language Models (LLMs) are frequently trained on vast, loosely curated datasets that may inadvertently include proprietary data or evaluation benchmarks. This phenomenon, often exacerbated by "benchmaxxing" (optimizing for benchmark performance during post-training), raises concerns regarding data contamination and the reliability of model evaluations. While recent work has explored dataset watermarking—embedding detectable signals into datasets prior to release to audit their usage in model training—existing methods suffer from two critical limitations:

1. Access Requirements: Prior approaches (e.g., STAMP, Radioactive) rely on open-model access, requiring visibility into model internals such as log-probabilities, next-token distributions, or perplexity scores. They are ineffective against closed-model access (black-box API), which is the standard for most commercial chatbots.
2. Lack of Theoretical Guarantees: Existing methods often lack rigorous, distribution-free theoretical bounds on the false positive rate (FPR) of detection.

This paper addresses the problem of detecting whether a specific dataset was used to train a target LLM, given only black-box API access to the model's outputs, while providing provable statistical guarantees on detection reliability.

Methodology

The proposed framework operates in two stages: Embedding and Detection. The core intuition leverages the known tendency of deep learning models to memorize spurious features and statistical correlations present in training data.

1. Embedding Stage (Watermark Injection)

The data owner rephrases the dataset to inject a detectable statistical signal without altering the dataset's utility:

• Secret Key Generation: A secret key ($sk$) is formed by randomly sampling a set of word pairs from the dataset's vocabulary.
• Lexical Variant Construction: For each word in the key, a set of lexical variants (synonyms, co-occurring terms) is generated.
• Rephrasing for Co-occurrence: The dataset is rephrased using an LLM (e.g., ChatGPT) to increase the co-occurrence frequency of the selected word pairs.
  • Samples containing variants of both words in a pair are preserved or enhanced.
  • Samples containing one word but not its pair are modified to include the pair.
  • Crucially, co-occurrence is defined at the sample level; words need not be consecutive, only present within the same sample.
    This process introduces controlled, artificial correlations that are unlikely to arise naturally in unwatermarked data.

2. Detection Stage (Statistical Testing)

Given a target model $M$ and the secret key $sk$, the auditor queries the model to generate text and analyzes the co-occurrence statistics of the word pairs.

• Correlation Measurement: The method computes the Pearson correlation between indicator variables for the presence of word pairs in generated text.
• Deviation Calculation: It calculates the deviation ($A_{w_i, w_j}$) between the co-occurrence behavior in the model's output and a reference (non-watermarked) dataset.
• Detection Score: A score is computed by comparing the fraction of secret key pairs exceeding a deviation threshold $\tau$ against the fraction of non-key pairs exceeding the same threshold.
• Decision Rule: If the detection score exceeds a threshold $t$, the presence of the watermark is declared.

Theoretical Guarantees

The paper provides a distribution-free and model-agnostic upper bound on the False Positive Rate (FPR).

• Theorem 4.1: Under the null hypothesis (the model was not trained on the watermarked data), the probability of a false positive is bounded by $e^{-2t'^2d}$, where $d$ is the size of the secret key and $t'$ is a function of the detection threshold.
• This guarantee holds for any fixed LLM architecture or training procedure, requiring no assumptions about the underlying data distribution or model internals.

Key Contributions

1. First Closed-Model Dataset Watermarking: The authors introduce the first dataset watermarking method capable of detecting training data usage using only black-box API access, eliminating the need for log-probabilities or internal model states.
2. Provable Detection: The method offers theoretical guarantees on the false positive rate that are independent of the model's architecture or the data distribution.
3. Robustness in Data Mixtures: The approach remains effective even when the watermarked dataset constitutes a small fraction (approx. 1%) of the total fine-tuning tokens.
4. Preservation of Utility: The watermarking process maintains the semantic integrity and utility of the benchmark datasets, ensuring they remain valid for model evaluation.

Experimental Results

The method was evaluated on LLaMA-3-8B and Gemma-2-2B across three benchmarks: MMLU, ARC-easy, and ARC-challenge.

• Vanilla Setting: The method achieved statistically significant detection ($p < 0.01$) across all models and datasets. In contrast, the "Radioactive" baseline failed to produce meaningful signals under closed-model access ($p \approx 0.2\text{--}0.3$).
• Partial Contamination (Data Mixture): The method successfully detected watermarks when the watermarked data comprised only 1.1%–1.3% of the total fine-tuning tokens. At a more extreme 0.5% ratio, LLaMA-3-8B maintained significant detection, while Gemma-2-2B showed reduced sensitivity, suggesting larger models may better retain injected statistical signals.
• Robustness to Modifications: The watermark demonstrated superior robustness against lightweight text modifications (random deletion, synonym substitution, emoji insertion) compared to baselines. While baselines like Radioactive saw p-values rise above significance thresholds after 20% modification, the proposed method maintained low p-values.
• Utility Preservation:
  • Benchmark Performance: Models evaluated on watermarked datasets showed performance comparable to those on original datasets, preserving the relative ranking of models.
  • Semantic Similarity: The rephrased datasets maintained high semantic similarity (SBERT scores) to the originals.
  • Lexical Overlap: The proposed method achieved significantly higher n-gram overlap (3-gram, 4-gram) compared to KGW-based watermarking, indicating less aggressive rewriting.

Significance and Limitations

The paper positions this work as a tool for auditing dataset usage, enabling data providers to verify if their proprietary or benchmark data has been incorporated into model training via black-box queries.

Limitations acknowledged by the authors:

• Proactive Requirement: The method requires rephrasing the dataset before release; it cannot be applied retroactively to already released, unwatermarked datasets.
• Adversarial Robustness: The method is not designed to withstand fully adversarial transformations intended to eliminate statistical signals, as strong watermarking guarantees are theoretically impossible under arbitrary rewriting.
• Evaluation Scope: Experiments were limited to the post-training (fine-tuning) stage. Evaluating effectiveness at the pre-training scale remains an open problem due to computational constraints.

In conclusion, this work establishes a new paradigm for dataset watermarking that is compatible with the closed-model ecosystem, offering provable detection capabilities while preserving the utility of the underlying benchmarks.