Technical Summary: Gap-K% - Measuring Top-1 Prediction Gap for Detecting Pretraining Data

Problem Definition

The opacity of massive pretraining corpora in Large Language Models (LLMs) creates significant privacy and copyright risks, including the potential leakage of Personally Identifiable Information (PII) and copyrighted content. Furthermore, the inadvertent inclusion of benchmark datasets in pretraining data can inflate model performance, leading to unfair comparisons. The core challenge addressed is pretraining data detection, a specific instance of Membership Inference Attacks (MIA), which aims to determine whether a given text sequence was part of a model's training dataset.

Existing state-of-the-art methods (e.g., Min-K%, Min-K%++) primarily rely on token-level likelihoods, treating tokens independently. These approaches often overlook two critical aspects:

1. The gap between the model's top-1 prediction and the target token.
2. The local correlations between adjacent tokens within a sequence.

Methodology: Gap-K%

The authors propose Gap-K%, a detection method grounded in the optimization dynamics of autoregressive LLM pretraining.

Theoretical Insight

The method is motivated by the behavior of the next-token prediction objective. During training, the cross-entropy loss explicitly penalizes cases where the model's top-1 prediction ($v_{max}$) does not match the ground truth target token ($y_t$).

• Training Data: The model learns to minimize the gap between the log probability of the target token and the top-1 prediction. Consequently, for training samples, the target token is frequently the top-1 prediction, resulting in a minimal gap.
• Non-Training Data: For unseen data, the model relies on learned syntactic and semantic correlations rather than memorization. This often leads to "confident mispredictions," where the model assigns high probability to an incorrect token that is distinct from the target. This results in a larger log-probability gap between the target and the top-1 prediction.

Algorithmic Components

Gap-K% quantifies this phenomenon through three steps:

1. Top-1 Gap Scoring:
   For each token $x_t$, the method computes a normalized gap score $g_t$:
   $$g_t = \frac{\log p(x_t | x_{<t}) - \max_{v \in V} \log p(v | x_{<t})}{\sigma_t}$$
   Where $\sigma_t$ is the standard deviation of the log probabilities. Unlike Min-K%++, which normalizes relative to the mean of the distribution, Gap-K% normalizes relative to the mode (top-1 prediction). This allows the metric to distinguish between uncertain predictions (flat distribution) and confident mispredictions (peaked distribution on an incorrect token).

2. Sequential Smoothing:
   To capture local correlations and mitigate token-level fluctuations, a sliding window of size $w$ is applied to the gap scores:
   $$\bar{g}^{(w)}_t = \frac{1}{w} \sum_{i=0}^{w-1} g_{t+i}$$
   This step aggregates signals over contiguous segments, reflecting the observation that membership signals often span phrases rather than isolated tokens.

3. Gap-K% Metric:
   The final detection score is the average of the lowest $k\%$ of the smoothed gap scores in the sequence. A score closer to 0 indicates a small gap (suggesting membership), while a large negative score indicates a significant gap (suggesting non-membership).

Key Contributions

1. Identification of a New Signal: The authors identify the discrepancy between top-1 predictions and target tokens as an effective detection signal, grounded in the gradient dynamics of next-token prediction.
2. Novel Method (Gap-K%): They propose a method that leverages these top-1 prediction gaps and aggregates token-level signals via sequential smoothing to capture local correlations, addressing the limitations of independent token analysis.
3. State-of-the-Art Performance: The method achieves superior performance on standard benchmarks across various model sizes and input lengths.

Experimental Results

The authors evaluated Gap-K% on two benchmarks: WikiMIA (Wikipedia-based) and MIMIR (Pile-based, more challenging with minimal distributional differences).

• WikiMIA Performance:

  • Across five models (Mamba-1.4B, Pythia-6.9B/12B, LLaMA-13B/65B) and input lengths (32, 64, 128), Gap-K% consistently outperformed baselines including Loss, Zlib, Neighbor, Min-K%, and Min-K%++.
  • In the original setting, Gap-K% achieved absolute AUROC improvements of 9.7% over the average of existing baselines and 2.4% over the strongest baseline (Min-K%++).
  • In the paraphrased setting, it achieved gains of 5.7% over the baseline average and 1.7% over Min-K%++.
  • The method also demonstrated robustness on recent models (LLaMA 3.1, Gemma2) and their instruction-tuned variants.

• MIMIR Performance:

  • On the more challenging MIMIR benchmark, Gap-K% attained the highest average performance across Pythia models ranging from 1.4B to 12B parameters, outperforming Min-K%++ across all evaluated sizes.

• Ablation Studies:

  • Sequential Locality: Smoothing only improved performance when applied to the original token order, confirming that membership signals exhibit sequential locality.
  • Component Analysis: Replacing the mean-based normalization with the Top-1 gap alone yielded marginal gains, and smoothing alone improved performance moderately. The combination of both (the full Gap-K% method) yielded the largest performance gain, indicating that the Top-1 gap provides a discriminative signal that is stabilized and enhanced by sequential smoothing.
  • Robustness: Gap-K% remained effective under strong adversarial paraphrasing (using DIPPER), outperforming all baselines.

Significance and Limitations

Significance:
The paper argues that understanding the optimization dynamics of LLMs—specifically how training forces alignment between top-1 predictions and ground truth—provides a robust mechanism for detecting pretraining data. By moving beyond simple token likelihoods to analyze the "confidence" of mispredictions and their local context, Gap-K% offers a more nuanced and effective detection tool. This perspective highlights the importance of top-1 prediction behavior and sequential correlations in understanding memorization and privacy risks.

Limitations:

• Gray-Box Assumption: The method requires access to token-level probability outputs (logits), limiting its application to models or APIs that expose this information. It does not currently function in a fully black-box setting.
• Model Scope: While tested on recent families (LLaMA 3.1, Gemma2), the evaluation is limited to models up to tens of billions of parameters; behavior at the hundreds-of-billions scale is unverified.
• Adversarial Robustness: While robust to standard paraphrasing, the method has not been tested against adaptive adversaries specifically optimized to evade detection by manipulating token-level statistics.

The authors emphasize that their work is intended for non-commercial, academic research to improve transparency and privacy understanding in LLMs.