Technical Summary: The Reliability Gap in Benchmark Auditing

Problem Statement

The validity of Large Language Model (LLM) assessment is increasingly threatened by benchmark contamination, where evaluation examples appear in a model's training data. While statistical tools exist to detect training-data membership (e.g., LLM Dataset Inference, Post-Hoc Dataset Inference, and CoDeC), they have been validated almost exclusively in controlled academic regimes characterized by large, homogeneous pre-training corpora and transparent, single-stage training pipelines.

It remains unclear whether these methods remain reliable in realistic auditing scenarios. The authors identify two under-studied failure modes that may compromise detection in the wild:

1. Distribution Shift: Suspect and validation sets violating the Independent and Identically Distributed (IID) assumption (e.g., comparing a benchmark train split to its test split).
2. Scale Constraints: Benchmarks are orders of magnitude smaller than pre-training corpora, limiting the efficacy of methods that rely on large data volumes for synthetic generation or statistical power.

Methodology

The authors systematically evaluate three leading contamination detection paradigms across 27 models (including Pythia, OLMo 2, specialized medical and Polish LLMs) and frontier industry models. The evaluation spans 335 distinct detection challenges across four tasks:

1. Task 1 (Limited Reference Data): Simulating benchmark-scale auditing by restricting reference data to 2,000 documents per dataset (subsampled from The Pile) to test performance under data scarcity.
2. Task 2 (Split-Level Exposure): Auditing instruction-tuned OLMo 2 models to determine if methods can distinguish between seen (train) and unseen (test) splits of the same benchmark (GSM8K, Competition Math, MMLU, DROP).
3. Task 3 (Specialized Post-Training): Evaluating detection on domain-specific datasets (Medical QA and Polish-language models) where held-out splits are small and distributions are specialized.
4. Task 4 (Industry Models): Applying comparative auditing to closed-source industry models (Gemma 3, Qwen 3, Llama 3.2) where ground-truth membership is unknown, focusing on signal consistency across families.

Evaluated Methods:

• LLM Dataset Inference (LLM DI): Aggregates membership inference signals (loss/perplexity) across a suspect set and compares them to an unseen validation set via a one-sided t-test.
• Post-Hoc Dataset Inference (Post-Hoc DI): Constructs a synthetic validation set using a generator model trained on the suspect corpus to relax the need for natural unseen data.
• CoDeC (Contamination Detection via Context): Measures how adding in-context examples from the same dataset affects next-token likelihood; a drop in confidence indicates potential memorization.

Key Results

Across 335 evaluations, only 199 (approx. 60%) yielded correct outcomes. The study reveals systematic reliability gaps:

• LLM Dataset Inference:
  • Success: Works reliably in controlled settings (Task 1) where a genuinely unseen and IID validation set is available.
  • Failure: Generates significant false positives under distribution shift. When benchmark test splits are used as validation proxies, the method mistakes distributional differences (difficulty, style) for training exposure (e.g., flagging MMLU and DROP as contaminated for OLMo 2).
• Post-Hoc Dataset Inference:
  • Failure: Is underpowered at benchmark scale. With only ~2,000 samples, the generator cannot produce realistic synthetic validation data. The statistical signal is dominated by the distribution shift between real and synthetic text rather than actual training membership, leading to uninformative calibration statistics.
• CoDeC:
  • Limitation: Provides only coarse provenance signals. While it effectively separates pre-training corpora from evaluation-only benchmarks in aggregate, it fails to distinguish between train and test splits of the same benchmark. Scores are often similar for seen and unseen splits, making it insufficient for verifying individual benchmark integrity.
• Specialized & Industry Models:
  • Detection performance degrades further in specialized domains (medical, Polish) due to small dataset sizes.
  • For industry models, signals are suggestive rather than definitive. For instance, CoDeC and LLM DI consistently flagged most families on GSM8K, except for Google's Gemma-3, which showed lower scores and was not flagged, suggesting it was likely not trained on GSM8K.

Key Contributions

1. First In-the-Wild Evaluation: The paper provides the first systematic evaluation of these detection paradigms beyond controlled academic corpora, applying them to realistic, multi-stage instruction-tuned models.
2. Identification of Failure Modes: The authors rigorously characterize distribution shift and scale constraints as primary causes of failure, demonstrating that methods validated on massive pre-training corpora do not transfer directly to small benchmark auditing.
3. Open-Source Benchmark: The authors release a codebase and benchmark to facilitate further research into contamination detection.

Significance and Claims

The paper concludes that there is currently no "silver bullet" for detecting contamination in realistic settings. The transition from academic validation to practical auditing is fraught with challenges that render current statistical tools unreliable as sole arbiters of leaderboard integrity.

The authors assert that transparent data provenance remains the most reliable basis for benchmark-integrity claims, with statistical auditing serving only as complementary evidence. They caution that relying on these methods without verifying distributional alignment or accounting for data scale can lead to misleading conclusions about model capabilities and dataset contamination.