Technical Summary: CapBencher

Problem Statement

The publication of Large Language Model (LLM) benchmarks, particularly the release of ground-truth answers, poses a significant risk of data contamination and test-set overfitting. When benchmarks are public, models may be unintentionally trained on them, or developers may intentionally "game" leaderboards by iteratively querying private evaluation servers to overfit their models. While keeping benchmarks private and using a "submit-and-evaluate" protocol is a common mitigation, it fails to prevent overfitting through repeated queries (leaderboard hacking) and does not offer a robust method to detect if a model has already seen the test data. Existing detection methods often require access to model internals (log probabilities) or specific benchmark text (canary strings), making them unsuitable for black-box models or scenarios where the benchmark text itself must remain hidden.

Methodology: CapBencher

The authors propose CapBencher, a protocol for publishing benchmarks that obscures ground-truth answers while preserving the ability to evaluate and rank models. The core mechanism involves intentionally reducing the Bayes accuracy (the theoretical maximum accuracy achievable by an optimal decision rule) of the benchmark.

Core Mechanism

Instead of revealing a single correct answer, CapBencher prepares a set of multiple logically correct answers, $F(x)$, for each question $x$. The benchmark then randomly selects one realized answer $Y' \in F(x)$ to serve as the solution.

• Obfuscation Strategy: The true answer is hidden. For example, in a direct-answer task like "What is 3 times 6?", the instruction might be: "Randomly add 1 or subtract 1 from your solution." The true answer is 18, but the realized answer $Y'$ is randomly chosen from $\{17, 19\}$. If all questions are randomized this way, the Bayes accuracy drops to 50%.
• Disclosure-Allowed Strategy: In cases where revealing the true answer is acceptable (e.g., to maintain a private leaderboard), the method still injects randomness to make contamination detectable. For instance, appending a random word (e.g., "apple" or "orange") to the correct answer.

Statistical Detection

The reduction in Bayes accuracy creates a statistical "ceiling." If a model's performance on the capped benchmark significantly exceeds this ceiling, it serves as a strong signal of data contamination (memorization of the specific randomized answers).

• Binomial Test: The authors propose using a one-sided binomial test to determine if the observed accuracy is statistically significantly higher than the known Bayes accuracy $\alpha$. The null hypothesis assumes the model's performance is at or below $\alpha$.
• Unbiased Estimation: Despite the noise introduced by randomization, the paper derives an unbiased estimator to reconstruct the original accuracy score from the capped score, allowing for the tracking of model progress. The relationship between capped and original scores is shown to be an increasing affine mapping under specific assumptions (uniform shifts, independence, and random permutations of options).

Key Contributions

1. Novel Benchmark Protocol: A method to publish benchmarks that obfuscates ground-truth answers by capping the Bayes accuracy, thereby preventing models from achieving perfect scores even if they solve the underlying problem.
2. Built-in Contamination Alarm: A statistical framework (using binomial tests) that detects test-set overfitting and data leakage without requiring access to model internals or the original benchmark text.
3. Theoretical Guarantees: The paper provides theoretical proofs showing that:
   • The capped score is an affine transformation of the original score, preserving model rankings.
   • An unbiased estimator exists to recover the original accuracy.
   • The binomial test is the uniformly most powerful test for detecting contamination in this context.
4. Empirical Validation: Extensive experiments across diverse benchmarks (MMLU, GSM8K, GPQA, HumanEval, etc.), model families (Llama, Qwen, DeepSeek), and training methodologies (pretraining, continual pretraining, model merging).

Results

• Tracking Progress: CapBencher successfully tracks LLM improvements. The correlation between capped and original accuracy rankings is high (Kendall's $\tau \approx 0.92$ across benchmarks), demonstrating that the method does not obscure model progress.
• Contamination Detection: The method reliably detects data contamination. In experiments where models were continuously pre-trained on capped benchmark data, their accuracy consistently surpassed the Bayes accuracy ceiling (e.g., 50%), triggering the binomial test alarm.
• Robustness:
  • Deep Contamination: The method detects contamination even when models are trained on rephrased or translated versions of the data (cross-lingual contamination), though detection is harder when reasoning is required in non-training languages.
  • Reverse Engineering: Experiments showed that even with access to the randomized answer and the randomization rule, models struggled to reliably reverse-engineer the original ground truth. In many cases, models failed to follow the randomization instruction or could not deduce the original answer, suggesting the "weak clue" leaked by the method is insufficient for sophisticated attacks.
  • Comparison to Baselines: CapBencher outperformed state-of-the-art baselines like Min-k% and canary string methods. Unlike Min-k%, it does not require model logits (working with black-box models) and does not rely on asymptotic approximations. Unlike canary strings, it is resilient to the removal of specific strings.

Significance and Claims

The paper positions CapBencher as a step toward more durable and reliable LLM evaluation. By reducing the Bayes accuracy, benchmark creators can publish questions while turning any statistically significant performance above the cap into a "black-box signal" of leakage.

The authors claim that this approach:

• Preserves Evaluation Integrity: It allows for open evaluation of LLMs without fully disclosing ground-truth answers.
• Detects Overfitting: It provides a principled, statistical method to detect test-set overfitting and leaderboard hacking, even in private evaluation settings where repeated queries are made.
• Is Model-Agnostic: It works with black-box models and does not require access to internal model states.

The paper acknowledges limitations, noting that the method relies on the assumption that models will not perfectly reverse-engineer the randomized answers. While current frontier models could not reliably exploit the leaked signal, the authors note that stronger future models might succeed, suggesting future work in stronger obfuscation strategies and cryptographic extensions (e.g., zero-knowledge proofs). The method also introduces a trade-off: lower Bayes accuracy (tighter caps) makes contamination easier to detect but increases the variance of the accuracy estimator, potentially making it harder to distinguish between very similar models.