💬 NLP

How Much Do Large Language Model Cheat on Evaluation? Benchmarking Overestimation under the One-Time-Pad-Based Framework

This paper introduces ArxivRoll, a dynamic evaluation framework inspired by one-time pad encryption that utilizes a semi-annual, automated benchmark generated from recent ArXiv articles to quantify and mitigate LLM overestimation caused by data contamination and training bias.

Original authors: Zi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zi Liang, Liantong Yu, Shiyu Zhang, Qingqing Ye, Haibo Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Open-Book" Test

Imagine you are a teacher trying to test how smart your students are. You give them a standard math test. But, the students have secretly memorized the answers to that specific test from a study guide they found online.

When they take the test, they get 100% on the questions they memorized. But if you ask them a new math problem they haven't seen before, they might fail.

This is exactly what is happening with Large Language Models (LLMs) (like the AI you talk to).

The Cheat: Many AI models are being trained on data that includes the answers to popular tests (benchmarks) used to measure their intelligence.
The Result: They aren't actually "learning" or "reasoning"; they are just reciting answers they've seen before. This makes them look smarter than they really are, leading to unfair comparisons between different AI companies.

The Solution: ArxivRoll (The "One-Time Pad" Test)

The researchers from The Hong Kong Polytechnic University created a new system called ArxivRoll to catch these cheaters. They took inspiration from a concept in cryptography called the "One-Time Pad."

The Analogy: The Disposable Secret Key
In cryptography, a "One-Time Pad" is a secret key used to encrypt a message. The rule is: Use the key once, then throw it away. If you use the same key twice, the secret is compromised.

ArxivRoll applies this rule to testing:

Fresh Ingredients: Instead of using old, static test questions, they generate new tests every six months using brand-new research papers from ArXiv (a website where scientists post their latest, unpublished work).
The "Unseen" Test: Because these papers are so new, no AI has ever seen them during its training. It's like giving a student a test based on a book that was just printed yesterday.
Use and Discard: Once the test is over, the answers are made public so everyone can check the work, but the specific test questions are marked "expired" and never used again. This ensures no future AI can memorize them.

How the Test Works: The "Fill-in-the-Blanks" Game

To make these tests automatically (without needing humans to write thousands of questions), they use a method called SCP (Sequencing, Cloze, and Prediction). Think of it as a high-level version of a "Mad Libs" or a jigsaw puzzle:

Sequencing: The AI is given a paragraph where the sentences are shuffled like a deck of cards. It has to put them back in the correct order.
Cloze: The AI is given a paragraph with some sentences missing (masked). It has to pick the right sentence to fill the gap from a list of options.
Prediction: The AI reads a story and has to guess what the very next sentence will be, choosing from four different options.

Because these questions are generated randomly from fresh text, the AI can't just "memorize" the pattern; it actually has to understand the logic.

The Scorecard: "Rugged Scores"

The paper introduces a new way to score these models called Rugged Scores (RS). Imagine you are judging a runner:

Public Score: How fast they run on the track they practiced on (the old, contaminated tests).
Private Score: How fast they run on a brand new, unknown trail (the ArxivRoll test).

The Rugged Score measures the gap between these two.

Low Rugged Score: The runner performed similarly on both tracks. They are genuinely fit.
High Rugged Score: The runner was super fast on the practice track but slow on the new trail. This means they were "cheating" by memorizing the practice track.

What They Found

The researchers tested many popular AI models (like Llama, Qwen, GPT, and Claude) using this new system.

The "Overestimation" is Real: Many models that looked like geniuses on public leaderboards dropped significantly in rank when tested on the fresh, private ArxivRoll tests.
Some Models are "Overtrained": They found that some models were specifically tuned to crush specific types of questions (like math or finance) but failed miserably at others. It's like a student who only studied for the history final but failed the science quiz.
The Gap is Huge: For some models, the difference between their "Public Score" and "Private Score" was massive, proving that a large chunk of their reported intelligence was actually just memorization.

Summary

The paper argues that we have been fooled by AI benchmarks. By using a system that constantly refreshes its questions with brand-new, unseen research (ArxivRoll) and measures the gap between old and new tests (Rugged Scores), they can see how much of an AI's "intelligence" is real and how much is just cheating. It's a way to ensure that when we say an AI is smart, it actually is smart, not just good at memorizing the test.

Technical Summary: ArxivRoll – Benchmarking Overestimation Under The One-Time-Pad-Based Framework

1. Problem Statement

The evaluation of Large Language Models (LLMs) faces a critical challenge: overestimation of capabilities due to data contamination and biased overtraining.

Data Contamination: Test samples from public benchmarks (e.g., MMLU, GSM8K) are often inadvertently or intentionally included in the pre-training or fine-tuning datasets of LLMs. This allows models to "memorize" answers rather than demonstrate genuine reasoning, leading to inflated performance metrics.
Biased Overtraining: Developers may strategically optimize models for specific benchmark domains while neglecting others, creating an illusion of comprehensive capability.
Limitations of Existing Solutions: Current mitigation strategies, such as private benchmarks maintained by third parties (e.g., Chatbot Arena) or symbolic formatting benchmarks (e.g., GSM-Symbolic), suffer from a lack of transparency, reproducibility, or narrow domain coverage. Furthermore, existing methods fail to quantify the extent of overestimation, offering only a binary pass/fail or a single performance score without distinguishing between genuine ability and cheating.

2. Methodology: The ArxivRoll Framework

The authors propose ArxivRoll, a dynamic evaluation framework inspired by the One-Time Pad (OTP) encryption principle in cryptography. Just as OTP uses a unique, secret key for a single use to ensure perfect secrecy, ArxivRoll utilizes unique, private benchmarks that are "used once" and then expired to prevent contamination.

Core Components

The framework consists of two primary mechanisms:

A. SCP (Sequencing, Cloze, and Prediction)

To generate fresh, confidential test cases that LLMs have not seen during training, the authors utilize ArXiv preprints published within a specific recent window (e.g., the last six months). Since these papers are conceptually unseen by models trained on older data, they serve as a secure source of evaluation data. The SCP generator automates the creation of test cases through three objective tasks:

Sequencing: Sentences from a text fragment are shuffled; the model must select the correct order.
Cloze: Specific sentences within a fragment are masked; the model must select the correct sentences to fill the gaps from a set of candidates.
Prediction: Given a context and a correct subsequent sequence, the model must identify the correct next sequence among distractors.

This process ensures the benchmarks are confidential (unseen during training), difficult (flexible and complex, not fixed patterns), objective (multiple-choice selection), and comprehensive (covering eight domains: CS, Econ, EESS, Math, Phy, Bio, Fin, Stat).

B. Rugged Scores (RS)

To quantify overestimation, the framework introduces Rugged Scores, which measure the "ruggedness" or discrepancy between a model's performance on public benchmarks versus private benchmarks.

RSI (Absolute Rugged Score): Measures the performance gap between a model on a public benchmark ( $T_p$ ) and its corresponding private benchmark ( $T_c$ ) within the same domain. A high RSI indicates significant contamination or overfitting to the public benchmark.
$RSI = \frac{M(m, T_p) - M(m, T_c)}{M(m, T_p) + M(m, T_c)}$
(Where $M$ is the performance metric).
RSII (Biased Overtraining Score): Measures the variance of a model's performance across different private benchmarks. High variance suggests the model is over-trained on specific domains while underperforming in others.
$RSII = \sqrt{\sum (M(m, T_c) - \bar{M})^2}$

Workflow

Generation: SCP generates a new private benchmark (ArxivRollBench) from recent ArXiv papers.
Evaluation: LLMs are evaluated on both public benchmarks and the new private benchmark.
Scoring: Rugged Scores are calculated to quantify overestimation.
Release & Expiration: The private benchmark is released publicly to ensure reproducibility but is marked as "expired" to prevent its use in future evaluations, maintaining the "one-time" security property.

3. Key Contributions

Novel Benchmark Construction Strategy (SCP): An automated method to generate high-quality, challenging, and fresh test cases from ArXiv, ensuring confidentiality and reducing the risk of data contamination.
Quantification of Overestimation (Rugged Scores): The first study to propose metrics (RSI and RSII) that explicitly measure the proportion of overestimation and biased overtraining, moving beyond simple performance ranking.
Dynamic Evaluation Framework (ArxivRoll): A system that integrates private benchmarks and rugged scores to provide a systematic, transparent, and reproducible evaluation of LLMs, including a leaderboard that accounts for overestimation.

4. Experimental Results

The authors conducted extensive experiments on a wide range of open-source and closed-source LLMs (including Llama, Qwen, Phi, GPT, Claude, and Gemini series) using ArxivRollBench2024b.

Benchmark Quality: Meta-evaluations confirmed that the SCP generation strategy is stable (low standard deviation across random seeds) and exhibits high correlation (Spearman $\rho \approx 0.76$ ) with established private benchmarks like Chatbot Arena, validating its reliability.
Performance vs. Overestimation:
- Open-Source vs. Closed-Source: Some open-source models (e.g., Kimi-K2) achieved performance comparable to or exceeding proprietary models on private benchmarks.
- Overestimation Detection: The study revealed significant overestimation in several models. For instance, the Qwen and Phi series exhibited RSI values exceeding 100% in some cases, indicating their public benchmark scores were heavily inflated by contamination.
- Biased Overtraining: Models like Llama-3.1-Nemotron-70B showed high RSII scores, suggesting uneven performance across domains, which was masked by their high average public scores.
- Small Language Models (SLMs): Models like Phi-3-mini showed poor performance on private benchmarks despite claims of high capability, suggesting their public scores may be over-claimed.

5. Significance and Claims

The paper claims that ArxivRoll addresses the fundamental limitations of current LLM evaluation by:

Ensuring Freshness: By using a "one-time-pad" approach with dynamic, time-bound benchmarks, it effectively mitigates the risk of data contamination.
Providing Transparency: Unlike "black box" private leaderboards, ArxivRoll releases the benchmarks after evaluation, allowing for full reproducibility.
Quantifying Integrity: It shifts the focus from "how well a model performs" to "how much of that performance is genuine," offering a more honest assessment of model capabilities.

The authors conclude that while various robust evaluation methods exist, ArxivRoll offers a unique combination of dynamic generation, transparency, and quantitative overestimation metrics, providing a necessary tool for the fair and reliable assessment of LLMs in an era of widespread data contamination.