🤖 AI

Review Arcade: On the Human Alignment and Gameability of LLM Reviews

This paper empirically evaluates LLM-generated reviews on 2025 ACL Rolling Review papers, finding that while alignment with human reviews is limited and inconsistent, authors can effectively "game" the system by iteratively revising their work based on LLM feedback to achieve statistically significant score increases for up to 35% of submissions.

Original authors: Hans Ole Hatzel, Sebastian Steindl, Jan Strich

Published 2026-05-29

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Hans Ole Hatzel, Sebastian Steindl, Jan Strich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world of academic research as a massive, high-stakes talent show. Every year, thousands of scientists submit their "acts" (papers) to be judged by a panel of human experts (reviewers). The goal is to get a standing ovation (acceptance) so the act can be performed on the world stage.

Recently, a new kind of judge has entered the arena: AI (Large Language Models). These AI judges are being tested to see if they can help human judges keep up with the sheer volume of submissions. But this paper asks a tricky question: What happens when the performers start using AI to cheat the judges?

Here is the story of what the researchers found, broken down into simple concepts.

1. The Setup: The AI Judge vs. The Human Judge

The researchers took 984 real scientific papers that were submitted to a major conference (ACL 2025). They asked different AI models to read these papers and write reviews, just like a human would.

The Goal: They wanted to see if the AI's review (the score and the comments) matched what the human judges thought.
The Analogy: Imagine a human judge gives a dance routine a "7 out of 10" because the dancer stumbled. The researchers asked the AI, "What score would you give this?"
The Result: The AI was okay at guessing the score, but not great. Sometimes it was close, but often it was off.
- The "Human" Problem: Even the human judges didn't agree with each other! If you asked two different humans to judge the same paper, their scores often differed. The AI was actually about as consistent as a human, but not better.
- The "Prompt" Problem: The AI was very sensitive to how you asked the question. If you asked it like a strict professor, it gave low scores. If you asked it like a friendly teacher, it gave high scores. It was like a chameleon that changed its opinion based on the color of the room it was in.

2. The Twist: The "Paper Laundering" Game

This is the most exciting (and scary) part of the study. The researchers asked: "If a writer knows the AI is judging them, can they trick the AI into giving them a better score?"

They set up a game called "Iterative Submission Improvement."

The Loop:
1. The AI reads the paper and says, "This is bad because your grammar is weak."
2. The writer (using another AI) fixes the grammar.
3. The AI reads it again and says, "Better, but your conclusion is vague."
4. The writer fixes the conclusion.
5. They repeat this 10 times.
The Three Levels of Cheating:
1. The "Polite" Player (Constrained): The writer is only allowed to fix typos, clarify sentences, and make the paper look nicer. They cannot change the actual science or add new data.
2. The "Average" Player (Default): The writer can make bigger changes but still can't lie.
3. The "Villain" (Adversarial): The writer is told to do whatever it takes to get an "Accept." They are allowed to lie, make up fake data, and invent experiments that never happened.

3. The Results: Can You Beat the AI?

The researchers found some surprising things:

Yes, you can trick the AI.
In the "Polite" scenario, about 35% of the papers got significantly higher scores just by being rewritten to sound better. The AI was fooled into thinking the paper was better, even though the core science hadn't changed much. It was like a student rewriting their essay to use fancier words, and the teacher giving them an A+ without noticing the ideas were still the same.
But the "Villain" strategy didn't work as well as expected.
The researchers thought that if writers were allowed to lie and make up fake results, the scores would skyrocket. They were wrong.
- Why? The AI judges have "guardrails" (safety filters) that stop them from believing obvious lies. Also, when you make up fake data, the story of the paper starts to fall apart and become inconsistent. The AI noticed the story didn't make sense and didn't give a huge score boost.
- The Lesson: You can't easily "game" the system by just lying; the AI is smart enough to spot when a story doesn't add up.

4. The Big Warning: Goodhart's Law

The paper ends with a warning based on an old economic rule called Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."

The Analogy: Imagine a school decides to measure student success by how many hours they spend in the library. Suddenly, every student sits in the library for 10 hours a day, but they aren't actually studying; they are just sleeping there to "game" the system. The library hours are no longer a good measure of intelligence.
The Paper's Warning: If scientists start writing papers specifically to please AI reviewers (by using fancy words, clarifying vague points, or tweaking the structure), the papers might get high scores from the AI. But those high scores might no longer mean the science is actually good. The AI might be rating the "packaging" rather than the "product."

Summary

AI Reviewers are inconsistent: They don't always agree with humans, and they change their minds based on how you ask them.
AI Reviews can be "gamed": Writers can use AI to rewrite their papers to sound better, tricking the AI into giving higher scores without actually improving the science.
Lying doesn't always work: Trying to fake data to fool the AI is risky and often fails because the AI can spot inconsistencies.
The Danger: If we rely too much on AI to judge papers, we might end up with a system where papers are "optimized" to look good to a robot, but lose their true value to human readers.

The paper concludes that while AI can help with the heavy lifting of reviewing, we must be very careful not to let it become the only judge, or we might lose the ability to tell what is truly a good scientific discovery.

Technical Summary: Review Arcade

Problem Statement

The paper addresses the growing reliance on Large Language Models (LLMs) in the academic peer-review ecosystem. It posits a dual-adoption scenario where both authors utilize LLMs to revise manuscripts and reviewers employ LLMs to generate or assist in writing reviews. This dynamic raises concerns regarding Goodhart's law: if authors optimize papers specifically to satisfy LLM review criteria, the resulting scores may cease to be a valid measure of paper quality. The authors investigate three core research questions (RQs):

Validity: Can LLMs produce reviews sufficiently aligned with human reviews?
Stability: Are LLM reviews consistent across different models, prompts, and repeated runs?
Gameability: Can LLM reviews be "gamed" via automated, iterative editing workflows that aim to maximize review scores?

Methodology

The study conducts empirical experiments on 984 real submissions from the 2025 ACL Rolling Review (ARR). The dataset includes both accepted and rejected papers, with a specific focus on the "Overall" score as the primary metric.

Experimental Setup

Models: The authors evaluated five models (three open-weight: Gemma-3-27B, Qwen-3.6-35B, Llama-3.3-70B; and two closed-weight: GPT-5.4-mini, GPT-5.4).
Prompts: Five distinct prompting strategies were tested, ranging from a minimal "simple" prompt to complex, persona-driven prompts ("acl_senior") tailored to ARR guidelines.
Metrics:
- Alignment: Measured via Mean Absolute Error (MAE) against human scores and Pearson correlation ( $r$ ) with the best-matching human review.
- Content: Semantic overlap of identified strengths and weaknesses using an LLM-judge (recall-based metric).
- Stability: Variance in scores across repeated runs and different prompts.

Iterative Submission Improvement (ISI)

To address RQ3, the authors simulated an adversarial workflow where an author iteratively edits a submission based on LLM feedback. This process was run for 10 iterations under three distinct settings:

Constrained: Only superficial, cosmetic edits (e.g., grammar, clarification) are allowed; no new experiments or content changes.
Default: A middle-ground approach inspired by prior work, allowing edits but prohibiting non-cosmetic changes like fabricating results.
Adversarial: The LLM is explicitly permitted to "game" the system, including fabricating evidence, modifying results, and inventing citations to satisfy reviewer concerns.

A taxonomy of edits was established, categorizing actions from "Copy-Editing" and "Clarification" to "Methodological-Augmentation" and "Hallucinated-Evidence."

Key Results

1. LLM Review Validity (RQ1)

Score Alignment: LLMs generally fail to match human judgments in terms of MAE. The best-performing models (GPT-5.4 variants) achieved an MAE of ~0.7, whereas human reviewers achieved ~0.17. Notably, a naive baseline (predicting the midpoint score of 2.5) outperformed the best LLMs in MAE (0.64 vs. 0.70).
Correlation: While MAE is poor, Pearson correlation is more competitive. GPT-5.4 reached a correlation of 0.276 against human reviews. However, this is comparable to the low human-human correlation observed in the dataset (0.312), suggesting that LLMs are not significantly better or worse than the natural variance between human reviewers.
Split Disparity: Performance varied significantly between accepted and rejected papers. Human agreement was much higher for rejected papers (0.408) than accepted ones (0.210), and LLMs mirrored this trend, struggling more to differentiate high-quality papers.

2. LLM Review Stability (RQ2)

Prompt Sensitivity: LLM reviews are highly unstable across different prompts. For instance, a model's MAE could vary drastically depending on whether a "simple" or "senior" prompt was used.
Run Instability: Even with the same prompt and model, repeated runs produced inconsistent scores. Approximately 36.9% of papers received different scores across three runs, with 20% showing a delta greater than 0.5 points. This indicates that LLM reviews are currently too unstable to be reliable as a standalone metric.

3. LLM Review Gameability (RQ3)

Constrained Setting: Iterative editing with only superficial changes led to a statistically significant increase in scores. Approximately 35% of papers improved after 10 iterations, while 22% saw a decrease. The effect size was small-to-medium.
Adversarial Setting: Surprisingly, allowing the LLM to fabricate evidence or results did not yield the largest score increases. The effect size was weaker than in the constrained setting. The authors hypothesize that model guardrails prevent significant hallucination and that methodological inconsistencies introduced by fake data may penalize the paper in subsequent LLM reviews.
Edit Types: The most successful "gaming" strategy relied on Clarification and Refactoring (restructuring without changing meaning), rather than substantive content fabrication.

Key Contributions

Large-Scale Empirical Evaluation: The first extensive study evaluating LLM reviews on real ARR submissions, covering multiple models, prompts, and runs.
Adversarial Attack Simulation: An investigation into automated paper editing as a method to manipulate LLM review scores, demonstrating that "gaming" is feasible even with superficial edits.
Edit Taxonomy: A grounded taxonomy for categorizing paper edits, distinguishing between cosmetic changes and substantive (or fabricated) modifications.

Significance and Claims

The paper concludes that while LLMs can produce reviews that correlate with human judgments in specific best-case scenarios, they are not universally reliable due to instability across prompts and runs. The authors argue that the current state of LLM reviews is insufficient for official adoption without rigorous safeguards.

Crucially, the study highlights a vulnerability in the peer-review process: LLM reviews can be gamed. Authors can iteratively refine papers using LLM feedback to artificially inflate scores, particularly through superficial clarifications and structural reorganization. The authors invoke Goodhart's law to warn that as LLMs become a target for optimization, their utility as a measure of true paper quality will degrade. They urge the community to exercise extreme caution regarding automated reviews, emphasizing that official implementations must be designed to prevent gameability and ensure diverse perspectives, rather than relying solely on score-based metrics.