🤖 AI

Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem

This position paper argues that the machine learning community must urgently develop an AI-augmented peer-review ecosystem, leveraging Large Language Models as collaborative tools to address the crisis of scale while emphasizing the need for structured data access and a rigorous research agenda to maintain scientific integrity.

Original authors: Qiyao Wei, Samuel Holt, Jing Yang, Markus Wulfmeier, Mihaela van der Schaar

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qiyao Wei, Samuel Holt, Jing Yang, Markus Wulfmeier, Mihaela van der Schaar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: A Library Too Big for Its Librarians

Imagine the world of Machine Learning (ML) is a massive library where scientists submit new books (research papers) every day. In 2014, the library received about 1,600 books. By 2024, that number exploded to nearly 17,500.

The problem? The "librarians" (the expert reviewers who check these books) are human. There are only so many of them, and they have limited time. They are drowning in work. Because of this, the quality of checking is suffering:

Fatigue: Librarians are tired and rushing, so they might miss mistakes.
Inconsistency: One librarian might give a book a "5-star" rating, while another gives the same book a "2-star" rating just because they are different people.
Shallow Checks: Because they are so busy, reviews sometimes become generic and miss the deep, important details.

The Proposed Solution: AI as a "Super-Powered Assistant"

The authors argue that we cannot just hire more human librarians fast enough. Instead, we need to build a team-up between humans and Artificial Intelligence (AI).

Think of the AI not as a robot that takes the librarian's job away, but as a high-tech co-pilot or a super-intelligent intern that helps the human do their job better.

Here is how this "AI Co-pilot" would help three different groups:

1. Helping the Authors (The Book Writers)

The Analogy: Imagine a writing coach who reads your draft before you submit it.
What the AI does: It checks if your story makes sense, if you forgot to mention important previous stories (citations), and if your writing is clear. It acts like a "simulated reviewer" to tell you, "Hey, this paragraph is confusing," or "You missed a key reference," so you can fix it before the real librarians see it.

2. Helping the Reviewers (The Librarians)

The Analogy: Imagine a librarian has a magical magnifying glass that instantly finds typos, checks facts against a giant encyclopedia, and highlights contradictions.
What the AI does:
- Fact-Checking: It quickly scans the book to see if the author's claims match known facts.
- Code Checking: If the book includes computer code, the AI can run a "sanity check" to see if the code actually works as described.
- Quality Control: After a librarian writes a review, the AI can give them a "Report Card." It might say, "You gave a low score, but you didn't explain why clearly," or "You missed a major flaw in the experiment." This helps reviewers write better, more consistent feedback.

3. Helping the Area Chairs (The Head Librarians)

The Analogy: Imagine a Head Librarian who has to read 50 different reviews for one book to decide if it should be published. It's overwhelming.
What the AI does: It acts as a summary assistant. It reads all 50 reviews and says, "Here is the main argument for accepting this book, here is the main argument for rejecting it, and here is where the reviewers are disagreeing." It helps the Head Librarian make a fair decision faster, but the Head Librarian still makes the final call.

The Missing Ingredient: Better Data

The paper makes a crucial point: You cannot build a good AI assistant without good training data.

Currently, the data we have is like a transcript of a conversation where we only see the final scores, but not the reasoning.

The Problem: We know a reviewer changed a score from a 5 to a 7, but we don't know which sentence in the author's reply made them change their mind.
The Solution: The authors want the community to start collecting "richer" data. They want to record the thought process: "I changed my score because the author clarified point X." Without this detailed map of how humans think and argue, the AI can only guess the outcome, not learn the logic.

What the Experiments Showed

The authors tried out some early versions of these AI tools on real data from a major conference (ICLR).

Good News: The AI was pretty good at finding the "Strengths" of a paper and summarizing what the author said in their reply.
Bad News: The AI struggled to find the "Weaknesses" or predict exactly what score a human would give. It often missed deep, subtle flaws that a human expert would catch.
The Takeaway: The AI is a helpful tool, but it is not ready to replace humans. It needs more training and better data to get smarter.

The Warning Signs (Risks)

The authors are careful to warn about potential dangers:

Lazy Librarians: If reviewers rely too much on the AI, they might stop thinking critically themselves (a process called "de-skilling").
Fake Books: Authors might try to use AI to write the whole book and trick the system.
Privacy: We need to make sure the data used to train these AIs is anonymous and safe.

The Bottom Line

The paper is a call to action. The Machine Learning community is growing too fast for humans to handle alone. We need to build an AI-augmented ecosystem where AI handles the heavy lifting, fact-checking, and summarizing, freeing up humans to do the deep, critical thinking.

But to make this work, we need to stop treating peer review as just a "black box" of scores and start recording the reasoning behind those scores. Only then can we build AI that truly understands how to help us validate scientific truth.

Technical Summary: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem

1. Problem Statement

The machine learning (ML) community faces a "scalability crisis" in peer review. Submission volumes to premier venues (e.g., NeurIPS, ICML, ICLR) have grown exponentially (e.g., NeurIPS submissions increased 10.4× from 2014 to 2024), outpacing the finite pool of qualified reviewers. This deluge, exacerbated by LLM-assisted writing tools, has led to:

Reviewer Fatigue and Superficiality: Reviews often lack depth or methodological rigor due to time constraints.
Inconsistency: High inter-reviewer variance exists, with up to 23% of acceptance decisions potentially flipping based on reviewer assignment.
Ineffective Rebuttals: Author rebuttals frequently fail to influence final decisions due to reviewer disengagement or time limits.
Systemic Risk: The current manual model threatens the integrity, timeliness, and fairness of scientific validation.

The paper argues that the community cannot sustain high-quality peer review without integrating AI not as a replacement for human judgment, but as a sophisticated collaborator within a comprehensive ecosystem.

2. Methodology and Proposed Framework

The paper proposes a holistic, AI-augmented peer-review ecosystem where Large Language Models (LLMs) act as "co-pilots" for authors, reviewers, and Area Chairs (ACs). The methodology relies on two pillars: the development of specific AI tools and the creation of a richer data infrastructure.

2.1. Foundational AI Tools

The ecosystem relies on several key technical capabilities:

Retrieval Augmented Verification (RAV) & Grounding: Connecting LLMs to authoritative scientific repositories (e.g., Semantic Scholar, arXiv) to cross-reference claims, suggest citations, and ensure factual soundness.
Code Analysis & Reproducibility: Tools to parse methodologies and source code to identify common errors, missing dependencies, or inconsistencies between code and text.
Review Quality Feedback ("Report Cards"): LLMs analyzing human-written reviews against criteria (coverage, specificity, evidence, tone) to provide structured feedback to reviewers.
Content Provenance & Authenticity: Methods (e.g., statistical analysis of perplexity, watermarking) to detect AI-generated text, though the paper acknowledges these are currently imperfect and require human oversight.
Decision Support for ACs: Systems to synthesize multiple reviews, highlight consensus/disagreement, and flag unanswered concerns to assist in meta-review preparation.

2.2. The Critical Data Gap

The paper posits that current public datasets are insufficient for training advanced AI assistants because they lack structured deliberation traces. Current data captures outcomes (scores) but not the reasoning behind them. The authors argue that developing robust AI requires:

Structured Reviewer Reasoning: Explicit data on why scores changed post-rebuttal.
Interaction Traces: Semantic annotations of dialogue acts (clarification, concession) and links between rebuttal points and specific manuscript sections.
AC Deliberation Traces: Anonymized data on how ACs weigh conflicting evidence.
Proposed Solution: The paper calls for "active elicitation interfaces" (e.g., low-friction prompts asking reviewers to justify score changes) and a tiered access model for privacy-preserving data collection.

3. Illustrative Experiments and Results

The authors conducted preliminary experiments on ICLR 2019–2025 corpora to demonstrate LLM potential and current limitations.

3.1. Review Component Generation

Task: Using few-shot prompting, LLMs generated strengths, weaknesses, and key rebuttal points from papers and reviews.
Results:
- Strengths: High recall (0.927 ± 0.060 for ICLR 2025).
- Weaknesses: Lower recall (0.632 ± 0.000 for ICLR 2025), indicating a "critical thinking gap" where models struggle to identify non-obvious flaws.
- Rebuttals: High recall (0.911 ± 0.040).
Interpretation: LLMs can summarize content well but struggle with deep critical evaluation, reinforcing the need for human oversight.

3.2. Rating Prediction

Task: Using In-Context Learning (ICL) with varying numbers of examples ( $n=0$ to $3$), LLMs predicted initial ratings, final ratings, and score changes.
Results:
- Initial Ratings: High Mean Absolute Error (MAE $\approx$ 2.28 with $n=2$ ).
- Final Ratings: Lower MAE ( $\approx$ 0.67 with $n=1$ ), suggesting models perform better when provided with review context.
- Score Change Prediction: Remained difficult (MAE $\approx$ 0.73–0.93).
Interpretation: The high error rates are attributed not to fundamental LLM limitations, but to the lack of structured reasoning data in current public datasets. Without explicit traces of why a score changed, models are forced to "guess" outcomes rather than learn the underlying judgment logic.

4. Key Contributions

Positional Argument: The paper asserts that AI-assisted peer review is an urgent infrastructure priority to address the scalability crisis, moving beyond isolated tools to a cohesive ecosystem.
Ecosystem Architecture: It defines specific roles for AI in four areas:
- Reviewers: Enhancing factual rigor, providing "Report Cards" for self-improvement, and detecting AI-generated content.
- Authors: Offering pre-submission feedback, simulating diverse reviewer personas, and aiding rebuttal construction.
- Area Chairs: Providing decision support, conflict highlighting, and meta-review drafting assistance.
- Process: Instrumenting the workflow to capture causal traces of scientific judgment.
Data Agenda: It identifies the "key near-term bottleneck" as the lack of granular, ethically-sourced process data (rationales, deliberation traces) rather than model architecture alone.
Experimental Baseline: It provides initial empirical evidence of LLM capabilities and limitations in peer review tasks, highlighting the gap between current public data and the needs of advanced AI.

5. Significance and Claims

The paper claims that the ML community is uniquely positioned to solve this problem because ML research is an "extreme case" of submission volume and is already highly instrumented (digital platforms, code artifacts).

Human-in-the-Loop: The core claim is that AI should augment, not replace, human judgment. Humans must remain the "senior partners" and final arbiters of scientific validity and authorship.
Sociotechnical Challenge: The paper frames peer review scaling as a sociotechnical problem requiring principled co-design of AI tools and benchmark infrastructure.
Integrity and Scalability: The ultimate goal is to build a system that maintains high standards of peer review while scaling to handle exponential growth, ensuring the continued integrity of scientific validation.
Modest Scope: The authors explicitly state they are not proposing a deployed system or claiming current models are ready for gatekeeping. Instead, they call for a research agenda to develop the necessary tools, data, and ethical frameworks to make such an ecosystem viable.