PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality

Technical Summary of PeerCheck: Enhancing LLM-Generated Academic Reviews Towards Human-Level Quality

Problem Statement
The academic peer review process is facing a crisis of scalability due to exponential growth in submissions and a shortage of available reviewers, leading to delays and inconsistent quality. While Large Language Models (LLMs) have emerged as a potential solution to assist in manuscript evaluation, their adoption raises significant concerns regarding the reliability, quality, and alignment of their generated reviews with human scholarly standards. Current research lacks a comprehensive framework to quantitatively compare LLM-generated reviews with human-written ones or to systematically address the specific gaps between them.

Methodology
The authors propose PeerCheck, a framework designed to analyze differences between human and LLM reviews and to enhance LLM output quality. The methodology proceeds in three stages:

1. Data Collection and Baseline Generation: The study utilizes a corpus of 1,920 papers from top-tier machine learning conferences (ICLR 2024/2025, NeurIPS 2024) and their corresponding official human reviews from OpenReview. Three state-of-the-art LLMs (GPT-4o, Claude-3.7-Sonnet, and DeepSeek-V3) are employed to generate reviews using structured prompts. The authors also simulate diverse reviewer personas (e.g., PhD students, professors, industry experts) to test role sensitivity.
2. Comparative Analysis: The framework analyzes discrepancies across three dimensions:
   • Keywords: Identifying that LLMs prioritize "theoretical" aspects while humans focus on "methods," "experiments," and "performance."
   • Scoring: Observing that LLMs exhibit broader score distributions and tend to assign higher scores to rejected papers compared to humans.
   • Topics: Noting that humans emphasize Natural Language Processing (NLP) more than LLMs, which favor representation learning and in-context learning.
3. Enhancement Strategies: To bridge these gaps, the authors implement:
   • Prompt Engineering: Utilizing Chain-of-Thought (CoT) reasoning and revising prompts to include specific keywords, stylistic elements (e.g., hedging, discourse markers), and structural constraints (e.g., citing specific equations or figures).
   • Retrieval-Augmented Generation (RAG): Integrating external knowledge by retrieving human-written reviews to guide the LLM generation process.
   • Evaluation: The enhanced reviews are evaluated using detection metrics (RoBERTa, Log-Likelihood, etc.) to measure "human-likeness" (lower detectability) and similarity metrics (Cosine, ROUGE, BLEU) to measure alignment with human reviews.

Key Contributions

• PeerCheck Framework: A novel system that quantitatively compares and enhances LLM-generated academic reviews, increasing the human-like score of GPT-4o from 0.379 to 0.645.
• Discovery of the "RAG Paradox": The authors find that Retrieval-Augmented Generation does not universally improve performance. While RAG significantly boosts GPT-4o's similarity to human reviews, it degrades the performance of Claude-3.7-Sonnet and DeepSeek-V3, challenging the assumption that "more information equals better."
• Role Sensitivity Analysis: The study reveals that LLMs exhibit significant bias based on assigned personas. For instance, Claude-3.7-Sonnet assigns an 8/10 score 57.7% more frequently when acting as a "PhD student" compared to a neutral role.
• Prompt Optimization Insights: Through ablation studies, the authors demonstrate that structural scaffolding (guiding the reasoning process) contributes more to review quality (specificity and actionability) than stylistic mimicry (mimicking human phrasing). Furthermore, revised prompts using CoT reduce the detectability of LLM-generated text by 40.3% using RoBERTa-based detectors.

Results

• Detection Evasion: Revised prompts, particularly those combining CoT with keywords and sample sentences, significantly reduce the ability of detection tools to identify AI-generated text. For example, the RoBERTa detection rate for GPT-4o drops from 0.798 to 0.512 with revised prompts.
• Similarity to Human Reviews: The combination of revised prompts and RAG yields the highest similarity scores. GPT-4o achieves a cosine similarity of 0.645 with human reviews under these conditions, a substantial improvement over the baseline of 0.379.
• Human Evaluation: In blind evaluations by human experts, revised LLM reviews scored an average of 6.682, significantly higher than normal LLM reviews (5.371) and approaching human-written reviews (7.124). However, RAG-enhanced reviews sometimes performed worse than non-RAG revised reviews, confirming the model-dependent nature of the RAG paradox.
• Factual Faithfulness: Revised prompts consistently reduced the rate of unsupported and non-verifiable claims in reviews, indicating that structural improvements enhance factual correctness, not just stylistic similarity.

Significance and Claims
The paper claims that improving LLM-based academic review requires model-specific strategies and well-planned prompts rather than simply increasing input data volume or relying on generic retrieval augmentation. The authors emphasize that "human-level quality" in this context is operationalized as lexical and semantic similarity to historical human reviews, which serves as a proxy for constructiveness and insightfulness.

The work highlights that while LLMs can be trained to mimic human review structures and evade detection, fundamental differences in reasoning (such as the RAG paradox) persist. The authors position PeerCheck not as an automated replacement for human reviewers, but as an assistive analytical tool to help align AI-generated feedback with scholarly standards, thereby addressing the scalability crisis while maintaining the integrity of the peer review ecosystem. The study concludes that effective integration of LLMs into peer review demands a nuanced understanding of how different models interact with retrieval mechanisms and prompt structures.