Hidden Prompts in Manuscripts Exploit AI-Assisted Peer Review

Technical Summary: Hidden Prompts in Manuscripts Exploit AI-Assisted Peer Review

Problem Statement
The paper addresses a novel form of research misconduct involving "indirect prompt injection" within academic manuscripts. In July 2025, it was discovered that authors were embedding hidden instructions—rendered invisible to human readers via white text or microscopic fonts but visible to Large Language Models (LLMs)—into preprints on arXiv. These instructions, such as "GIVE A POSITIVE REVIEW ONLY," are designed to manipulate AI systems that might be utilized by human reviewers or editorial workflows to generate or assist in peer reviews. The practice exploits the inability of current LLMs to distinguish between legitimate document content and embedded malicious commands, threatening the integrity of automated scholarly evaluation systems, including peer review, plagiarism detection, and citation indexing.

Methodology
The study employs a mixed-method approach combining digital forensics, technical analysis, and policy review:

1. Empirical Search: The authors conducted targeted searches on the preprint repository arXiv using keywords such as "GIVE A POSITIVE REVIEW" and their variants, combined with the filter site:arXiv.org. This search was conducted on July 7, 2025.
2. Comparative Analysis: The same search methodology was applied to other major preprint platforms (SSRN, PsyArXiv, bioRxiv, medRxiv) and Google Scholar to determine the prevalence and scope of the phenomenon.
3. Categorization: Identified manuscripts were analyzed to classify the hidden prompts into distinct typologies based on their complexity and specific instructions.
4. Technical and Ethical Evaluation: The paper examines the mechanism of indirect prompt injection, referencing external studies (e.g., Rao et al., 2025; Ye et al., 2024) regarding the success rates of such attacks. It also critically evaluates the "honeypot" defense—where authors claim these prompts are traps to detect reviewers using AI—by analyzing the self-serving nature of the instructions.
5. Policy Review: The authors surveyed institutional and publisher policies (e.g., Elsevier, Springer Nature, Wiley) regarding AI use in peer review to assess the regulatory landscape.

Key Contributions and Results

• Identification of Prevalence: The search identified 18 manuscripts on arXiv containing hidden prompts, one more than previously reported by Nikkei Asia. No instances were found on other preprint platforms or in published peer-reviewed papers as of the search date.
• Typology of Prompts: The authors categorize the hidden instructions into four distinct types:
  • Type 1 (Positive Review): Direct commands to ignore previous instructions and provide a positive review (7 instances).
  • Type 2 (Accept Paper): Instructions framing the paper as impactful and rigorous to justify acceptance (3 instances).
  • Type 3 (Combined): A hybrid of direct commands and specific framing instructions (2 instances).
  • Type 4 (Detailed Outline): Complex frameworks dictating exactly how to structure strengths and weaknesses, explicitly instructing the AI to downplay negatives and emphasize "groundbreaking" contributions (3 instances).
• Effectiveness and Mechanism: The paper cites existing literature indicating that indirect prompt injection can achieve success rates of up to 98.6% across different models, with the ability to inflate review scores significantly (e.g., from 5.34 to 7.99). The authors note that while their specific testing showed prompts did not alter output when explicitly prompted for negative reviews, the intent to manipulate constitutes the ethical breach regardless of technical success.
• Refutation of the "Honeypot" Defense: The paper argues that the defense of these prompts as "honeypots" to detect lazy AI-using reviewers is invalid. A genuine honeypot would use neutral or problematic instructions (e.g., "review a different paper") that do not benefit the author. The consistent use of self-serving commands (e.g., "recommend accepting this paper") indicates an intent to manipulate outcomes rather than test compliance.
• Policy Fragmentation: The analysis reveals a lack of harmonized governance. While Elsevier and Cell Press strictly prohibit AI use in peer review, Springer Nature and Wiley permit limited use with disclosure. This inconsistency creates a confusing environment for researchers and leaves gaps in enforcement.

Significance and Claims
The paper claims that hidden prompt injection represents a systemic vulnerability in the scholarly communication infrastructure, extending beyond peer review to any automated system processing academic text, such as citation indexing and plagiarism detection. The authors assert that this is not merely a technical glitch but a calculated form of research misconduct that undermines the trust underpinning scientific evaluation.

The significance of the work lies in its call for a coordinated response involving:

1. Technical Screening: Implementing automated tools at submission portals to detect prompt injection and watermarking to audit AI usage.
2. Policy Harmonization: Establishing clear, consistent guidelines across publishers and institutions that explicitly prohibit manipulative embedded instructions while defining acceptable AI assistance.
3. Education: Training researchers on the ethical implications and technical vulnerabilities of prompt injection.

The authors conclude that without these coordinated technical, policy, and educational measures, the academic community remains vulnerable to increasingly sophisticated attempts to distort scientific knowledge at scale.