PaperMentor: A Human-Centered Multi-Agent Writing Tutor for AI Research Papers on Overleaf
PaperMentor is an open-source, human-centered multi-agent writing assistant for Overleaf that leverages a curated expert skill library to provide actionable, inline feedback on AI research papers, significantly outperforming baseline models in a user study while preserving human authorship.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a student trying to write a very important research paper. You have great ideas, but you aren't sure if your story flows well, if your formatting is correct, or if you're explaining your methods clearly enough for top experts to understand. Usually, you would need a wise, experienced professor to look over your draft and give you specific advice. But these professors are busy, and getting that kind of detailed, personalized feedback is hard to find.
Enter PaperMentor. Think of it as a team of 12 specialized digital teaching assistants that live right inside your writing software (Overleaf). Instead of rewriting your paper for you, they act like a helpful editor sitting next to you, pointing at specific sentences and saying, "Hey, this part is a bit vague," or "You forgot to mention how you picked these numbers."
Here is how it works, broken down into simple steps:
1. The "Expert Handbook" (The Skill Library)
Before PaperMentor can help, it had to learn how to be a good teacher. The creators didn't just tell the AI to "be smart." Instead, they built a massive digital handbook (called a Skill Library) containing over 40 files of advice.
- Where did the advice come from? It was gathered from real, experienced researchers, famous writing guides, and thousands of actual reviews from top conferences.
- What's in it? It covers everything from "How to write an Introduction" to "How to format a table" and "What specific journals expect."
- The Analogy: Imagine a student who memorized every rulebook, style guide, and "how-to" manual written by the world's best scientists. That is PaperMentor's brain.
2. The "Team of Specialists" (Multi-Agent System)
PaperMentor doesn't use one giant brain to do everything. Instead, it splits the work among 12 different agents (specialized AI workers), each with a specific job:
- The Section Specialists: One agent looks only at the Abstract, another only at the Methods, another only at the Results.
- The Global Specialists: Some agents look at the whole paper to check if the writing style is consistent or if the math looks right.
- The Context Experts: Two agents look at your specific goals (e.g., "I want to submit to Conference X") and find a "role model" paper to compare your work against.
The Analogy: Imagine a sports team. You don't ask the goalkeeper to fix the striker's footwork. You have a specific coach for the defense, one for the offense, and a head coach for the overall strategy. PaperMentor works the same way.
3. How It Works in Real Life
When you upload your paper to PaperMentor:
- Phase 1 (The Setup): The system reads your paper, figures out what kind of paper it is (e.g., a new method, a dataset, or a review), and organizes the sections.
- Phase 2 (The Review): The 12 agents read their specific parts of your paper using the "Expert Handbook." They generate comments like: "This sentence is too vague. Try saying exactly what your method improves."
- Phase 3 (The Cleanup): Sometimes two agents might say the same thing. The system cleans up the duplicates, keeps the most important warnings, and places the comments directly into your document as highlighted notes, just like a human professor would.
4. Did It Work? (The Results)
The researchers tested PaperMentor with 14 AI researchers who acted as judges. They compared PaperMentor against a standard AI (without the expert handbook).
- The Verdict: PaperMentor was significantly better.
- 90.6% of its comments were rated as "actionable" (meaning the student knew exactly what to fix).
- 67.5% were rated as "valid" (meaning the advice was actually correct and relevant).
- The Comparison: The standard AI (without the handbook) was less accurate and gave less useful advice. The expert handbook made the difference.
5. The Golden Rule: You Are Still the Author
The most important part of PaperMentor is what it doesn't do.
- It does not rewrite your sentences for you.
- It does not change your ideas.
- It leaves the actual writing entirely in your hands.
The Analogy: Think of PaperMentor as a GPS for writing. It tells you, "You're on the right road, but there's a pothole ahead; take a left to avoid it." It doesn't drive the car for you. You are still the driver, making the final decisions on how to fix the paper.
Summary
PaperMentor is a tool that brings the wisdom of senior professors to junior researchers who might not have access to them. By using a team of specialized AI agents guided by a curated library of expert advice, it gives specific, helpful feedback directly inside your writing software, helping you improve your paper without taking away your own voice or control.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.