Defining Manuscript Peer Review: An International Delphi Consensus Study
This international three-round Delphi study established a consensus-based definition of manuscript peer review as a structured, human-led process aimed at improving research quality and trustworthiness, supported by eight core pillars: Principled, Constructive, Timely, Transparent, Professional, Quality, Confidential, and Pluralistic.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
Science relies on a quiet, invisible filter that stands between a researcher's discovery and the world that needs to know about it. When a scientist writes up a new study, they do not simply publish it; they send it to a group of other experts who have not seen the work before. These experts read the study, check the math, question the methods, and decide if the work is good enough to be trusted. This process is called peer review. For centuries, it has been the standard way to ensure that what appears in journals is accurate and reliable. Yet, despite its long history and central role, no one has ever agreed on exactly what this process is. Different fields do it differently, different journals have different rules, and even the people who do the reviewing often cannot agree on what the term actually means. Without a clear definition, it is difficult to study whether the system is working, to fix the parts that are broken, or to distinguish legitimate science from fake or misleading work.
A team of international researchers set out to solve this confusion by bringing together a diverse group of experts to agree on a single, shared definition. They did not just ask people what they thought; they used a structured, multi-step method to guide the group toward a consensus. First, they gathered a large number of experts from around the world, including scientists, doctors, editors, and patient partners, to vote on a long list of ideas about what peer review should look like. In the first round, 143 people rated dozens of features, such as whether the process should be anonymous, how fast it should happen, and who should be allowed to do it. The researchers then took the results, refined the list, and asked the same group to vote again. In the second round, 109 of the original participants returned to narrow down the list further. Finally, to resolve the remaining disagreements, the researchers invited a smaller, carefully selected group of 16 experts to a virtual meeting. This final group discussed the tricky points and made the final decisions on what belongs in the definition and what does not.
The result of this three-step journey is a clear, agreed-upon definition that describes manuscript peer review as a structured process where human experts assess written research papers. The goal of this process, the group decided, is to improve the quality and trustworthiness of published research while following the unwritten rules of the scientific community, where everyone helps everyone else. The researchers also identified eight core qualities that define a good peer review system: it should be principled, constructive, timely, transparent, professional, focused on quality, confidential, and pluralistic. "Pluralistic" means that the system should welcome many different types of people and perspectives, not just a single kind of expert.
One of the most important decisions the group made was to draw a clear line between the act of reviewing and the act of publishing. In the past, some people argued that the editor's final decision to accept or reject a paper was part of the peer review process. The group in this study concluded that these are two separate stages. Peer review is the specific moment when experts evaluate the science itself. The editor's decision comes after, based on that evaluation and other factors. This distinction matters because it helps researchers and journals understand exactly where the responsibility lies. It allows them to study the quality of the review separately from the quality of the final publication decision.
The study also clarified who should be called a "peer." The group agreed that the term is appropriate, but it does not mean that the reviewers must be identical to the author. Instead, a peer is someone with the necessary expertise to judge the work, regardless of their background, location, or career stage. The definition also emphasizes that the process must be human-led. While computers and artificial intelligence can help with tasks like checking for plagiarism or formatting, the final assessment of the science must come from a human mind. This ensures that the judgment involves the nuance, creativity, and ethical reasoning that machines cannot yet provide.
By establishing this shared language, the researchers hope to make it easier to study the system and improve it. If everyone uses the same definition, they can compare results from different studies, measure how well the system is working, and design better training for reviewers. The definition is not a rigid set of rules that forces every journal to operate exactly the same way. Instead, it provides a common foundation that allows for flexibility. Journals can still choose different models, such as open review where names are known, or blind review where they are hidden, as long as the core purpose remains the same: a structured, human evaluation to ensure the research is trustworthy. This agreement gives the scientific community a solid starting point to build a more reliable future for how knowledge is shared.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.