Development and Validation of OlivePICO: A Browser-Based Pre-Screening Tool for Reducing Title and Abstract Screening Burden in Systematic Reviews
OlivePICO is a free, browser-based pre-screening tool that utilizes structured PICO domain matching to significantly reduce the title and abstract screening burden in systematic reviews while maintaining high sensitivity and improving protocol relevance compared to non-structured keyword filtering.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of modern medicine, the most reliable way to understand a treatment or a disease is to gather every single study ever written on the subject and examine them all together. This process, known as a systematic review, is the gold standard for evidence. However, the sheer volume of medical research has exploded to a point where this task has become overwhelming. A single review can easily pull up tens of thousands of records from global databases. Before researchers can read the full text of a study to see if it answers their question, they must first read the title and the short summary, known as the abstract, of every single one of those thousands of papers. This initial screening is slow, expensive, and exhausting, often requiring teams to spend months reading through irrelevant material just to find the few dozen studies that actually matter.
To solve this, researchers have begun using software to help sort the pile, but many of these tools are expensive or require complex technical skills to operate. A new tool called OlivePICO offers a different approach. It is a free, browser-based application that acts as a smart filter before the human screening even begins. Instead of just looking for random keywords, it uses a structured method called PICO, which stands for Population, Intervention, Comparator, and Outcome. This framework helps the software understand the specific ingredients of a research question: who the study is about, what treatment was tested, what it was compared against, and what result was measured. By matching these specific ingredients in the titles and abstracts, the tool can quickly separate the wheat from the chaff, sending only the most promising records to human reviewers while automatically setting aside the rest.
The developers of OlivePICO, Mohammad Sayeed Syed and Jehath Syed, built this tool to run entirely inside a web browser, meaning it requires no installation, no subscription fees, and no data to be uploaded to a remote server. The process is straightforward: a researcher uploads their list of thousands of records, defines their specific PICO criteria, and sets any exclusion rules, such as filtering out editorials or case reports. The software then scans every record. If a study matches the required population and outcome, it is flagged for review. If it matches some but not all criteria, it is marked as uncertain for a human to check later. If it clearly does not fit, it is excluded. The system also has a safety feature that automatically detects and separates any records that have been officially retracted, ensuring that flawed studies do not accidentally make it into the final pool.
To see if this tool actually works, the team tested it against a completed review on deprescribing medications for older adults. In this test, the tool successfully identified every single one of the sixteen studies that the human team had originally included, achieving perfect sensitivity. To ensure the tool wasn't just lucky, they also randomly checked a sample of the studies that the software had rejected. They found that only one out of 150 rejected studies should have been kept, a very low error rate that is well within acceptable limits. The researchers then put OlivePICO to work on two new, independent reviews about falls in older adults, comparing its performance against a custom-built computer program that used a simpler, less structured method of keyword matching.
The results showed a clear difference in quality. In the first review, the custom computer program sent thousands of extra records to the humans that were not actually relevant, while OlivePICO sent far fewer. More importantly, when the researchers looked at the records that only OlivePICO found, nearly two-thirds of them turned out to be relevant to the study. In contrast, the records that only the custom program found were almost entirely irrelevant. In the second review, the pattern held true: OlivePICO captured a high percentage of relevant studies that the other tool missed, while the other tool captured very few relevant studies that OlivePICO missed. This suggests that the structured PICO approach is far better at understanding the context of a study than simple keyword searching.
The study also highlighted how the tool handles real-world messiness. For instance, the software successfully processed data from engineering databases, which use different formatting than medical databases, without needing any special adjustments. It also demonstrated that the tool can be refined; when the initial test missed a study because a specific term was missing from the keyword list, the researchers simply added that term and the tool immediately improved. This ability to be audited and adjusted makes the process transparent, unlike some artificial intelligence systems where the decision-making process is a black box. The researchers conclude that OlivePICO is a practical, free solution that can significantly reduce the workload for research teams, allowing them to focus their human effort on the studies that truly matter without sacrificing the thoroughness required for high-quality medical evidence.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.