Large Language Models Across the Clinical Trial Lifecycle: A Scoping Review of Applications, Methodologies, and Validation Gaps
This scoping review of 71 studies published between 2023 and 2025 reveals that while large language models are increasingly applied to clinical trial recruitment and data analysis, their transition to routine practice is hindered by fragmented evidence, a lack of comparative benchmarks, and insufficient prospective validation.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Clinical trials are the rigorous, expensive, and time-consuming experiments that determine whether a new medicine or treatment is safe and effective for people. Before a drug can reach a pharmacy, it must pass through a series of carefully controlled steps: researchers design the rules of the experiment, find the right volunteers to participate, guide them through the study, and finally analyze the mountains of data they produce. For decades, these steps have relied heavily on human effort, requiring teams of experts to read thousands of pages of medical records, write complex legal documents, and match patients to the right studies. Recently, a new type of computer program has entered this world. These programs, known as large language models, are trained on vast amounts of text and can read, write, and reason with human language in ways that were impossible for earlier software. They offer the promise of speeding up the slow, labor-intensive parts of clinical research, but because the technology is so new, it was unclear exactly where these tools were being used, how well they actually worked, and whether they were ready for real-world use.
A team of researchers set out to map this rapidly changing landscape by reviewing every study they could find that applied these language models to clinical trials between January 2023 and August 2025. They examined 71 distinct studies to see how the technology was being used from the very first idea of a trial to the final analysis of its results. Their investigation revealed that the use of these models is growing fast, but it is also uneven. The technology is currently most active in two specific areas: analyzing data after a trial is over, and helping to recruit participants before a trial begins. In the realm of data analysis, the models are being used to pull specific facts out of thousands of medical papers, summarize complex findings, and check if reports follow the necessary rules. In recruitment, the models are helping to translate complicated medical eligibility rules into simple lists and matching patient records to the right studies. However, the researchers found that the models are rarely used during the actual running of the trial, such as when interacting directly with patients or managing the day-to-day activities of the study.
The review also looked closely at how these computer programs were being tested. In nearly half of the studies, the researchers compared the new models against older computer methods and found that the language models performed better. Yet, in almost as many studies, no such comparison was made at all, leaving it unclear if the new tools were truly an improvement. More importantly, the researchers found a significant gap between how well the computers performed on paper and how well they would work in a real hospital or clinic. While about one-third of the studies included some form of human evaluation, where people checked the computer's work, these checks were often done after the fact by looking at old records. Very few studies tested the models in real-time with actual patients or doctors to see if the advice the computer gave was safe and useful. When human experts did look at the results, they sometimes found that the models missed important details or made recommendations that were not quite right.
Another critical finding concerned the data these models were trained on. Most of the studies relied on information that was already public, such as medical journals and trial registries available on the internet. Only a small number of studies used private, sensitive patient records from hospitals. When private data was used, the researchers often failed to clearly explain how they protected patient privacy or whether they had official approval to use the information. This lack of clear reporting makes it difficult to know if the technology is being used responsibly. The authors of the review conclude that while these language models are powerful tools that are expanding quickly, they are not yet ready to be trusted as standard parts of the clinical trial process. Before they can be used routinely, more work is needed to prove they are safe, to test them in real-world settings, and to ensure that the data they use is handled with the utmost care. The technology is moving fast, but the evidence supporting its use is still catching up.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.