A Systematic Literature Review of Large Language Models in Clinical Triage from Risk Stratification to Human Escalation
This systematic and bibliometric review synthesizes 89 studies published between 2022 and 2026 to analyze the application of Large Language Models in clinical triage, evaluating their adaptation strategies, safety mechanisms, and performance metrics while proposing a new 5-dimension framework to guide their safe translation into emergency care settings.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Every day, emergency rooms face a relentless tide of people seeking care. Some arrive with minor scrapes, while others are fighting for their lives. The critical job of sorting these patients, known as triage, falls to nurses and doctors who must decide instantly who needs immediate attention and who can wait. This process relies on standardized scales that assign a level of urgency, from the most critical cases requiring resuscitation to those that are non-urgent. However, as hospitals become more crowded and the volume of digital health intake grows, the pressure on these human gatekeepers intensifies. In recent years, a new type of artificial intelligence has emerged that can read and understand human language with remarkable fluency. These systems, known as large language models, can interpret complex stories about symptoms and suggest medical advice. But while they offer a potential tool to help manage the flow of patients, they also carry significant risks. If such a machine misjudges a situation, the consequences can be severe, ranging from unnecessary delays for critical patients to the overwhelming of emergency resources with those who do not need them.
A team of researchers from institutions in the United States, India, and Europe set out to understand exactly how these artificial intelligence systems are being tested for this high-stakes role. They conducted a massive review of ninety scientific studies published between 2022 and 2026. Their goal was not just to see if the technology works, but to examine how it is built, how it is tested, and whether it is safe enough to ever be used in a real hospital. The researchers found that while these computer programs are becoming increasingly common in research, they are not yet ready to make life-or-death decisions on their own. The studies they analyzed revealed a landscape where the technology is improving rapidly, yet it still struggles with the most dangerous errors: missing a patient who is in critical condition.
The review began by gathering every available study that looked at using these language-based systems for patient sorting. The researchers sifted through hundreds of records, eventually settling on ninety primary studies that met strict criteria. They discovered that nearly half of these studies focused on emergency departments, where the need for speed is greatest. Another significant portion looked at outpatient clinics and pre-hospital emergency services. The researchers noticed a clear trend in how these systems were being developed. In the earliest studies, scientists simply asked the computer to answer questions without any special training, a method known as zero-shot prompting. However, as time went on, researchers began to use more sophisticated techniques. They started feeding the systems specific medical guidelines or connecting them to databases of hospital rules to help them make better decisions. Some teams even built systems where multiple computer agents worked together, with one agent proposing a diagnosis and another acting as a critic to check for errors.
Despite these technical advances, the performance data told a complex story. When tested against standard medical scales, the artificial intelligence systems often achieved high overall accuracy, correctly categorizing patients in roughly eighty percent of cases. However, the researchers emphasized that a high average score hides a dangerous flaw. In the world of emergency medicine, not all mistakes are equal. A system that overestimates the severity of a minor injury simply sends a patient to the front of the line, causing a delay for others but rarely causing harm. In contrast, a system that underestimates the severity of a critical condition sends a dying patient to the back of the line, potentially leading to death. The review found that while the systems were generally good at identifying minor issues, they frequently failed to spot the most urgent cases. In some studies, the rate of missing critical patients was as high as twenty-five percent. One specific study highlighted that the systems missed a significant number of cases involving children with objects stuck in their airways, a life-threatening emergency that requires immediate action.
The researchers also examined how these systems handle uncertainty. A human doctor knows when they are unsure and will ask for help or run more tests. The computer programs, however, often do not know when they are guessing. The review found that in nearly seventy percent of the studies, the researchers did not include any mechanism to measure how confident the system was in its answer. Without this safety check, a computer might give a wrong answer with absolute certainty, which is far more dangerous than a hesitant one. Furthermore, many of the studies relied on data that was already written down in hospital records or created by scientists in a lab setting. These static records do not capture the chaotic reality of a busy emergency room, where a patient's condition can change minute by minute, or where a family member might describe symptoms in a confused or emotional way.
To address these dangers, the authors proposed a new framework for how these systems should be built and tested before they are ever used on real people. They argued that a computer should never be the first line of defense for the most critical emergencies. Instead, they suggested a layered approach. The first layer should be a simple, unchangeable set of rules that instantly flags specific keywords like "chest pain" or "difficulty breathing" and routes those patients directly to a human doctor, bypassing the complex language model entirely. The second layer would involve the language model itself, but only if it is constantly checked against medical guidelines and if it is programmed to admit when it is unsure. If the system is not confident, it should stop and ask a human for help. The final layer is the human doctor, who must always have the final say. The review concluded that while these tools hold promise for helping doctors manage their workload, they are not yet ready to replace human judgment. The technology is moving fast, but the safety measures needed to trust it with human lives are still being built. Until those safeguards are in place, the role of these systems must remain that of a supportive assistant, never an autonomous decision-maker.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.