Evidence-Grounded Large Language Model Reasoning for Patient Safety Using Knowledge Graphs and Retrieval-Augmented Generation: Development and Evaluation Study

Hospitals generate a constant, overwhelming stream of information. Every day, doctors and nurses write notes, order tests, record medications, and log vital signs. Much of this critical data lives in unstructured text, hidden inside paragraphs of narrative that are difficult for computers to read and connect. For years, researchers have tried to build artificial intelligence that can sift through these records to spot patient safety issues, such as dangerous drug interactions or signs of a worsening condition. The hope is that machines could act as a second pair of eyes, catching errors that busy humans might miss. However, simply teaching a computer to read text is not enough. If the machine guesses a problem without a clear reason, or if it invents a connection that does not exist, it could cause more harm than good. The challenge is not just finding the information, but proving that the conclusion is backed by real medical evidence and that the reasoning can be traced back to a source a doctor can trust.

In a new study, researchers set out to build a system that solves this problem by combining three distinct tools: a large language model, a structured map of medical knowledge, and a method for pulling in fresh evidence from outside sources. They wanted to see if they could create a reasoning engine that does not just guess, but explains its thinking using a solid foundation of facts. The team took data from thousands of adult patients in intensive care units, including their medical notes and test results, and fed it into their system. The goal was to see if the machine could identify safety concepts, link them together logically, and support its findings with relevant medical literature, all while keeping a clear record of where every piece of information came from.

The researchers built their system in layers. First, they took the messy, unstructured notes from patient records and cleaned them up, turning different ways of writing the same thing into a single, standard term. They then organized these terms into a structured map, known as a knowledge graph. Think of this map as a detailed family tree for medical concepts, where it explicitly shows how a specific drug connects to a specific side effect, or how a lab result relates to a patient's condition. This map acts as a rigid skeleton, ensuring that the computer understands the relationships between different medical facts before it tries to write anything. Next, the system was equipped with a retrieval tool. Instead of relying only on what the computer already knew from its training, this tool reached out to a vast library of medical journals and safety reports. When the system encountered a patient case, it would pull in the most relevant articles and guidelines to serve as supporting evidence. Finally, a large language model, which is a type of artificial intelligence trained on vast amounts of text, used all this information—the structured map, the patient data, and the retrieved articles—to generate a reasoned explanation of any potential safety issues.

To test if this approach actually worked, the researchers ran a series of experiments where they removed one piece of the system at a time to see how much each part mattered. They found that the system was highly effective at identifying safety concepts, achieving a high level of accuracy. However, the most important discovery came when they looked at what happened when they took away the structured map. Without the knowledge graph, the system's performance dropped significantly. The accuracy fell, and the ability to distinguish between true safety signals and false alarms weakened. This proved that the rigid, structured map of medical relationships was the most critical component, providing a necessary foundation that the other tools could not replace on their own.

The role of the retrieval tool was more subtle. When the researchers removed the part that fetched external medical articles, the system's performance dipped slightly, but not enough to be statistically certain that the drop was real. This suggests that while having fresh, external evidence is helpful and makes the system's reasoning more grounded, it was not the primary driver of the system's success in this specific test. The study also looked at how well the system adapted to the medical field. They used a technique to fine-tune the large language model, making it more specialized for patient safety. When they removed this fine-tuning, the system's performance barely changed, indicating that in this setup, the structured knowledge and the retrieval of evidence were doing the heavy lifting, rather than the specific adjustments made to the language model itself.

Human experts reviewed the system's output to see if it made sense in a real-world context. They found that the documents the system retrieved were indeed relevant to the questions it was asked. More importantly, the explanations the system generated were judged to be clinically useful and plausible. The experts felt that the system did not just spit out a random guess; it provided a reasoning path that a doctor could follow, backed by the specific articles and data points the system had gathered. A preliminary check of the user interface showed that clinicians could navigate the results without trouble, though they suggested that the display could be simplified to highlight the most important information first.

The study concludes that combining a structured map of medical knowledge with a language model and external evidence retrieval creates a powerful tool for patient safety. The results show that the structured map is the most vital part, providing the logical backbone that allows the system to reason correctly. The ability to pull in outside evidence adds a layer of transparency, allowing the system to show its work and cite its sources. While the system is not yet ready to replace human doctors or make autonomous decisions, it demonstrates a clear path forward for building artificial intelligence that is trustworthy, explainable, and grounded in real medical facts. The researchers emphasize that before such a system can be used in routine hospital care, it must be tested in many different hospitals and evaluated in real-time workflows to ensure it works safely across diverse patient populations. For now, the study proves that it is possible to build a machine that does not just read medical notes, but understands them in a way that is both accurate and accountable.