Explainable Artificial Intelligence for Predictive Modeling in Nephrology: A Scoping Review
This scoping review of 18 studies demonstrates that while explainable AI models, particularly those using XGBoost and SHAP, achieve high predictive accuracy for kidney disease outcomes, their clinical translation is currently hindered by a lack of prospective, multi-center validation and integration into real-world workflows.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Every year, millions of people around the world face the threat of kidney failure, a condition that can silently progress until it requires dialysis or a transplant to survive. Doctors have long relied on standard tests, like measuring the level of a waste product called creatinine in the blood, to spot these problems early. In recent years, powerful computer programs known as machine learning have begun to help physicians predict who is most likely to get sick. These programs can sift through vast amounts of patient data—such as age, medical history, and lab results—to find complex patterns that the human eye might miss. However, these programs often operate like a sealed black box: they give a prediction, but they do not explain how they reached that conclusion. This lack of transparency makes many doctors hesitant to trust the computer's advice, especially when a wrong guess could mean the difference between life and death. To solve this, researchers are developing a new approach called explainable artificial intelligence, which forces the computer to show its work, revealing exactly which factors led to its prediction.
A team of researchers set out to see how well this new approach is working specifically for kidney disease. They gathered and reviewed eighteen different studies published between 2019 and 2025 that used these explainable tools to predict outcomes like acute kidney injury, the worsening of chronic kidney disease, or death. Their goal was to determine if these models were not only accurate but also ready to be used in real hospitals. The researchers found that the computer models were indeed very good at making predictions. In the studies they examined, the models correctly identified patients at risk with a high degree of accuracy, often matching or beating the performance of traditional statistical methods. The most common type of computer program used was a sophisticated algorithm known as XGBoost, and the most popular way to make these programs explain themselves was a method called SHAP. This technique acts like a spotlight, highlighting the specific details that mattered most for each patient. In almost every case, the computer pointed to the same things that human doctors already know are important: high levels of creatinine, low kidney filtration rates, older age, and a history of diabetes.
Despite these promising results, the review uncovered a significant gap between what the computers can do in a lab and what is actually happening in patient care. The vast majority of the studies, about ninety-four percent, looked backward at data that had already been collected, rather than testing the models on new patients in real time. Furthermore, most of the research was done at a single hospital, meaning the models had not been tested to see if they would work in a different city or a different country. While the models performed well on the data they were trained on, none of the eighteen studies reported that these tools had been successfully integrated into the daily workflow of a hospital. No doctor was using a dashboard or an alert system powered by these explainable models to make decisions for a patient right now. The researchers noted that while the technology is technically sound and identifies the right risk factors, it remains stuck in the proof-of-concept stage.
The authors conclude that while explainable artificial intelligence holds great promise for nephrology, it is not yet ready for prime time. The tools are accurate and transparent enough to be trusted in theory, but they have not yet proven they can survive the messy reality of a busy hospital. To move forward, the field needs studies that test these models on new patients as they arrive at the hospital, rather than just on old records. Researchers also need to figure out how to connect these computer systems to the electronic records doctors use every day. Until these steps are taken, the powerful predictions of these explainable models remain a potential solution waiting for the right moment to be put into practice.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.