Evaluating Cross-Method Explanation Consistency in XGBoost-Based Heart-Disease Classification

In the modern landscape of healthcare, artificial intelligence has become a powerful tool for spotting patterns in medical data that human eyes might miss. These computer systems can learn to predict whether a patient has a specific condition by analyzing a list of measurements, such as age, blood pressure, and heart rate. However, a significant challenge remains: while these models can make accurate predictions, they often operate as "black boxes." They provide an answer without explaining the reasoning behind it. In high-stakes fields like medicine, knowing the diagnosis is not enough; doctors and patients need to understand which factors led to that conclusion. This is where the field of explainable artificial intelligence steps in, offering methods to peek inside the machine's logic. Yet, a new complication has emerged: different methods of peering inside the machine can sometimes tell different stories. One technique might highlight a patient's age as the most critical factor, while another points to their cholesterol levels. This inconsistency raises a vital question: if the tools used to explain the model disagree with each other, can we trust the explanation at all?

A recent study by Aritrik Ghosh at Swami Vivekananda University tackles this very problem by examining how well different explanation methods agree when applied to a model designed to detect heart disease. The researcher focused on a specific type of machine learning model known as XGBoost, which is widely used for its ability to handle complex data. The study utilized a well-known collection of medical records containing 297 patient cases, each described by thirteen different health indicators and a final diagnosis of either heart disease or no heart disease. The goal was not to invent a new way to predict heart disease, but to rigorously test whether the various ways of explaining the model's decisions were consistent. The researcher trained the model on a portion of the data and then applied three distinct explanation techniques to see if they identified the same most important health factors.

The first technique used was the model's own internal measure of importance, which simply counts how often a specific health indicator was used to make a decision. The second method, known as permutation importance, works by scrambling the values of one health indicator at a time to see how much the model's accuracy drops; if the model stumbles significantly when a specific number is jumbled, that number is deemed important. The third approach, called SHAP, calculates the specific contribution of each factor to a prediction based on a mathematical framework that ensures the contributions add up correctly. When the researcher compared the results from these three methods on a fixed set of test cases, they found a strong, though not perfect, agreement. All three methods consistently identified the same top few factors as the most influential: the number of major blood vessels with blockages, the result of a thallium stress test, and the type of chest pain a patient reported.

However, the study revealed that consistency is not guaranteed, especially when the data is small or the model is tested on different groups of patients. When the researcher split the data into five different groups and tested the model repeatedly, the ranking of the most important factors began to shift. While the top three factors remained stable across these different tests, other factors that seemed important in one group moved up and down in the rankings in others. This finding suggests that a single explanation generated from one specific slice of data might not be reliable enough to represent the whole picture. The study also compared the model's predictions against other standard machine learning methods. On the specific test set used to mimic the original setup, a logistic regression model and a random forest model actually achieved higher accuracy than the XGBoost model, reaching 90 percent correct predictions compared to XGBoost's 86.67 percent. This highlights that the most complex model is not always the most accurate one, and that simpler models can sometimes perform just as well.

Perhaps the most intriguing part of the research involved looking at how these explanation methods handle individual patient cases. The researcher took three specific patient records that had been previously analyzed with a different explanation tool called LIME and compared those old results with new calculations using the SHAP method. While both methods agreed on the direction of the influence—meaning they both said a factor was either increasing or decreasing the risk of disease—they did not always agree on which specific factors were the top five most important for that individual. In some cases, the overlap between the two lists was only partial. This demonstrates that even when two tools seem to be looking at the same patient, they can prioritize different details. It is similar to two experts looking at a complex machine; one might focus on the gears while the other focuses on the fuel line, and both might be correct about the machine's function, yet they would list different parts as the most critical.

The study concludes that while artificial intelligence can be a valuable partner in medicine, the tools used to explain its decisions must be treated with caution. The research shows that predictive performance and explanation consistency are related but distinct dimensions. A model can be accurate without its explanations being perfectly stable, and different explanation methods can yield different lists of important factors. The findings emphasize that in healthcare, where decisions affect human lives, relying on a single explanation from a single model run is risky. Instead, a robust understanding requires checking multiple methods and acknowledging that some factors, like the number of blocked vessels, are consistently important, while others may vary depending on the specific data being analyzed. The work serves as a reminder that before these systems can be fully trusted in clinical settings, we must understand not just what they predict, but how stable and consistent their reasoning truly is.