💻 computer science

CardioTrust: Trustworthy Retrieval Augmented Clinical Decision Support for Personalized Cardiovascular Disease Prediction

CardioTrust is a novel, trustworthy AI framework that integrates machine learning risk stratification, prediction-guided Hybrid Retrieval Augmented Generation, and SHAP explainability to deliver highly accurate, transparent, and clinically grounded personalized cardiovascular disease predictions, significantly outperforming traditional baseline models.

Original authors: Veerababu Reddy, Vasavi Durga Potla, Baji Noorbasha, Karthik Reddy Vajrala, Sadwika Bollu

Published 2026-08-07

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Veerababu Reddy, Vasavi Durga Potla, Baji Noorbasha, Karthik Reddy Vajrala, Sadwika Bollu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you have two very different assistants. One is a super-fast calculator that can crunch numbers and tell you, "There is a 95% chance the culprit is the butler," but it refuses to explain why. It just gives you the number. The other assistant is a brilliant storyteller who can write a beautiful, detailed narrative about the butler, the weather, and the candlestick, but sometimes, in their excitement, they make up facts that never happened. In the world of medicine, this is a huge problem. Doctors need to know why a patient is at risk for heart trouble, and they need that information to be 100% true, not made up. This is where the field of "Trustworthy Artificial Intelligence" comes in. It's the quest to build computer systems that are not only smart but also honest, explainable, and safe enough to help doctors save lives.

This paper introduces a new detective team called CardioTrust. Instead of letting the calculator and the storyteller work separately, the authors built a system where they work together in a strict, organized line. First, a powerful calculator (a Random Forest model) looks at a patient's data and predicts their risk of heart disease. But here is the twist: it doesn't just send a simple "high risk" or "low risk" note to the storyteller. Instead, it sends a specific "mission brief" that tells the storyteller exactly which risk factors are the most important (like high cholesterol or chest pain) and how confident the calculator is in its answer. The storyteller (a Large Language Model) then uses this brief to search a massive library of real medical rules and studies to find the perfect evidence. Finally, before the story is ever shown to a doctor, a strict "Safety Inspector" (the Trust Validation module) checks the whole package. It asks: "Did we find the right evidence? Does the story match the calculator's math? Is the story complete?" If the answer to any of these is "no," the report is sent back for a human to review.

The team found that this teamwork works incredibly well. When they tested CardioTrust on a dataset of 70,000 patient records, it predicted heart disease with 95.84% accuracy, which was significantly better than other standard computer models. But the real magic happened in the storytelling part. While a standard computer storyteller made up facts (hallucinations) about 21.64% of the time, CardioTrust's strict safety checks reduced those made-up facts to just 2.84%. The system also proved that the story it told always matched the math behind it, ensuring that the doctor isn't just getting a pretty story, but a truthful one backed by real medical guidelines. The authors suggest that this approach could be a game-changer for personalized heart care, provided it is tested on even more real-world hospital data in the future.

The Problem: The Calculator vs. The Storyteller

To understand why CardioTrust is such a big deal, we have to look at the two tools doctors have been using to predict heart disease, and why they both have flaws.

First, there are Machine Learning Models. Think of these as the super-fast calculators. They look at a patient's age, blood pressure, cholesterol, and other numbers, and they spit out a risk score. They are great at spotting patterns that humans might miss. However, they are often "black boxes." They give you a number, but they don't tell you why. If a doctor sees a high-risk score, they can't easily see if it's because of the patient's age or their smoking habit. Without that explanation, doctors are hesitant to trust the machine.

Second, there are Large Language Models (LLMs). These are the brilliant storytellers. They can read medical textbooks and write a detailed plan for a patient, explaining in plain English what might be wrong. The problem? They sometimes "hallucinate." This means they might confidently tell a doctor that a specific medicine cures a condition, even if that medicine doesn't exist or isn't recommended. In a hospital, a made-up fact can be dangerous.

For a long time, researchers tried to fix this by using a method called RAG (Retrieval Augmented Generation). This is like giving the storyteller a library card. Instead of making things up from memory, the storyteller is forced to look up the answer in a library of trusted medical books before writing the story. This helps, but the old way of doing it was a bit clumsy. The storyteller would just ask the library, "Tell me about heart disease," and get back a generic answer that didn't fit the specific patient's unique situation.

The Solution: A Team That Talks to Each Other

The authors of this paper realized that the calculator and the storyteller needed to talk to each other before the story was written. They built CardioTrust, a system where the math and the words are locked together.

Here is how the team works, step-by-step:

The Calculator (Random Forest): The system starts by analyzing the patient's data. It doesn't just say "High Risk." It calculates the exact probability (like 95.84%) and, crucially, it identifies the "villains" in the story. It uses a tool called SHAP to figure out exactly which factors are driving the risk. Is it the patient's age? Their chest pain? Their cholesterol? The calculator creates a "mission brief" listing these specific villains.
The Smart Search (Hybrid RAG): This is the first big innovation. Instead of asking the library, "Tell me about heart disease," the system sends the mission brief. It tells the library, "We have a patient with high ST depression and chest pain; find us the specific rules for that." This ensures the evidence retrieved is perfectly tailored to the patient's specific math.
The Storyteller (Llama 3.2): The storyteller takes the retrieved evidence and the mission brief and writes a personalized recommendation. Because it has the specific rules and the specific patient data, it doesn't need to guess.
The Safety Inspector (Trust Validation): This is the final gatekeeper. Before the report goes to the doctor, the Safety Inspector runs a checklist. It asks five questions:
- Is the calculator confident in its math?
- Did we find the right evidence?
- Does the evidence match up with other evidence?
- Does the story mention the same "villains" (risk factors) that the calculator found?
- Is the story complete (does it have a diagnosis, a plan, and advice)?

If the report gets a high enough score (above 0.75 on a scale of 0 to 1), it is sent to the doctor. If the score is too low, it is flagged for a human to review manually. This ensures that no bad or made-up advice ever reaches the patient.

The Results: Truth and Trust

The authors tested this system against many other methods, including standard calculators, standalone storytellers, and older versions of the library-search system. The results were impressive.

The Math: The core calculator in CardioTrust achieved an accuracy of 95.84% with a precision of 0.941 and a recall of 0.952. This means it was better at spotting heart disease than almost any other model they tested, including XGBoost and Support Vector Machines.

The Story: The real win was in the quality of the recommendations.

A standard storyteller (without the library) made up facts 21.64% of the time.
A standard library-search system (without the smart mission brief) made up facts 15.31% of the time.
CardioTrust made up facts only 2.84% of the time.

Furthermore, the system achieved a Trust Score of 0.96, which is a measure of how reliable the whole package is. The authors also checked if the story matched the math. They found that the "Explanation Agreement" was 0.95, meaning the story almost perfectly reflected the reasons the calculator gave for the risk.

The "What If" Check: The authors also tested what would happen if they removed parts of the system (an ablation study).

Without the smart search, the system was less trustworthy.
Without the Safety Inspector, the hallucination rate jumped back up to 8.21%.
Without the Safety Inspector and the smart search, the hallucination rate was 24.82%.

This proved that every single part of the team is necessary. You can't just have a good calculator; you need the smart search and the safety inspector to make it trustworthy.

Why This Matters

The authors are careful to say that this is a promising step, not a finished product. They tested the system on existing datasets (like the Kaggle Cardiovascular Disease Dataset with 70,000 records and the UCI Heart Disease Dataset with 1,025 records). While the results are strong, the system hasn't been tested in a real hospital with real doctors yet. The authors suggest that the next step is to see how it performs in the messy, complex reality of different hospitals and to get real doctors to rate the reports.

However, the core idea is clear: by forcing the math and the words to agree with each other, and by adding a strict safety check at the end, we can build AI that doctors can actually trust. CardioTrust isn't just a calculator or a storyteller; it's a team that checks its own homework before handing it in.

Technical Summary: CardioTrust

Problem Statement

Cardiovascular disease (CVD) remains a leading cause of global mortality, necessitating timely and precise risk stratification. While traditional machine learning (ML) models excel at predicting CVD risk, they often function as "black boxes," providing numerical scores without sufficient clinical interpretability, which limits clinician trust. Conversely, Large Language Models (LLMs) offer narrative clinical reasoning but suffer from hallucinations and ungrounded recommendations, posing significant safety risks in healthcare. Furthermore, existing Retrieval Augmented Generation (RAG) systems in healthcare typically retrieve information based on static semantic similarity, failing to incorporate patient-specific risk predictions, resulting in recommendations that lack personalization. There is a critical gap in integrating accurate ML prediction, evidence-grounded retrieval, explainable reasoning, and systematic trust validation into a unified clinical decision support framework.

Methodology

The authors propose CardioTrust, a trustworthy AI framework that unifies five tightly coupled modules into a closed-loop workflow:

Cardiovascular Disease Prediction Module: Utilizes a Random Forest ensemble classifier to predict disease probability $P(y=1|x)$ and estimate epistemic uncertainty. The confidence score ( $P_c$ ) is derived from the variance of tree-level probability estimates within the ensemble.
Prediction-Guided Hybrid RAG Module: Unlike conventional RAG, this module dynamically injects the predicted risk category and dominant SHAP risk drivers into the retrieval query. It retrieves evidence from authoritative sources (AHA/ESC guidelines and PubMed) using BAAI BGE Base EN v1.5 embeddings indexed in ChromaDB.
SHAP Explainability Module: Employs TreeSHAP to generate patient-specific local feature attributions ( $\Phi$ ), ranking clinical risk drivers. These attributions serve as both context for the LLM and a metric for validation.
Evidence-Grounded Clinical Reasoning Module: A Llama 3.2:3B model generates personalized clinical recommendations based on a fused context containing the patient profile, risk probability, confidence, SHAP vector, and retrieved evidence. The model operates at a low temperature ( $T=0.2$ ) for deterministic inference.
Trust Validation and Calibration Module: Before a report is presented, a quantitative audit computes a composite Trust Score ( $T$ ) based on five dimensions:
- Prediction Confidence ( $P_c$ )
- Retrieval Relevance ( $R_r$ )
- Evidence Consistency ( $E_c$ )
- Explanation Agreement ( $S_a$ ): Measures the alignment between SHAP feature importance and the features mentioned in the LLM's generated narrative.
- Recommendation Completeness ( $C_r$ )
If $T \geq \tau$ (threshold set at 0.75), the report is accepted; otherwise, it is routed to a manual audit queue.

Key Contributions

Prediction-Guided Hybrid RAG: The system dynamically conditions evidence retrieval on the specific predicted risk profile and SHAP drivers, rather than relying solely on generic semantic similarity.
Explanation Agreement as a Trust Signal: Introduces a quantitative metric ( $S_a$ ) that verifies whether the LLM's narrative reflects the same clinical features the underlying classifier relied on, closing the loop between XAI and text generation.
Unified Framework: Integrates Random Forest prediction, Hybrid RAG, TreeSHAP, and a multi-dimensional Trust Validation module into a single end-to-end pipeline with an explicit accept/audit decision gate.
Trust Validation Module: A pre-delivery safety gateway that assesses confidence, relevance, consistency, alignment, and completeness, instantiated here with an equal-weighted baseline configuration.
Comprehensive Evaluation: Includes an ablation study isolating the contribution of each component and rigorous cross-validation against multiple baselines.

Results

The framework was evaluated using a 5-fold cross-validation on the Kaggle Cardiovascular Disease Dataset (70,000 records) and an out-of-distribution check on the UCI Heart Disease Dataset (1,025 records).

Predictive Performance: The Random Forest core achieved 95.84% ± 0.45% accuracy, 0.982 AUROC, 0.941 precision, and 0.952 recall, significantly outperforming baselines like Logistic Regression (84.21%), SVM (86.50%), and XGBoost (92.15%).
Retrieval Quality: The Hybrid RAG module achieved superior retrieval metrics (Precision@5 = 0.946, NDCG = 0.952) compared to Lexical Search, Standard Dense Retrieval, and GraphRAG.
Recommendation Quality: CardioTrust achieved the highest semantic alignment with guidelines (BERTScore = 0.96) compared to standalone LLMs and other RAG variants.
Trust and Hallucination Reduction: The full framework achieved a composite Trust Score of 0.96 and reduced the hallucination rate to 2.84%, a significant improvement over the 21.64% observed in zero-shot LLM generation.
Ablation Study: Results confirmed that while predictive accuracy remained constant (governed by Random Forest), the integration of Hybrid RAG, TreeSHAP, and Trust Validation progressively reduced hallucinations and increased system reliability.

Significance and Claims

The paper claims that CardioTrust provides a robust, transparent, and clinician-aligned solution for personalized cardiovascular care. By unifying predictive modeling with evidence-based reasoning and automated trust assessment, the framework addresses the critical barrier of bridging high-accuracy prediction with verifiable, explainable clinical reasoning. The authors emphasize that the system mitigates the risks of LLM hallucinations and ungrounded therapy suggestions through its automated pre-delivery validation.

The paper maintains a modest stance regarding its current validation:

The Trust Score is an automated proxy based on embeddings and confidence; it has not yet been validated against a Clinician Acceptance Rate from a real expert reader study, which is identified as a necessary next step.
The weighting coefficients in the Trust Validation module are currently set to an equal-weighted baseline; empirical calibration against expert adjudicated labels is proposed as future work.
The UCI dataset results are presented as an out-of-distribution transferability check rather than a definitive proof of generalization, noting the need for prospective multicenter validation.

The authors conclude that CardioTrust represents a significant step toward trustworthy AI in healthcare but explicitly frames the current work as a foundation for future empirical calibration and real-world deployment testing.