AI-driven selection of patients with non-valvular atrial fibrillation for oral anticoagulation therapy: a multi-cohort validation and impact evaluation study

This multi-cohort study demonstrates that the AI-driven TRisk model significantly outperforms established clinical scoring systems in predicting thromboembolic and bleeding events for atrial fibrillation patients across UK and US populations, offering a pathway to reduce unnecessary anticoagulant prescriptions and generate substantial healthcare cost savings without compromising patient safety.

The Gist

The Big Picture: Fixing the "One-Size-Fits-All" Medicine Problem

Imagine you have a very common heart rhythm problem called Atrial Fibrillation (AF). Think of your heart as a drum that's supposed to beat in a steady rhythm, but in AF, it's drumming chaotically. This chaos can throw tiny clots (like little pebbles) into your bloodstream, which can travel to your brain and cause a stroke.

To stop this, doctors usually prescribe a blood thinner called a DOAC (Direct Oral Anticoagulant). It's like putting a shield on your blood so those pebbles can't stick together.

The Problem:
Right now, doctors decide who gets this shield using a simple checklist called CHA₂DS₂-VASc. It's like a basic weather forecast: "If you are over 65, have high blood pressure, and had a stroke before, you get a raincoat."

• The Flaw: This checklist is a bit outdated. It's like using a 1990s map to navigate a city that has built three new highways and a subway system since then. It misses a lot of nuance. Sometimes it tells healthy people to take the shield (wasting money and risking bleeding), and sometimes it misses people who actually need it. Also, your risk changes over time, but the checklist doesn't get smarter as you get older or your health changes.

The Solution:
The researchers built a new, super-smart AI tool called TRisk. Think of TRisk not as a checklist, but as a high-tech personal historian. Instead of just looking at three or four facts, it reads your entire medical history—every doctor visit, every medication, every lab test, and every procedure you've ever had. It uses a type of AI called a "Transformer" (the same tech behind smart chatbots) to understand the story of your health, not just the bullet points.

---

How They Tested It: The "School Exam" Analogy

The researchers didn't just build the tool; they put it through a rigorous exam to see if it was better than the old checklist.

1. The Training: They taught TRisk using the medical records of over 400,000 patients in the UK. They let the AI study the patterns of who got strokes and who got bleeding problems.
2. The Exam (UK): They tested it on a new group of 98,000 UK patients.
3. The Exam (US): They even sent the AI to the US to take a test on 16,000 American patients to see if it could handle a different healthcare system.

The Results:

• The Old Checklist (CHA₂DS₂-VASc): Got a score of about 0.71. (Think of this as a "C" grade). It was okay, but it missed a lot of details.
• The New AI (TRisk): Got a score of 0.82. (Think of this as an "A" grade).
• The Bleeding Test: The AI was also better at predicting who might bleed too much from the medication, outperforming the old bleeding checklists too.

The "Fairness" Check:
They made sure the AI didn't have bias. It worked just as well for men, women, different races, and even for people whose health records were created during the chaotic days of the pandemic. It didn't discriminate; it just looked at the data.

---

The Real-World Impact: Saving Money and Lives

Here is the most exciting part. The researchers asked: "If we use this smart AI instead of the old checklist, what happens?"

Imagine a hospital waiting room with 1,000 people who might need the blood thinner.

• With the Old Checklist: The doctor would say, "Okay, 814 of you need this shield."
• With the AI: The doctor says, "Actually, only 743 of you need it. The other 71 people are fine without it."

Why does this matter?

1. Fewer Bleeding Accidents: By not giving the shield to people who don't need it, fewer people suffer from dangerous bleeding.
2. Huge Savings: Blood thinners are expensive.
   • In the UK, this switch could save the healthcare system £5.5 million a year just for new patients, and nearly £50 million if applied to everyone already on the drug.
   • In the US, the savings are massive: $456 million for new patients, and a staggering $1.8 billion if applied to everyone.

The "Magic" Number:
The study calculated that even if the AI system cost £50 per patient (or $508 in the US) to set up and run, it would still save money overall because it prevents so many expensive hospital visits for bleeding. It's a "win-win": better health and lower costs.

---

The "Black Box" Mystery: How Does the AI Think?

Usually, AI is a "black box"—you put data in, and a number comes out, but you don't know why. The researchers used a special technique called Explainability to peek inside the box.

They found that the AI was thinking like a brilliant doctor:

• It knew that a history of heart attacks or strokes increased risk (just like the old checklist).
• But it also knew things the checklist missed: It noticed that certain types of digestive issues or specific past injuries increased bleeding risk.
• It understood time: It realized that if a patient started taking a specific blood thinner (like Apixaban), their risk of a stroke went down dynamically. The old checklist couldn't see that change; it just saw a static list of facts.

The Bottom Line

This paper is about upgrading our medical GPS.

• The Old Way: Using a paper map that hasn't been updated in 20 years. It gets you to the destination, but it might take you through traffic jams or wrong turns.
• The New Way (TRisk): Using a live, real-time GPS that knows about traffic, road closures, and your specific driving habits.

By using this AI, doctors can give the right blood thinner to the right person at the right time. This means fewer strokes, fewer bleeds, and billions of dollars saved, all while making the healthcare system smarter and more precise. It's a major step toward "precision medicine" where treatment is tailored specifically to your unique story, not just a generic rule.

Technical Summary

1. Problem Statement

Current clinical guidelines for managing non-valvular atrial fibrillation (AF) rely on static risk scores (e.g., CHA₂DS₂-VASc for thromboembolism and HAS-BLED/ORBIT for bleeding) to determine the initiation of Direct Oral Anticoagulants (DOACs). The authors identify three critical limitations in these existing tools:

• Modest Predictive Performance: These scores consistently demonstrate low discrimination (C-index < 0.8 for thromboembolism and < 0.7 for bleeding), leading to suboptimal risk stratification.
• Static Nature: Traditional scores rely on cumulative factors that generally only increase over time (e.g., aging), failing to capture dynamic improvements in a patient's health or evolving risk profiles. They are often developed on DOAC-naïve cohorts and may not accurately reflect residual risk in treated patients.
• Clinical Inefficiency: Due to poor calibration and the complexity of periodic reassessment, guidelines recommending regular risk re-evaluation are often not followed, leading to over-prescription of DOACs and unnecessary bleeding risks.

2. Methodology

The study developed and validated TRisk, a Transformer-based survival model designed to predict 12-month thromboembolic and bleeding events using longitudinal Electronic Health Record (EHR) data.

Data Sources & Cohorts

• UK Derivation & Validation: Utilized the Clinical Practice Research Datalink (CPRD) Aurum dataset (1,442 English general practices).
  • Derivation: 1,079 practices (n=313,674 patients).
  • External Validation: 363 practices (n=98,176 patients).
  • Cohort: 411,850 prevalent non-valvular AF patients (aged ≥18) between 2010–2020.
• US External Validation: Utilized the All of Us (AoU) Research Program dataset (n=16,218 patients) between 2010–2023.
• Baseline Definition: Baseline was randomly selected from the period between the first AF diagnosis and the study end to simulate dynamic risk assessment at any stage of the disease journey, including patients already on DOACs.

Model Architecture (TRisk)

• Core Architecture: A Transformer-based survival model adapted from the BEHRT architecture, utilizing SODEN (Scalable Continuous-Time Survival Model through Ordinary Differential Equation Networks) for efficient handling of censored survival data.
• Input Data: Processes complete patient medical histories as sequences of timestamped records, including:
  • 4,687 diagnosis codes (mapped to ICD-10).
  • 1,312 medication codes (mapped to RxNorm ingredients).
  • 2,624 procedure codes (mapped to SNOMED).
  • 52 laboratory tests.
  • Total: ~8,675 unique codes in the UK model; expanded to ~9,251 for the US model.
• Key Innovations:
  • Temporal Dynamics: Incorporates relative ordering of visits and patient age at each encounter.
  • No Imputation: Utilizes raw EHR data without imputing missing values, relying on the model's ability to handle sparsity.
  • Explicit Calibration: Optimizes D-calibration alongside empirical loss to ensure predicted probabilities match observed risks.
  • Transfer Learning: For the US cohort, the UK-trained model was fine-tuned using ontology-driven vocabulary mapping. New US-specific codes were mapped to their closest UK ontological matches, and their embeddings were initialized using the pre-trained weights of the closest existing codes.

Benchmarking & Evaluation

• Comparators: TRisk was compared against CHA₂DS₂-VASc and CHA₂DS₂-VA (thromboembolism) and HAS-BLED and ORBIT (bleeding).
• Metrics: Discrimination (C-index, AUPRC), Calibration (calibration curves), and Clinical Utility (Decision Curve Analysis).
• Explainability: Used Integrated Gradients to identify which specific medical encounters contributed most to risk predictions.
• Impact Analysis: A decision model estimated the economic and clinical impact of replacing standard-of-care guidelines with TRisk-guided recommendations, focusing on DOAC initiation rates, bleeding events, and healthcare costs.

3. Key Contributions

1. Development of TRisk: The first AI model specifically designed to capture evolving, multi-factorial risk profiles in AF patients using deep learning on longitudinal EHRs, moving beyond static "snapshot" scoring.
2. Multi-Cohort & Cross-Geographic Validation: Successfully validated the model in two distinct healthcare systems (UK and US) and demonstrated temporal generalizability (performance remained robust from 2010 through 2023).
3. Transfer Learning Framework: Demonstrated a novel method for adapting a complex AI model from one country to another with minimal retraining (fine-tuning on <6,500 patients) by mapping ontological embeddings, solving the "domain shift" problem.
4. Economic & Clinical Impact Quantification: Provided a rigorous cost-benefit analysis showing that AI-driven selection can reduce DOAC prescriptions without compromising safety, leading to substantial cost savings.

4. Key Results

Predictive Performance

• Thromboembolism (UK Validation): TRisk achieved a C-index of 0.82 (95% CI: 0.81–0.83), significantly outperforming CHA₂DS₂-VASc (0.71) and CHA₂DS₂-VA (0.72). This represents a ~13% improvement in discrimination.
• Thromboembolism (US Validation): TRisk maintained high performance with a C-index of 0.82 (0.80–0.84), compared to 0.72 for CHA₂DS₂-VASc.
• Bleeding (UK Validation): TRisk achieved a C-index of 0.70 (0.69–0.71), outperforming HAS-BLED (0.63) and ORBIT (0.64).
• Bleeding (US Validation): TRisk achieved a C-index of 0.71 (0.69–0.74) vs. 0.66 for HAS-BLED.
• Subgroups: The model performed equitably across sex, age, race, and during the COVID-19 pandemic, with no evidence of bias.

Explainability

• The model identified established risk factors (e.g., prior stroke, cardiovascular events) and novel predictors (e.g., cardiac arrest).
• It successfully captured protective effects: The presence of DOACs (e.g., apixaban, rivaroxaban) in the patient history attenuated the predicted thromboembolic risk, demonstrating the model's ability to account for treatment effects dynamically.

Impact Analysis (Economic & Clinical)

• UK: Implementing TRisk would reduce DOAC prescriptions by 8% (avoiding initiation in ~8,857 patients annually) while capturing the same number of true positive thromboembolic events as standard care.
  • Savings: £5.5 million annually for new initiators; rising to £48.6 million if extended to all DOAC users.
  • Safety: Prevents ~211 bleeding events annually.
• US: TRisk would reduce DOAC prescriptions by 7% (avoiding ~65,688 initiators annually).
  • Savings: $456.2 million annually for new initiators; rising to $1.8 billion if extended to all users.
  • Safety: Prevents ~4,335 bleeding events annually.
• Cost-Effectiveness: The strategy is cost-saving (dominant) under standard willingness-to-pay thresholds (NICE: £25k-£35k/QALY; US: ~$100k-$150k/QALY).

5. Significance

This study demonstrates that AI-driven dynamic risk assessment can significantly outperform established clinical guidelines in managing atrial fibrillation.

• Clinical Safety: By reducing false positives (over-treatment), TRisk can prevent thousands of bleeding events annually without missing high-risk patients who need anticoagulation.
• Economic Efficiency: The potential for hundreds of millions (US) and tens of millions (UK) in annual savings highlights the viability of AI in healthcare cost containment.
• Scalability: The successful transfer learning approach suggests that complex AI models can be rapidly adapted to new geographic regions and coding systems, overcoming a major barrier to global AI deployment in healthcare.
• Paradigm Shift: The study advocates for a shift from static, guideline-based scoring to continuous, data-driven risk monitoring that evolves with the patient's medical journey.

Conclusion: TRisk offers a precise, unbiased, and economically superior alternative to current AF risk stratification tools, capable of optimizing DOAC therapy to maximize patient safety and minimize healthcare costs.