Automatic Cardiac Risk Management Classification using large-context Electronic Patients Health Records

This study demonstrates that a custom Transformer architecture outperforms both traditional machine learning models and zero-shot generative LLMs in automatically classifying cardiac risk from large-context, unstructured Dutch electronic health records, offering a robust alternative to manual administrative coding for geriatric cardiovascular risk management.

The Gist

Imagine a busy hospital waiting room filled with elderly patients. Each patient has a thick, messy file folder containing years of handwritten notes, lists of medicines, and scattered test results. In the past, a nurse had to manually read through these thousands of pages to decide who was at high risk for heart disease and needed extra care. This was slow, prone to human error, and often missed subtle clues hidden deep in the text.

This paper describes a new automatic "smart assistant" designed to read these messy files instantly and accurately, acting like a super-powered triage nurse.

Here is the breakdown of how they built it, using simple analogies:

1. The Problem: The "Needle in a Haystack"

The hospital had a system where nurses assigned patients to a heart-risk program based on simple appointment codes (like "check-up" vs. "emergency"). But this was like sorting mail only by the color of the envelope, ignoring the actual letter inside. It missed important details about a patient's history, leading to mistakes. They needed a way to read the actual content of the medical notes.

2. The Ingredients: The "Digital Patient"

The researchers gathered data on 3,482 patients. They didn't just look at numbers (like age or blood pressure); they fed the computer the unstructured text of the doctors' notes.

• The Text: Long, complex stories written by doctors about what happened during visits.
• The Meds: A list of drugs, translated into a standard code (like translating "Aspirin" and "Bayer" both into the same universal ID).
• The Stats: Simple facts like age and gender.

3. The Contenders: Who is the Best Detective?

To find the best way to read these notes, they tested three different types of "detectives" (AI models):

• The Old School Detective (Traditional Machine Learning): This is like a detective who uses a highlighter to find specific keywords (e.g., "heart attack," "diabetes"). It's fast but can miss the context. If a doctor writes, "The patient did not have a heart attack," the old detective might get confused.
• The Genie (Large Language Models - LLMs): These are the famous AI chatbots (like the ones you might know). The researchers asked them to read the notes and guess the risk without teaching them anything first (Zero-Shot).
  • The Result: The Genie was surprisingly bad at this specific job. It was like asking a brilliant literature professor to read a technical medical manual in a foreign language; they understood the words but missed the specific medical rules.
• The Custom Architect (Hierarchical Transformer): This was the team's own invention. Imagine a detective who doesn't just read a sentence, but understands the entire story structure.
  • The Analogy: Think of a medical record as a long movie. A standard AI might only remember the last 5 minutes. This custom AI is like a director who can watch the entire 3-hour movie, remember the opening scene, and understand how the plot twists in the middle connect to the ending. It uses a "hierarchical" approach, meaning it looks at small details (words) and big pictures (the whole visit) simultaneously.

4. The Showdown: Who Won?

The results were clear:

• The Custom Architect (Hierarchical Transformer) won hands down. It achieved the highest accuracy.
• Why? Because it was specifically built to handle long, complex medical stories. It could connect a mention of "high blood pressure" in a note from 2018 with a new symptom in 2023, creating a complete picture of the patient's health.
• The LLMs (The Genies) struggled because they weren't trained specifically on Dutch medical guidelines and couldn't handle the massive amount of text without getting confused.

5. The "Late Fusion" Secret Sauce

The researchers also tried a trick called Late Fusion.

• The Analogy: Imagine you are judging a cooking contest. You have the chef's written recipe (the text) and you also have the actual ingredients on the table (the numbers like age and BMI).
• Instead of mixing the ingredients into the recipe before reading, they let the AI read the recipe first, then looked at the ingredients, and then combined both opinions to make the final decision. This made the system even more robust.

6. Why This Matters (The "Learning Healthcare System")

The ultimate goal is to create a Learning Healthcare System.

• Old Way: A nurse manually sorts files. If she makes a mistake, the system doesn't learn from it.
• New Way: The AI automatically sorts thousands of files every day, 100% of the time, never getting tired. It learns from the data to get better.
• Privacy: Unlike the "Genie" models that often need to send data to the cloud (which is risky for patient privacy), this custom model can run right inside the hospital's secure computer, keeping patient data safe.

The Bottom Line

This paper proves that you don't need a generic, massive AI to solve medical problems. Instead, building a specialized, custom AI that understands the unique structure of long medical stories is the key to saving lives. It turns a mountain of messy paperwork into a clear, actionable plan to protect elderly patients from heart disease.

Technical Summary

Here is a detailed technical summary of the paper "Automatic Cardiac Risk Management Classification using large-context Electronic Patients Health Records."

1. Problem Statement

The study addresses the limitations of manual administrative coding in Geriatric Cardiovascular Risk Management (CVRM).

• Current Workflow: Eligibility for CVRM programs at the University Medical Center Utrecht (UMCU) is determined by nurses manually assigning "agenda codes" based on appointment types. This process is prone to selection errors, ignores physiological parameters and clinical history, and fails to capture the complex, synergistic risk factors (comorbidities, polypharmacy, frailty) typical of the elderly.
• Data Challenge: Electronic Health Records (EHRs) contain vast amounts of unstructured data (clinical narratives, consultation reports) alongside structured data (medications, demographics). Extracting actionable risk insights from these long-context, longitudinal texts is difficult for traditional methods.
• Goal: To develop an automated framework that accurately classifies patients as "eligible" or "non-eligible" for CVRM by leveraging the full spectrum of unstructured and structured EHR data, thereby supporting a Learning Healthcare System (LHS).

2. Methodology

The authors utilized a dataset of 3,482 geriatric patients from UMCU. The data was categorized into Class 0 (non-eligible, 80.6%) and Class 1 (eligible, 19.4%), presenting a significant class imbalance.

Data Processing

• Unstructured Text: Consultation reports (consultverslagen) were collected, containing multidimensional evaluations (symptoms, cognitive status, history).
• Medications: Raw drug names were mapped to Anatomical Therapeutic Chemical (ATC) codes. Descriptions of these codes were converted into 768-dimensional semantic embeddings using the BioLORD-2023 (Sentence-Transformer) model.
• Structured Data: Age and gender were extracted as anthropometric features.
• Input Strategy: Two strategies were compared:
  1. Text-Only: Using only consultation narratives.
  2. Late Fusion: Concatenating text embeddings with structured vectors (medication embeddings + anthropometrics) before the final classification layer.

Model Architectures

The study benchmarked three distinct modeling paradigms:

1. Traditional Machine Learning:
   • LinearSVC: Trained on TF-IDF representations.
   • Dummy Classifier: Stratified baseline.
2. Deep Learning (Custom):
   • ResNet-1D: A custom 1D convolutional network for sequential text.
   • Hierarchical Transformer (H-Transformer): A custom encoder-only architecture designed for large-context sequences (up to 8,192 tokens).
     • Key Features: Uses Rotary Positional Embeddings (RoPE) and Hierarchical Self-Attention to efficiently process long medical texts by prioritizing local dependencies while capturing long-range dependencies.
     • Aggregation: Tested two pooling strategies: CLS Token (Classification token) and Global Average Pooling.
3. Generative Large Language Models (LLMs):
   • Zero-Shot Setting: GPT-4o, GPT-4o-mini, GPT-4.1, and GPT-4.1-mini.
   • Prompting: A two-step pipeline: (1) Translate Dutch consults to English; (2) Extract binary eligibility based on a summarized extract of the 400+ page CVRM guidelines.

Training & Optimization

• Optimizer: AMSGrad.
• Validation: Stratified 5-fold cross-validation to handle class imbalance.
• Privacy: All patient data was de-identified using DEDUCE. LLMs were queried via secure Azure environments.

3. Key Contributions

1. Automated CVRM Framework: Developed a robust pipeline to transform highly unstructured, longitudinal geriatric EHRs into a structured risk stratification tool.
2. Custom Hierarchical Transformer: Introduced a specialized Transformer architecture capable of processing 8k-token contexts with hierarchical attention, outperforming both standard CNNs and general-purpose LLMs in this specific domain.
3. Multimodal Late Fusion: Demonstrated a strategy to effectively integrate unstructured clinical narratives with structured medication embeddings and demographics without requiring complex internal architectural changes to the core sequence model.
4. Benchmarking LLMs: Provided empirical evidence that zero-shot general-purpose LLMs (even with guideline summaries) underperform compared to specialized, supervised models for non-English clinical tasks.

4. Results

The Hierarchical Transformer consistently outperformed all other models across both text-only and late-fusion settings.

• Best Performance: The H-Transformer with CLS Token pooling (Late Fusion) achieved:
  • F1-Score: 92.48%
  • Matthews Correlation Coefficient (MCC): 0.758
  • Precision: 92.48%
  • Recall: 92.48%
• Comparison with Baselines:
  • LinearSVC: Strong performance (F1 ~86%), confirming the utility of traditional methods but falling short of deep learning.
  • ResNet-1D: Significantly lower performance (F1 ~85%), suggesting CNNs struggle with the long-range dependencies in medical text.
  • Zero-Shot LLMs: Performed poorly (F1 ~33-35%, MCC ~0.12). Despite high recall, precision was very low, indicating they lack the specific domain nuance required without fine-tuning.
• Impact of Multimodal Fusion: Adding structured data (medications/anthropometrics) via late fusion slightly improved or maintained the performance of the best Transformer models, validating the integration of diverse data modalities.

5. Significance and Conclusion

• Operational Efficiency: The proposed system offers a scalable, automated alternative to manual nurse coding, reducing selection errors and enabling continuous risk monitoring within a Learning Healthcare System.
• Privacy & Feasibility: The study highlights a critical trade-off: while generative LLMs are powerful, they are often computationally prohibitive and privacy-risky to fine-tune on-premise. The custom encoder-only Transformer offers a pragmatic solution that is computationally efficient, privacy-compliant (on-premise), and highly accurate.
• Clinical Insight: The success of the Hierarchical Transformer underscores the necessity of hierarchical attention mechanisms to capture the complex, multi-morbid clinical histories of geriatric patients, which standard models often miss.
• Future Direction: The authors suggest that while domain-specific fine-tuned LLMs are a future avenue, specialized encoder architectures remain the most viable solution for reliable, real-time risk stratification in current hospital infrastructures.