🤖 machine learning

Predicting one-year clinical instability and mortality in heart failure patients using sequence modeling

This paper demonstrates that autoregressive sequence models, such as Llama and Mamba, can accurately predict one-year clinical instability and mortality in heart failure patients using routine electronic health record data, providing a framework for risk stratification and personalized discharge planning.

Original authors: Falk Dippel, Yinan Yu, Annika Rosengren, Martin Lindgren, Christina E. Lundberg, Erik Aerts, Martin Adiels, Helen Sjöland

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Falk Dippel, Yinan Yu, Annika Rosengren, Martin Lindgren, Christina E. Lundberg, Erik Aerts, Martin Adiels, Helen Sjöland

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Heart Failure "Crystal Ball": Predicting the Future with AI

Imagine you are a doctor standing at the hospital exit. A patient with heart failure is about to be discharged. You have a heavy responsibility: How much help does this person need once they get home?

If you send them home with just a standard check-up, but they are actually at high risk of a sudden collapse, it’s a disaster. If you assign them a team of intensive nurses, but they were actually perfectly fine, you’ve wasted precious medical resources.

Currently, doctors rely on their "gut feeling" and experience. This paper describes a new way to use AI to turn that "gut feeling" into a high-tech, data-driven "crystal ball."

The Problem: The "Messy Diary" of Health

Every patient has a medical history, but it isn't a neat, organized book. It’s more like a messy, multi-year diary written by different people (nurses, doctors, lab techs) at different times.

One day, there’s a long entry about blood pressure.
The next month, there’s a tiny note about a single medication.
Then, a huge gap of silence.

Traditional computers struggle with this "messy diary" because they like things to be predictable and evenly spaced. They get "confused" by the gaps and the irregular notes.

The Solution: The "Super-Reader" (Sequence Modeling)

The researchers decided to stop treating medical data like a spreadsheet and start treating it like a story. They used advanced AI models (specifically ones called Transformers and Mambas) that are designed to read sequences—just like how your phone predicts the next word in a text message.

Think of these AI models as Super-Readers. Instead of just looking at a single blood test result in isolation, the Super-Reader reads the entire story of the patient. It notices that a slight change in heart rate three months ago, combined with a specific medication taken yesterday, creates a pattern that signals trouble is coming.

How They Did It: The Three-Part Recipe

To make this work, they built a "modular pipeline"—think of it like a professional kitchen:

The Chopping Station (Tokenization): They took raw data (like "Blood Pressure: 140/90") and chopped it into "tokens"—the smallest meaningful bits of information that the AI can digest.
The Slow-Cooker (Temporal Representation): They decided how much of the "past" to include. Should the AI look at the last 10 years, or just the last 10 months? They found that focusing on the most recent "chapters" of the story worked best.
The Master Chef (The Model): They tested different "chefs" (AI architectures). One chef, named Llama, turned out to be the best at reading the patient's story and predicting what would happen next.

The Results: Four Paths to Care

The AI didn't just give a "yes/no" answer. It helped categorize patients into four distinct pathways, much like a GPS rerouting a car based on traffic:

The "Smooth Sailing" Path: Low risk of instability and low risk of death $\rightarrow$ Standard Primary Care.
The "Watchful Eye" Path: High risk of returning to the hospital $\rightarrow$ Specialized Monitoring.
The "Gentle Support" Path: Low risk of instability but high risk of death $\rightarrow$ Regular Home Care/Palliative Care.
The "Red Alert" Path: High risk of both $\rightarrow$ Intensive Home Care.

Why This Matters (The "So What?")

The researchers proved that even "tiny" versions of these AI models (which are faster and cheaper to run) can outperform much larger, older systems.

By using this AI, hospitals can move away from "one-size-fits-all" medicine and toward personalized care. It ensures that the right patients get the right amount of help at exactly the right time, potentially saving lives and making sure hospital beds are available for those who truly need them most.

Technical Summary: Predicting One-Year Clinical Instability and Mortality in Heart Failure Patients Using Sequence Modeling

1. Problem Statement

Heart failure (HF) is a chronic, progressive condition characterized by high rehospitalization rates and mortality. A critical clinical challenge during HF discharge is risk stratification: determining which patients require specialized monitoring, intensive home care, or palliative care versus those who can be managed in primary care.

Current methods rely heavily on clinical judgment or simple scoring systems, which often lack the precision needed for individualized care. While Artificial Intelligence (AI) offers potential, most existing models are trained on general, heterogeneous patient populations. There is a lack of research into disease-specific models that leverage the unique longitudinal trajectories and specific clinical concepts (e.g., specific laboratory markers or medications) inherent to heart failure.

2. Methodology

The researchers developed a modular three-component framework to transform structured Electronic Health Records (EHRs) into patient sequences for prediction.

Cohort & Tasks: The study used a Swedish HF cohort ( $N = 42,820$ $N = 42, 820$ ). They defined three binary classification tasks for one-year post-discharge prediction:
1. T1: Clinical instability (unplanned rehospitalization) after the initial HF diagnosis.
2. T2: Mortality after the initial HF diagnosis.
3. T3: Mortality after the latest (most recent) hospitalization.
Modular Framework:
- Component 1 (Tokenization): Clinical concepts (diagnoses, vitals, labs, medications, procedures) were tokenized. Continuous values were discretized into bins, and categorical codes (ICD-10, ATC, KVÅ) were handled via hierarchical levels.
- Component 2 (Temporal Representation): The study explored different "observation horizons" (how far back to look) and "temporal aggregation" (summarizing events within a window, e.g., 1-day aggregation) to manage the irregular sampling of EHRs.
- Component 3 (Sequence Models): They compared six architectures across three classes:
  - Transformers: BERT, XLNet.
  - Transformers++: ModernBERT, Llama.
  - Mamba (State Space Models): Mamba, Mamba2.
Training Objective: They specifically tested Autoregressive Next Token Prediction (NTP) against Masked Language Modeling (MLM) and Permutation Language Modeling (PLM).

3. Key Contributions

Clinically Grounded Framework: Instead of providing a single risk score, the study links predictions to four distinct care pathways (Primary Care, Specialized Care, Regular Home Care, and Intensive Home Care), making the AI output actionable for discharge planning.
Modular Design Analysis: Rather than just comparing end-to-end models, the authors used controlled ablation studies to identify how tokenization, temporal aggregation, and model size independently affect performance.
Efficiency Insights: The study demonstrates that for short-context clinical settings ( $\leq 512$ tokens), advanced "tiny" configurations of modern models can outperform much larger conventional baselines.

4. Results

Superiority of NTP: Autoregressive Next Token Prediction (used by Llama and Mamba) consistently outperformed MLM (BERT) and PLM (XLNet) across all tasks.
Top Performer: Llama (an advanced Transformer++) achieved the best performance, with AUPRCs of 0.555 (T1), 0.582 (T2), and 0.854 (T3). It showed robust calibration and high discriminative power.
Model Scaling: Performance gains were driven more by context length (data quantity) than by model size (parameter count). Tiny configurations of Llama and Mamba often surpassed larger BERT-based models.
Temporal Aggregation: For mortality tasks (T2, T3), daily event aggregation (summarizing events within a 1-day window) improved performance, suggesting that temporal density is more important than raw event count.
Data Scalability: Llama demonstrated high training efficiency, reaching competitive performance using only 75% of the available training data. Furthermore, concept diversity (including labs and medications) was a stronger driver of performance than simply increasing the volume of training data.

5. Significance

This research provides a blueprint for developing disease-specific AI in healthcare. By proving that sequence models can accurately predict complex outcomes like clinical instability and mortality from routine hospital data, the study moves AI from a theoretical tool to a practical decision-support system.

The findings suggest that clinicians can use these models to transition from "one-size-fits-all" discharge planning to patient-centered, data-driven care, ensuring high-risk patients receive intensive support while optimizing resources for low-risk patients.