💬 NLP

Dementia-R1: Reinforced Pretraining and Reasoning from Unstructured Clinical Notes for Real-World Dementia Prognosis

Dementia-R1 is a reinforcement learning framework that leverages cold-start pretraining on verifiable clinical indices to enable large language models to effectively reason over complex, non-monotonic symptom trajectories for accurate longitudinal dementia prognosis from unstructured clinical notes.

Original authors: Choonghan Kim, Hyunmin Hwang, Hangeol Chang, Jaemin Kim, Jinse Park, Jae-Sung Lim, Jong Chul Ye

Published 2026-03-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Choonghan Kim, Hyunmin Hwang, Hangeol Chang, Jaemin Kim, Jinse Park, Jae-Sung Lim, Jong Chul Ye

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future of a patient's health, specifically whether they will develop dementia. You have a massive stack of their medical notes—thousands of pages of handwritten or typed observations from doctors over many years. These notes are messy, unstructured, and full of stories about how the patient felt, what they forgot, and how they walked.

The problem is that standard computer programs (and even advanced AI) are terrible at reading this "messy story" and figuring out the trend. They are good at answering a single question like "What is the patient's blood pressure?" but bad at answering "How has their memory changed over the last three years, and where is it heading?"

This paper introduces Dementia-R1, a new AI system designed to be a "medical detective" that can read these long, messy stories and predict the future with high accuracy.

Here is how it works, explained through simple analogies:

1. The Problem: The "Binary Trap"

Usually, when we train AI to predict disease, we give it a simple "Yes/No" reward.

The Old Way: You show the AI a patient's history and ask, "Will they get dementia?" If the AI guesses "Yes" and they do, it gets a gold star. If it guesses "No," it gets nothing.
The Issue: This is like trying to learn to drive a car by only getting a reward when you successfully park. You never learn how to steer, brake, or turn. The AI doesn't learn the process of reasoning; it just guesses the final answer. Because the "Yes/No" answer is so far away in the future, the AI gets confused and makes wild guesses.

2. The Solution: The "Training Wheels" (Cold-Start RL)

The authors realized that to predict the final destination (Dementia), the AI first needs to learn how to read the road signs along the way.

They introduced a Two-Stage Training Process:

Stage 1: The "Score Predictor" (Learning to Read the Signs)

Before asking the AI "Will they get dementia?", they first ask it to predict specific, measurable numbers found in the notes, like the MMSE (a memory test score) or the GDS (a scale of how bad the dementia is).

The Analogy: Imagine teaching a student to predict the final exam grade. Instead of just guessing the grade, you first teach them to predict the score on the next quiz based on their homework.
The Magic: The AI is rewarded for getting these intermediate numbers right. This forces the AI to learn how to track changes over time. It learns: "Oh, if the MMSE dropped from 24 to 21 over six months, that's a bad sign. If it dropped to 17, that's a crisis."
The Result: The AI builds a "reasoning muscle." It learns to connect the dots between visits, understanding the trajectory of the disease, not just the final snapshot.

Stage 2: The "Prognosis Expert" (The Final Prediction)

Now that the AI has learned how to track the disease progression (the "training wheels" are on), they ask it the hard question: "Will this patient develop dementia?"

The Analogy: Now that the student knows how to analyze quiz scores and homework trends, they are ready to predict the final exam grade with much higher confidence.
The Reward: Even though the final "Yes/No" answer is still a simple binary reward, the AI uses the reasoning skills it learned in Stage 1 to get there. It doesn't just guess; it reasons through the timeline.

3. The Results: Beating the Giants

The researchers tested this "Dementia-R1" system (which is relatively small, about 7 billion parameters) against massive AI models (some 10 times larger, like 70B or 72B parameters).

The Surprise: The small, specially trained Dementia-R1 beat the giant models.
Why? The giant models were like general-purpose libraries; they knew a lot of facts but didn't know how to "think" about long-term medical trends. Dementia-R1 was like a specialized detective who had practiced reading medical stories specifically for this job.
Real-World Proof: It worked on real hospital data from Korea (Asan Medical Center) and even generalized to a different disease (Parkinson's) and a different hospital, proving it wasn't just memorizing the data.

4. Why This Matters

Think of the current state of medical AI as a student who has memorized the answer key but doesn't understand the math. Dementia-R1 is a student who actually learned the math.

By forcing the AI to practice predicting intermediate steps (like memory scores) before making the final diagnosis, the system learned to "think" about time and change. This is crucial for diseases like dementia, which are defined by how a patient changes over months and years, not just by how they look on a single day.

In short: Dementia-R1 is a smarter, more efficient AI that learns to predict the future of a patient's health by first mastering the art of reading their medical history like a story, rather than just scanning it for keywords.

1. Problem Statement

The paper addresses the critical challenge of longitudinal dementia prognosis using unstructured Electronic Health Records (EHRs).

The Gap: While Large Language Models (LLMs) excel at static clinical text understanding, they struggle with longitudinal tasks that require reasoning over complex, non-monotonic symptom trajectories across multiple visits.
Data Challenge: Approximately 80% of EHR data is unstructured (clinical notes), making it difficult for traditional longitudinal models (designed for structured tables) to utilize rich behavioral and symptomatic descriptions.
Training Limitation: Standard Supervised Fine-Tuning (SFT) lacks explicit annotations for symptom evolution. Direct Reinforcement Learning (RL) for binary prognosis (Dementia vs. Non-Dementia) is hindered by sparse rewards (only a final binary label exists) and the lack of intermediate reasoning steps, leading to poor convergence and weak temporal reasoning capabilities.

2. Methodology: Dementia-R1 Framework

The authors propose Dementia-R1, a two-stage RL-based framework designed to instill explicit temporal reasoning capabilities in LLMs before tackling the final prognosis task. The framework utilizes Group Relative Policy Optimization (GRPO).

Core Strategy: Cold-Start RL with Verifiable Rewards

Instead of jumping directly to the final diagnosis, the model is first trained to predict intermediate, verifiable clinical indices. This acts as a "Cold-Start" to learn reasoning primitives.

Stage 1: Cold-Start Pre-training (Intermediate Index Prediction)

Objective: Predict specific clinical scores (MMSE, GDS, CDR) from historical unstructured notes.
Data Construction: An auxiliary LLM extracts verifiable clinical indices from raw notes to serve as ground truth. A human audit confirmed 98.5% extraction accuracy.
Reward Function ( $R_{cold}$ ): A tolerance-aware reward is used.
- For MMSE (0–30 scale), a tolerance of $\delta=2$ is applied (aligning with Minimal Detectable Change).
- For coarser scales (GDS, CDR), exact matching is required.
- $R_{cold} = 1$ if $|\hat{s}_t - s_t| \leq \delta$ , else $0$.
Optimization: The model is optimized using GRPO to maximize the probability of generating reasoning paths that yield correct index predictions, encouraging the model to learn disease progression patterns.

Stage 2: Task Fine-tuning (Binary Prognosis)

Objective: Predict the final binary clinical status (Dementia vs. Non-Dementia) based on the history.
Reward Function ( $R_{task}$ ): A sparse binary reward ($1$ for correct, $0$ for incorrect).
Mechanism: The reasoning primitives acquired in Stage 1 allow the model to effectively navigate the sparse reward landscape of Stage 2, enabling it to distinguish temporary fluctuations from persistent decline.

3. Key Contributions

Dementia-R1 Framework: A novel RL-based framework that enables explicit temporal reasoning on unstructured clinical notes for dementia prognosis.
Cold-Start RL Strategy: Introduction of a pre-training stage using verifiable intermediate clinical scores (MMSE, GDS, CDR) as rewards. The paper demonstrates that learning to estimate these intermediate states is crucial for accurate final prognosis, effectively solving the sparse reward problem in longitudinal tasks.
Cross-Disease and Cross-Cohort Generalization: The framework was validated on three distinct datasets:
- AMC: Real-world unstructured notes from Asan Medical Center (Korea).
- HPH: Independent hospital cohort for Parkinson's Disease Dementia (PDD) prediction, demonstrating cross-disease transfer.
- ADNI: Structured benchmark dataset for Alzheimer's Disease.
Superior Performance: The model achieves state-of-the-art results, outperforming general-purpose LLMs up to 10x larger (e.g., 7B model vs. 72B models) and specialized medical reasoning models.

4. Experimental Results

The model was evaluated using AUROC, AUPRC, F1-score, and Accuracy across five random seeds.

Asan Medical Center (AMC) Cohort:
- Dementia-R1 (7B) achieved an AUROC of 84.02% and F1 of 77.31%.
- It outperformed the strongest 72B baseline (Qwen2.5-72B, AUROC 83.41%) and all other specialized 7B baselines.
- It showed consistent gains over ablation variants (e.g., GRPO without Stage 1, SFT pipelines).
Haeundae Paik Hospital (PDD) Cohort:
- Demonstrated strong cross-disease generalization.
- Achieved AUROC of 78.37% and AUPRC of 51.03%, outperforming the 72B baseline (72.37% AUROC).
ADNI Benchmark (Structured Data):
- Achieved the highest AUROC (83.17%) among all LLM baselines, surpassing larger models like Qwen2.5-32B and Llama-3.3-70B.
- Showed particular strength in long-term prediction horizons (>36 months), where other models degraded significantly.
Temporal Robustness:
- Dementia-R1 maintained high performance across varying prediction horizons (6m to 36m+), whereas baselines struggled with longer time gaps.
- Ablation studies confirmed that Stage 1 pre-training density correlates positively with downstream performance.
Human Evaluation:
- A blinded evaluation by two board-certified neurologists showed Dementia-R1 achieved a 68% win rate in "Overall Clinical Utility" and 95% win/tie rate in "Temporal Reasoning Accuracy" compared to Qwen2.5-32B.
- Qualitative analysis revealed that baselines often hallucinated medical facts (e.g., misidentifying drug mechanisms) or confused scales, while Dementia-R1 provided structured, grounded reasoning.

5. Significance and Implications

Bridging the Gap: The paper successfully bridges the gap between the unstructured nature of real-world clinical notes and the rigorous requirements of longitudinal medical prediction.
Efficiency: It proves that a smaller, well-aligned model (7B) using a specific RL strategy can outperform massive general-purpose models (70B+) in specialized medical tasks, making deployment more feasible.
Reasoning over Memorization: By forcing the model to predict intermediate clinical scores, the framework shifts the learning objective from simple pattern matching to genuine longitudinal reasoning, allowing the model to understand disease trajectories rather than just static snapshots.
Clinical Utility: The framework's ability to generalize across different diseases (Alzheimer's to Parkinson's) and data modalities (unstructured notes to structured tables) suggests a robust pathway for AI-assisted clinical decision support in neurology.

Limitations Noted

Data Demographics: Validation was limited to Korean hospital cohorts and the ADNI dataset, potentially limiting generalizability to other ethnicities or languages.
Translation Artifacts: The pipeline involved translating Korean notes to English, which may lose subtle linguistic nuances important for cognitive assessment.
Dependency on Extraction: The reward mechanism relies on the accuracy of the auxiliary LLM for extracting clinical scores, though the 98.5% audit accuracy suggests this noise is minimal.