Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processing

Technical Summary: Early Risk Prediction with Temporally and Contextually Grounded Clinical Language Processing

Problem Statement
The paper addresses the challenge of predicting the risk of chronic diseases, specifically Type 2 Diabetes (T2D), using longitudinal Electronic Health Records (EHRs). While clinical notes contain rich temporal information regarding events, clinician reasoning, and lifestyle factors often missing from structured data, they present significant Natural Language Processing (NLP) hurdles. These include long text lengths, irregular event distributions, complex temporal dependencies, privacy constraints, and resource limitations in real-world healthcare settings. Existing approaches often rely on structured data (which can be noisy and incomplete) or treat clinical notes as static snapshots, failing to model the fine-grained temporal relations within a single note and the dynamics across multiple patient visits. Furthermore, while Large Language Models (LLMs) offer strong semantic understanding, they struggle with long-range context integration, structured reasoning, and are often too computationally expensive or privacy-restrictive for deployment in resource-constrained clinical environments.

Methodology
The authors propose two complementary methods to address these challenges:

1. HITGNN (Hierarchical Temporal Graph Neural Network):

   • Concept: A dynamic model designed to capture patient state evolution with fine-grained temporal granularity and medical knowledge augmentation.
   • Pipeline:
     • Temporal Relation Extraction (TREX): Uses state-of-the-art models (SPANTREX) to extract clinical entities (problems, treatments, tests) and time expressions from individual notes, identifying temporal relations (Before, After, Overlap).
     • Graph Construction: Constructs intra-note temporal graphs where nodes are entities and edges are temporal relations. These are augmented with a clinical knowledge graph (UMLS) to link entities to semantic types and concepts.
     • Normalization: Applies entity linking (via MetaMap to UMLS) and temporal graph denoising (via TimeGraph algorithm) to ensure consistency.
     • Modeling: Uses a GraphSAGE encoder to process intra-visit graphs, followed by a Bidirectional LSTM (BiLSTM) to model inter-visit dependencies across the patient's timeline. Node representations combine contextual embeddings (BioMedBERT) with knowledge graph embeddings (KG-embeddings).
   • Goal: To model patient trajectories by integrating intra-document temporal structures, inter-visit dynamics, and medical knowledge.

2. REVEAL (Reasoning with Verifier-Aided Labeling):

   • Concept: A lightweight, test-time framework that distills the reasoning capabilities of large LLMs into smaller, more efficient models suitable for resource-constrained settings.
   • Pipeline:
     • Reasoner: A larger LLM (e.g., Llama3.1-8B) generates multiple stochastic reasoning paths (predictions and explanations) for a patient's risk.
     • Verifier: A smaller, fine-tuned LLM (e.g., Llama3.2-1B with LoRA) evaluates the credibility of these paths, assigning a confidence score.
     • Aggregation: The final prediction is determined by aggregating the top-k highest-confidence predictions (via majority vote).
   • Goal: To preserve the interpretability and reasoning quality of large models while scaling performance without the full computational cost of training or deploying massive models.

Key Contributions

1. Novel Frameworks: Introduction of HITGNN, the first application of clinical temporal relation extraction (TREX) for risk prediction that integrates intra-document relations, inter-visit dynamics, and medical knowledge; and REVEAL, a test-time scaling framework for interpretable LLM inference.
2. Real-World Application: Demonstration of these methods in opportunistic T2D screening, specifically targeting immediate-risk horizons where intervention is most impactful.
3. Rigorous Data Curation: Creation of datasets from private (PH) and public (MIMIC-IV) sources with strict exclusion of post-diagnosis data to prevent label leakage and demographic matching for fair evaluation.
4. Comprehensive Ablations: Extensive analysis disentangling the impacts of temporal relations, knowledge graph augmentation, and pipeline components, revealing corpus-dependent trade-offs.
5. Fairness Analysis: An evaluation of model behavior across demographic subgroups, highlighting instability in LLM-based systems compared to the more equitable performance of HITGNN.

Results

• Predictive Performance: HITGNN achieved the highest predictive accuracy, particularly for near-term risk (≤1.5 years), outperforming both LLM-based baselines (including zero-shot and fine-tuned Llama variants) and structured baselines (CLSTM). On the PH corpus, HITGNN achieved an AUC of 72.24%, significantly outperforming the next best model (CLSTM at 68.24%).
• LLM Performance: While fine-tuned LLMs (Llama3.2-1B-ft) showed strong classification performance, they lacked reasoning capabilities. REVEAL improved upon zero-shot LLMs in terms of T2D recall but did not significantly outperform the fine-tuned classifier, though it retained explanatory reasoning.
• Efficiency: HITGNN and CLSTM demonstrated superior computational efficiency, with training times of ~1.5 and ~12 minutes respectively, and inference times of ~0.01 and ~0.02 seconds per patient. In contrast, LLM-based approaches required significantly more time and memory (e.g., REVEAL inference took ~62 seconds per patient).
• Fairness: HITGNN generally performed more equitably across subgroups (gender, race, ethnicity) without explicitly encoding demographic attributes. Conversely, LLM-based models exhibited instability and inconsistency, with performance heavily dependent on demographic conditioning, sometimes relying on spurious demographic shortcuts.
• Ablations: The study found that combining text and knowledge graph embeddings yielded the best results on the private corpus, while temporal edges improved T2D recall on the private data but had mixed effects on the public MIMIC-IV dataset.

Significance and Claims
The paper claims to establish a clinically grounded framework that integrates concept abstraction, temporal structure, and semantic enrichment to support low-cost and privacy-conscious AI systems for real-world clinical decision support. The authors emphasize that their work moves beyond static snapshots to dynamic patient representations, addressing the specific needs of opportunistic screening where patients may miss routine care. They assert that HITGNN offers a robust, efficient, and fair alternative to large proprietary models, while REVEAL provides a pathway to leverage LLM reasoning without prohibitive resource costs. The work highlights the importance of temporal structure in diagnostic reasoning and cautions against the uncritical use of demographic conditioning in LLMs due to potential reliance on unstable population-level shortcuts rather than individual clinical evidence. The authors position their models as "clinician-in-the-loop" tools for risk stratification, not as standalone diagnostic devices.