RarePathAI: Predictive Analytics to Identify Rare Diseases in Real-World Data

Technical Summary: RarePathAI

Problem Statement
Rare diseases (RDs) affect millions of individuals in the United States, yet patients frequently endure a "diagnostic odyssey" characterized by prolonged delays, misdiagnoses, and a lack of clinician familiarity. A primary barrier to timely diagnosis is the limitation of existing healthcare data and coding practices. Over half of known RDs lack specific International Classification of Diseases (ICD-10) codes, forcing reliance on nonspecific diagnostic codes that obscure patient phenotypes. Furthermore, integrating standardized, granular phenotype terminologies (such as the Human Phenotype Ontology, HPO) with real-world data (RWD) from administrative claims remains technically challenging. Traditional machine learning approaches often fail to capture the complex, temporal, and interdependent relationships inherent in patient journeys, particularly when symptom patterns are heterogeneous.

Methodology
The authors present RarePathAI, a framework integrating U.S. administrative claims data, rare disease ontologies, and Graph Neural Networks (GNNs) to support earlier diagnosis.

• Data Ecosystem: The system utilizes the Symphony Integrated Dataverse (~320 million unique individuals) covering July 2018 to June 2025. This is combined with reference data from Orphanet (disease nomenclature and epidemiology) and the Human Phenotype Ontology (HPO).
• Cohort Construction: The study focused on 98 curated rare disease cohorts, each with a prevalence of 1–50 per 100,000 in the U.S. and associated with at least three commonly observed clinical symptoms. Patient cohorts required at least one year of pre-diagnosis claims history and three clinical encounters. Diagnosis dates were established using unambiguous ICD-10 codes (requiring two occurrences 60 days apart) or disease-specific treatments.
• Graph Schema: A graph database was constructed where patients are represented as subgraphs.
  • Nodes: Derived from RWD (Patient, Encounter, Diagnosis, Procedure, Specialty, Age Range) and Reference Data (Rare Disease, HPO Term, Significant Age Triplets).
  • Embeddings: Reference data nodes were enriched with vector representations. Text descriptions were processed using BioBERT to generate 768-dimensional embeddings. Structural relationships within the reference network were encoded using ProNE to generate 512-dimensional embeddings.
  • Age Triplets: To address the lack of age-specific symptom data in reference ontologies, the authors used Claude 3.5 (LLM) to generate "age triplet" nodes (Age-ICD-RD), linking specific symptoms to typical age ranges of manifestation.
• Model Architecture: A two-step stacked predictive model was developed:
  1. Binary Classifier: Determines if a patient is likely to have any of the 98 target rare diseases versus a general population control.
  2. Multiclass Classifier: For patients flagged as positive, this model predicts the likelihood of specific diagnoses across the 98 classes.
  • The GNN utilizes a PyTorch implementation with GraphConv layers, featuring nine message-passing layers with 256 channels, Tanh activation, and mean aggregation.
  • Feature importance was assessed via perturbation analysis (removing nodes to measure score changes).

Key Results

• Scale: The final graph contained over 40 million nodes and more than 350 million edges.
• Binary Classification: The model achieved a Receiver Operating Characteristic Area Under the Curve (ROC-AUC) of 0.951. At a higher decision threshold, the model maintained a True Positive Rate of 84% while reducing the False Positive Rate to 5%.
• Multiclass Classification: The model generated ranked lists of likely diagnoses.
  • Top 10 Weighted Recall: 74.8% (meaning the correct diagnosis appeared in the top 10 suggestions for 74.8% of patients).
  • Top 15 Weighted Recall: 80.3%.
  • Top 20 Weighted Recall: 84.2%.
• Feature Importance: Perturbation analysis confirmed that nodes associated with key clinical signs (e.g., ataxia for Friedreich ataxia, developmental delays for Prader-Willi syndrome) were the primary drivers of model predictions.
• Data Alignment: The study observed that while HPO "Very frequent" symptoms were expected in nearly all patients, only 6.2% were observed in pre-diagnosis claims data, highlighting the limitations of claims-based matching and the necessity of the model's ability to infer patterns from sparse data.

Significance and Claims
The paper claims that RarePathAI offers a scalable, interpretable method to reduce diagnostic delays by moving rare disease detection from isolated research settings into routine clinical practice. Key contributions include:

1. Graph-Based Learning: Demonstrating that GNNs can capture complex, latent relationships between symptoms, procedures, and diseases that traditional models miss, without extensive manual feature engineering.
2. Ontology Integration: Successfully bridging the gap between unstructured claims data and structured ontologies (Orphanet/HPO) using embedding techniques and LLM-augmented age-specific data.
3. Clinical Utility: The two-step approach (screening followed by specific ranking) aligns with clinical reasoning, providing a ranked differential diagnosis list to narrow the search space for clinicians.
4. Future Vision: The authors propose an "AI-Agentic orchestration layer" for future deployment. This system would not only provide real-time diagnostic suggestions but also coordinate specialized AI agents to generate explainable insights and interactive guidance, potentially directing timely targeted testing (including genetic workups).

The authors emphasize that while the model shows strong predictive performance, clinical validation across diverse populations and integration into Electronic Medical Record (EMR) systems are essential next steps to ensure reliability and safety in real-world settings. They acknowledge limitations, including the granularity of claims data and inevitable semantic mismatches between HPO terms and ICD-10 codes.