RD-Embed: Unified representations of rare-disease knowledge from clinical records

RD-Embed is a lightweight, three-stage representation framework that unifies clinical text and coded signals to significantly improve rare-disease retrieval and diagnosis from heterogeneous electronic health records compared to existing models.

The Gist

Imagine you are a detective trying to solve a mystery, but the clues are scattered, messy, and written in different languages. Some clues are on official police forms (structured data), while others are scribbled in a messy notebook (unstructured text). This is exactly the challenge doctors face when trying to diagnose rare diseases.

Here is a simple breakdown of the paper "RD-Embed" using everyday analogies.

The Problem: The "Diagnostic Odyssey"

Rare diseases are like needles in a haystack. There are thousands of them, and they often look different in every patient.

• The Messy Reality: A doctor's notes are a mix of official codes (like "SNOMED" or "HPO") and free-flowing stories ("The patient seems tired and has a weird rash").
• The Old Tools: Previous computer tools were like rigid librarians. They only understood the official codes. If a doctor wrote a story without the exact code, the tool would say, "I don't know what this is," and fail to help.
• The New AI: Big AI models (like the ones you chat with) are like general encyclopedia experts. They know a lot about medicine, but they haven't studied the specific, obscure "rulebook" of rare diseases deeply enough. They often guess wrong when the clues are vague.

The Solution: RD-Embed (The "Universal Translator")

The authors created a new tool called RD-Embed. Think of it as a super-smart translator that can understand both the "official language" of medical codes and the "messy language" of doctor's notes, and then translate them both into a single, shared language.

They built this translator in three stages, like training a new employee:

Stage 1: Learning the Rulebook (Ontology Preservation)

• The Analogy: Imagine a student memorizing a massive, perfect dictionary of rare diseases, genes, and symptoms. They learn exactly how these things are supposed to relate to each other in a perfect world.
• What it does: It builds a solid foundation so the computer knows the "correct" relationships between diseases and symptoms, even before it sees a real patient.

Stage 2: The Field Trip (Clinical Alignment)

• The Analogy: Now, the student goes into a real hospital. They see that doctors don't always use the perfect dictionary words. They use slang, abbreviations, and incomplete sentences.
• What it does: This stage teaches the computer to listen to real doctor's notes and messy hospital records. It learns to say, "Ah, when the doctor writes 'tired and pale,' they probably mean the same thing as the official code 'anemia'." It bridges the gap between the perfect rulebook and the messy reality.

Stage 3: The Detective's Map (Graph Refinement)

• The Analogy: The student now draws a giant map connecting all the dots. They see that Disease A often leads to Symptom B, which is caused by Gene C. They use this map to fill in the blanks.
• What it does: It uses a "knowledge graph" to look at the big picture. If a patient has a few symptoms, the map helps the computer guess the missing pieces and find the most likely disease, even if the information is incomplete.

Why is this a Big Deal?

The paper tested RD-Embed against other tools and found some amazing results:

1. It works with messy notes: Even if a doctor only writes a paragraph of text without any official codes, RD-Embed can still find the right rare disease about 50% of the time (putting the right answer in the top 10 guesses). Other tools often failed completely in this scenario.
2. It's better than giant AI: Surprisingly, this specialized, lightweight tool performed better than massive, general-purpose AI models (like GPT) on these specific rare disease tasks. It's like a specialized mechanic fixing a Ferrari better than a general handyman who knows a little about everything.
3. It helps find the "Gene Needle": It doesn't just guess the disease; it helps narrow down which gene is broken, which is crucial for genetic testing.

The Bottom Line

RD-Embed is a bridge. It connects the clean, organized world of medical databases with the messy, real-world world of hospital notes.

Instead of forcing doctors to spend hours converting their notes into perfect codes, this tool lets them just type what they see. It then instantly searches through thousands of rare diseases to say, "Based on what you wrote, here are the top 10 possibilities we should check."

This could save patients years of uncertainty, turning a "diagnostic odyssey" (a long, confusing journey) into a much shorter, clearer path to getting the right treatment.

Technical Summary

1. Problem Statement

Rare disease diagnosis is hindered by the "diagnostic odyssey," where patients suffer from years of uncertainty due to incomplete, evolving, and heterogeneous clinical presentations. Key challenges include:

• Data Fragmentation: Clinical evidence is scattered across unstructured free-text notes, partial phenotype descriptions, and inconsistent disease coding (e.g., SNOMED CT) within Electronic Health Records (EHRs).
• Limitations of Existing Tools: Current phenotype-driven tools (e.g., LIRICAL, Phen2Gene) rely strictly on structured Human Phenotype Ontology (HPO) profiles. They degrade significantly when phenotypes are missing, noisy, or expressed only in narrative text.
• Limitations of General LLMs: While Large Language Models (LLMs) excel at general medical text tasks, they lack alignment with rare-disease-specific knowledge structures. They often fail to retrieve plausible diagnoses from sparse phenotypes because rare diseases are underrepresented in their training data and the reasoning requires subtle genotype-phenotype relationships not explicitly grounded in curated ontologies.
• The Gap: There is a lack of a unified representation that bridges structured ontologies (HPO, OMIM, Orphanet) with unstructured clinical text and codes, robust to missing data and capable of supporting gene prioritization.

2. Methodology: RD-Embed Framework

RD-Embed is a three-stage representation learning framework designed to create a shared embedding space for diseases, genes, phenotypes, and clinical descriptions. The model outputs 512-dimensional L2-normalized vectors, enabling efficient retrieval via cosine similarity.

Stage 1: Ontology-Aware Contrastive Learning (Base Space)

• Goal: Initialize the embedding space to preserve curated domain knowledge before introducing noisy clinical data.
• Architecture: Uses MedEmbed-large-v0.1 as the base text encoder, followed by a two-layer MLP projection head.
• Training: Trained on a symmetric contrastive objective (InfoNCE) using positive pairs derived from ontologies (HPO, OMIM, Orphanet, HGNC).
  • Positive Pairs: Disease-phenotype, disease-gene, and phenotype-phenotype links.
  • Negatives: Hard negatives from differential diagnosis lists, random negatives, and in-batch negatives.
• Outcome: Creates a coherent neighborhood structure where diseases, genes, and phenotypes are semantically aligned based on curated knowledge.

Stage 2: Clinical Text Alignment (The Bridge)

• Goal: Align the curated ontology space with real-world clinical documentation (free-text notes and SNOMED codes).
• Architecture: Introduces a dual-encoder setup. A clinical transformer (GatorTron-base) encodes clinical text spans and SNOMED concept strings. Stage 1 embeddings remain frozen as the target.
• SNOMED Bridge: SNOMED CT embeddings are initialized by averaging mapped Stage 1 embeddings (via HPO/Orphanet cross-references) and refined using hierarchical smoothing.
• Training Phases:
  1. SNOMED Anchoring: Fits the SNOMED projection to Stage 1 targets.
  2. Clinical Alignment: Trains the clinical encoder using bidirectional contrastive loss between text segments and linked entities.
  3. Joint Fine-tuning: Updates encoders with a composite objective combining cross-modal alignment and code-bridge loss.
• Outcome: Enables the model to retrieve rare diseases from unstructured EHR notes and coded histories even when structured phenotyping is incomplete.

Stage 3: Graph Refinement (Knowledge Propagation)

• Goal: Refine embeddings using the relational structure of biomedical knowledge graphs to improve long-tail generalization and neighborhood consistency.
• Architecture: A Heterogeneous Graph Transformer (HGT) with type- and relation-specific attention.
• Graph Construction: Nodes include diseases, phenotypes, genes, GO terms, pathways, and anatomical structures. Edges represent disease-phenotype, disease-gene, and ontology hierarchies.
• Training: Trains a single HGT layer with a contrastive objective over graph-linked triplets. Uses a residual mixing mechanism to combine HGT outputs with input embeddings, preserving semantic content while incorporating structural propagation.
• Outcome: Enhances the representation by propagating information across the knowledge graph, improving ranking for rare conditions with sparse data.

3. Key Contributions

1. Unified Representation Space: Developed a rare-disease-specific embedding space that integrates ontological structure with disease-gene-phenotype knowledge, allowing coherent similarity comparisons across heterogeneous entities.
2. Clinical Alignment: Successfully aligned this space to real-world clinical reality by learning from free-text descriptions and coded EHR signals (SNOMED), enabling retrieval even when phenotypes are missing or noisy.
3. Practical Clinical Interface: RD-Embed supports multiple clinical workflows: differential diagnosis ranking, gene prioritization, phenotype inference, and case-based retrieval. It acts as a lightweight, structured input layer for LLMs without requiring retraining on rare-disease data.

4. Results

The model was evaluated across 10 rare-disease datasets (including HMS, LIRICAL, DECIPHER, and EHR-derived cohorts) and compared against general biomedical embeddings (MedEmbed, SapBERT) and large language models (GPT-OSS-120B, DeepSeek-R1).

• Diagnosis Retrieval:
  • RD-Embed achieved >50% Recall@10 on combined text and phenotype inputs (e.g., LIRICAL corpus), significantly outperforming general baselines (~19% for MedEmbed).
  • In text-only stress tests (UDPS cohort), RD-Embed maintained non-trivial performance (~20% Recall@10), whereas general baselines collapsed to near-zero.
• EHR Stress Test:
  • In routine EHR data, Stage 1 (ontology-only) underperformed. However, after Stage 2 (Clinical Alignment), performance jumped from 16% to **45% Recall@10** for OMIM diseases, surpassing MedEmbed (~30%).
  • Combining SNOMED codes with text yielded the strongest results.
• Gene Prioritization:
  • RD-Embed improved gene recall, achieving ~45% Recall@10 in the MME dataset (COMBINED input).
  • It effectively narrowed the search space for genomic interpretation, even in text-only scenarios (~17% Recall@10).
• Differential Diagnosis Re-ranking:
  • On difficult cohorts, Stage 3 improved Mean Reciprocal Rank (MRR) from 36 (Stage 1) to **55**, placing the correct diagnosis in the top three for ~82% of cases in the RAMEDIS cohort.
• Comparison with LLMs:
  • RD-Embed was competitive with or superior to massive LLMs (e.g., 120B parameters) on difficult EHR-aligned retrieval tasks (e.g., ~39% Recall@10 vs. ~26% for GPT-OSS-120B on Orphanet tasks), demonstrating that purpose-built representations outperform scale alone for rare diseases.

5. Significance and Impact

• Bridging the Gap: RD-Embed solves the critical disconnect between curated rare-disease knowledge and messy, real-world clinical data. It allows clinicians to query EHRs directly without requiring perfect, pre-extracted HPO profiles.
• Lightweight Deployment: Unlike massive LLMs, RD-Embed is a lightweight model that can be integrated into existing hospital systems for fast, real-time retrieval and triage.
• Complementary to AI: It serves as a structured retrieval substrate that can ground downstream LLM reasoning, providing a "rare-disease-focused" context that general models lack.
• Clinical Utility: By supporting early triage from narrative notes and gene shortlisting for genomic labs, RD-Embed has the potential to shorten the diagnostic odyssey for millions of patients with rare diseases.

Limitations: The study relies on retrospective data with geographic bias (mostly English/Asian cohorts). It lacks prospective clinical trials and formal uncertainty calibration, which are necessary next steps for clinical deployment.