GEN-KnowRD: Reframing AI for Rare Disease Recognition

The paper introduces GEN-KnowRD, a framework that leverages large language models to generate schema-guided rare disease profiles and construct a computable knowledge base, thereby decoupling knowledge construction from patient-level inference to significantly improve rare disease recognition and early diagnosis compared to existing state-of-the-art methods.

The Gist

Imagine you are a detective trying to solve a mystery, but the clues are scattered across thousands of different, messy notebooks, and the "bad guys" (rare diseases) are incredibly good at hiding. This is the daily reality for doctors trying to diagnose rare diseases. Patients often spend years on a "diagnostic odyssey," bouncing from specialist to specialist, because the information needed to solve the case is incomplete, hard to find, or written in a language only a few experts understand.

This paper introduces a new tool called GEN-KnowRD. Think of it not as a detective who solves the case for you, but as a super-librarian who reorganizes the entire library so the detective can solve the case much faster.

Here is how it works, broken down into simple concepts:

1. The Old Problem: The Messy Library

Traditionally, to diagnose a rare disease, computers tried to match a patient's symptoms against a database of diseases. But this database was like a library where:

• Some books were written by experts but were outdated.
• Other books were missing pages.
• The books were written in different styles, making them hard to compare.
• To update the library, a team of human experts had to manually rewrite every single book, which took forever and couldn't keep up with new medical discoveries.

2. The New Solution: The "Super-Librarian" (GEN-KnowRD)

The researchers realized that instead of asking a super-intelligent AI (a Large Language Model or LLM) to act as the detective for every single patient (which is expensive, slow, and risky for privacy), they should use the AI to build a better library first.

• Step 1: The AI Writes the Books: They asked four different "super-intelligent" AIs (like Claude, Gemini, and OpenAI) to write a perfect, standardized "profile" for 1,300 rare diseases. These profiles cover everything: symptoms, how to test for them, and how to treat them.
• Step 2: The Quality Check: Just like a human editor checks a manuscript, the team had real doctors review these AI-written profiles to make sure they were accurate and useful. They found that the AI profiles were often better and more detailed than the old, human-written ones.
• Step 3: The "PheMAP-RD" Database: They turned these profiles into a structured, computer-readable database. This is the "computable knowledge base."

3. How It Solves Cases (The Detective's Workflow)

Now, when a patient comes in with a mystery illness, the system doesn't ask the expensive AI to "think" from scratch. Instead, it uses a lightweight, fast, local computer program that works like this:

• Stage 1: The Quick Scan (The Net): The system takes the patient's medical notes and casts a wide net. It uses two methods:
  • Keyword Matching: Looking for exact words (like "cough" or "fever").
  • Semantic Matching: Understanding the meaning (e.g., knowing that "shortness of breath" is similar to "trouble breathing").
  • It quickly narrows down thousands of diseases to a short list of top 20 suspects.
• Stage 2: The Deep Dive (The Magnifying Glass): The system takes those top 20 suspects and compares them very closely against the patient's specific story using the new, high-quality library profiles. It re-ranks the list to put the most likely diagnosis at the very top.

4. Why This is a Game-Changer

The paper tested this system against the old methods and found it was a massive winner:

• Speed & Cost: It's much cheaper and faster because the heavy lifting (writing the books) is done once, not for every patient.
• Privacy: The patient's private data never has to leave the hospital's secure network to be sent to a big AI company. The "library" is local.
• Accuracy: In tests involving nearly 10,000 patients, this system was significantly better at ranking the correct disease at #1 compared to the best existing methods. In one specific test for a lung disease (Idiopathic Pulmonary Fibrosis), it was able to spot the disease much earlier than before, potentially saving lives by catching it before it gets too severe.

The Big Metaphor

Imagine you are trying to find a specific needle in a haystack.

• The Old Way: You hire a genius (the AI) to look at every single piece of hay for every single person who walks in. It's slow, expensive, and the genius might get tired or make mistakes.
• The GEN-KnowRD Way: You hire the genius once to build a machine that sorts the hay into neat, labeled piles based on what the needle looks like. Now, when a new person walks in, you just drop their hay into the machine, and it instantly points to the right pile.

In short: GEN-KnowRD shifts the role of Artificial Intelligence from being the "doctor" who makes the diagnosis to being the "architect" who builds a better, smarter, and more organized knowledge base. This allows doctors and computers to find rare diseases faster, cheaper, and more accurately.

Technical Summary

1. Problem Statement

Rare diseases affect over 300 million people globally, yet patients face diagnostic delays of several years due to clinical complexity and a lack of clinician experience. Existing computational Rare Disease Recognition (RDR) approaches face two primary bottlenecks:

• Knowledge Limitations: Traditional methods rely on human-curated knowledge bases (e.g., HPO terms, NORD reports) that are often incomplete, heterogeneous, expensive to maintain, and slow to update with emerging evidence.
• LLM Limitations: Directly applying Large Language Models (LLMs) for end-to-end diagnosis or discrimination introduces issues with implicit knowledge (hard to trace/govern), high computational costs, data privacy concerns (sending patient data to external APIs), and reproducibility.

The authors hypothesize that the core limitation of RDR lies not in downstream inference but in the upstream synthesis and structuring of disease knowledge.

2. Methodology: GEN-KnowRD Framework

The authors propose GEN-KnowRD, a "knowledge-layer-first" framework that decouples knowledge construction from patient-level inference. The system consists of four main components:

A. Knowledge Layer Construction (RDK Generation)

Instead of using LLMs to reason over individual patients, the framework uses LLMs to generate reusable, schema-guided Rare Disease Profiles (RDPs).

• Source Data: 1,320 rare diseases from the National Organization for Rare Disorders (NORD) database.
• LLM Agents: Four state-of-the-art models (Claude Sonnet 4, DeepSeek R1, Gemini 2.5 Pro, OpenAI o3) were prompted to generate profiles across 10 clinical sections (e.g., etiology, clinical presentation, diagnostics).
• Knowledge Extraction: The generated profiles are processed to extract UMLS (Unified Medical Language System) concepts across four semantic groups: Symptoms & Conditions, Genetics & Molecular Biology, Diagnostics & Lab Findings, and Drugs & Procedures.
• Output: A computable knowledge base named PheMAP-RD, which maps clinical narratives to machine-actionable concepts.

B. Quality Evaluation

The generated profiles undergo a multi-faceted evaluation:

• Automated Metrics: Execution time, token usage, readability (SMOG score), and citation analysis.
• Expert Review: Clinical experts evaluated factual accuracy, completeness, utility, and specificity on a blind basis against NORD reports.

C. Inference Pipelines

The framework utilizes two lightweight, local inference pipelines that do not require sending patient data to LLMs:

1. General-Purpose Screening (Two-Stage Pipeline):
   • Stage 1 (Retrieval): Converts patient clinical notes into sparse (UMLS concepts) and dense (embeddings) representations. It uses BM25 for lexical matching and Qwen3-Embedding-8B for semantic matching. Results are fused using Reciprocal Rank Fusion (RRF).
   • Stage 2 (Reranking): A Qwen3-reranker-8B model re-evaluates the top 20 candidates from Stage 1, incorporating the full disease profile context to refine the ranking.
2. Specialized Discrimination:
   • Designed for specific diseases (e.g., Idiopathic Pulmonary Fibrosis - IPF).
   • Extracts temporal features and symptom co-occurrences from longitudinal EHR notes.
   • Uses weighted concept importance (TF-IDF and Symptom Importance Score - SIS) to train lightweight classifiers (Logistic Regression) to distinguish cases from controls.

D. Ensemble Strategy

A GEN-KnowRD-Ensemble approach was developed, using a gated mechanism based on Spearman rank correlation to select the most consensus-aligned ranking from the different LLM-generated knowledge sources for each patient.

3. Key Contributions

• Paradigm Shift: Repositions LLMs from "per-patient reasoners" to "knowledge synthesizers," creating a reusable, local, and auditable knowledge layer (PheMAP-RD).
• PheMAP-RD Resource: A new, computable knowledge base derived from LLMs that integrates UMLS concepts, offering broader coverage than traditional HPO-only approaches.
• Hybrid Architecture: Combines sparse (BM25) and dense (Embedding) retrieval with a knowledge-boosted reranker, achieving high performance with lightweight, local inference.
• Privacy-Preserving Design: By moving LLM usage to an offline knowledge construction phase, the system allows for on-premise deployment without transmitting sensitive patient data to external APIs.

4. Key Results

The framework was evaluated on 6 public benchmarks (9,290 patients, 798 diseases) and 2 private cohorts (511 IPF patients).

• General-Purpose Screening Performance:

  • vs. HPO-Centric Baseline (PhenoBrain): GEN-KnowRD-Ensemble achieved a 345.8% improvement in Top-1 success rate on the PMC benchmark.
  • vs. End-to-End LLM (OpenAI GPT-5): GEN-KnowRD-Ensemble outperformed GPT-5 by 129.1% in Top-1 success on the Non-PMC benchmark (structured HPO data), demonstrating that lightweight pipelines with structured knowledge outperform raw LLM reasoning on specialized tasks.
  • vs. Expert-Curated Knowledge: LLM-generated profiles (specifically from Claude Sonnet 4) consistently outperformed the gold-standard NORD reports in ranking accuracy.
  • Reranking Impact: Stage 2 reranking improved Recall@1 by 36.5% over Stage 1 alone when using NORD reports, and even more significantly for LLM-generated profiles.

• Specialized Discrimination (IPF Early Diagnosis):

  • In distinguishing IPF from non-IPF pulmonary diseases, the Claude Sonnet 4-based profile achieved an AUROC of 0.992 and F1 of 0.956 (using SIS weighting), significantly outperforming other sources.
  • In the more difficult task of distinguishing IPF from suspected IPF, LLM-generated profiles generally outperformed the NORD report, with OpenAI o3 showing the highest average performance in this specific borderline scenario.

• Quality Analysis:

  • Expert reviewers found LLM-generated profiles (particularly DeepSeek R1 and OpenAI o3) to be equal to or superior to NORD reports in factual accuracy, completeness, and utility.
  • Claude Sonnet 4 generated the most comprehensive profiles (highest UMLS concept coverage) and showed the best downstream performance, despite having the highest token usage and cost during generation.

5. Significance and Implications

• Scalability & Cost: GEN-KnowRD amortizes the high cost of LLM usage by generating knowledge once, which can then be reused infinitely for patient inference using cheap, local models. This makes RDR scalable for health systems.
• Living Knowledge Base: The modular pipeline allows for continuous updates. As new medical evidence emerges or better LLMs become available, the knowledge base can be regenerated and re-benchmarked without re-curating manually.
• Clinical Utility: The framework supports earlier specialist referral, clinical trial matching, and real-time decision support by reducing diagnostic odysseys.
• Future of AI in Medicine: The study suggests that the future of medical AI lies not in replacing clinicians with "black box" LLMs, but in using LLMs to build structured, high-quality knowledge infrastructures that empower lightweight, interpretable, and privacy-preserving diagnostic tools.