Learning lifetime disease liability reveals and removes genetic confounding in electronic health records

This paper introduces EDGAR, a deep learning framework that recovers lifetime disease liability from electronic health records to improve the specificity of genetic association studies by identifying and removing a systemic genetic confounder that causes spurious cross-disease correlations.

The Gist

Imagine you are trying to understand why people get sick by looking at their medical records. You want to find the "genetic recipe" that causes diseases like diabetes or anxiety.

The problem is that medical records (Electronic Health Records, or EHRs) are like a noisy, messy diary written by a busy doctor who is also influenced by the patient's wallet, their personality, and the hospital's billing rules.

• The Mess: If a patient is poor, they might not go to the doctor often, so their record says they are "healthy" even if they are sick. If a patient is anxious, they might visit the doctor for everything, so their record looks "sicker" than they really are.
• The Genetic Trap: Because these factors (money, personality, access to care) are partly inherited, scientists looking at the genetic data see a fake link. They think, "Oh, this gene causes anxiety!" when really, that gene just makes people more likely to visit the doctor or get diagnosed. It's a "circular bias" where the record looks like biology, but it's actually just a reflection of how the healthcare system works.

The Solution: EDGAR (The "Truth Detector")

The authors of this paper built a new AI tool called EDGAR. Think of EDGAR as a super-smart translator that can read the messy diary and translate it into the "true story" of a person's health.

Here is how it works, using a simple analogy:

1. The "Deep Phenotype" (The Gold Standard)

Imagine you have a few people who have been thoroughly checked by a team of specialists. They have blood tests, detailed interviews, and scans. This is the "Deep Phenotype"—the true, ground-truth health status.

• The Problem: You can't afford to do this deep check on 300,000 people. It's too expensive and takes too long.
• The Trick: The researchers used Active Learning. Imagine you are a teacher with a huge class and only enough time to grade 20 essays. Instead of picking essays at random, you use a smart algorithm to pick the most confusing or most interesting essays to grade first. This helps you learn the grading rules much faster. EDGAR does this: it picks the most "informative" patients to get those expensive deep checks, then uses that knowledge to guess the health status of everyone else.

2. The Translation (From Codes to Liability)

EDGAR looks at the messy EHR codes (like "ICD-10 code for cough") and the deep checks, and learns the pattern. It then predicts a "Lifetime Disease Liability" for everyone.

• Analogy: Instead of just counting how many times someone coughed (the messy code), EDGAR estimates the actual probability that they have the underlying lung disease, regardless of whether they went to the doctor or not.

Why This Matters: Three Big Wins

1. Finding the Real Genetic Clues (Better Power)
When scientists ran genetic tests (GWAS) using the messy EHR codes, they found fewer real genetic links. When they used EDGAR's "cleaned" predictions, they found more genetic links, and they were more accurate. It's like cleaning a dirty window; suddenly, you can see the stars (the real genes) that were hidden by the smudge.

2. Fixing the "Fake Friends" (Removing Confounding)
The researchers discovered a "Ghost Factor." This is a hidden genetic trait that makes people more likely to use the healthcare system, smoke, have lower education, or report errors. This factor makes it look like different diseases are genetically related when they aren't.

• The Fix: EDGAR identified this "Ghost Factor" and subtracted it. It's like removing the static from a radio signal. Once they removed this noise, the fake links between diseases disappeared, and the links to socioeconomic traits (like poverty) vanished.

3. Cleaning Up Other Databases (The Ripple Effect)
The best part? They found this "Ghost Factor" in the UK data (UK Biobank) and used it to clean up data from a different country (Finland).

• Analogy: Imagine you found a specific type of dirt that only appears on cars in London. You figured out exactly what that dirt looks like. Then, you drove to Paris, found cars covered in the same dirt, and used your London knowledge to wash it off, even though you never studied Paris cars directly. This means we can fix genetic studies in many places without needing new, expensive deep checks everywhere.

The Bottom Line

This paper is a game-changer because it teaches us how to separate the biology of disease from the bureaucracy of healthcare.

• Before: We were studying the "healthcare system's habits" and calling it "genetics."
• Now: With EDGAR, we can strip away the noise of insurance, income, and doctor visits to see the true genetic blueprint of disease.

It's like finally putting on a pair of glasses that filters out the glare, allowing us to see the patient's true health and their real genetic destiny.

Technical Summary

1. Problem Statement

Electronic Health Records (EHRs) have become a cornerstone for large-scale Genome-Wide Association Studies (GWAS) due to their massive sample sizes. However, EHR diagnostic codes are not pure reflections of biological disease; they are confounded by systemic biases arising from:

• Healthcare engagement: Variations in care-seeking behavior, access to care, and interactions with clinical systems.
• Operational factors: Billing incentives, heterogeneous coding practices (ICD vs. Read2/CTV3), and institutional biases.
• Heritability of confounders: Many of these behavioral and structural factors (e.g., socioeconomic status, health literacy) are themselves heritable.

This creates a "circularity of bias" where spurious genetic signals driven by healthcare access or reporting habits are mistaken for robust biological discoveries. Existing deep learning models for EHR phenotyping often rely solely on EHR codes for supervision, inadvertently replicating these systemic biases. There is a critical need for a framework that can recover lifetime disease liability (the true biological risk) while disentangling it from EHR-specific noise.

2. Methodology: The EDGAR Framework

The authors propose EDGAR (EHR Disease liability prediction for Genetic Architecture Recovery), a deep learning framework designed to align EHR records with "deep phenotypes" (clinically validated assessments) to predict true disease liability.

A. Data and Phenotypes

• Dataset: 337,129 White British individuals from the UK Biobank.
• Target Diseases: 9 common conditions (GAD, MDD, Anaemia, COPD, Hyper-LDL, NAFLD, Diabetes, Hypertension, Osteoporosis).
• Phenotype Definitions:
  1. EHR Phenotype: Binary presence of diagnosis codes (GP/Hospital).
  2. Deep Phenotype: Gold-standard labels derived from biomarkers, CIDI questionnaires, or clinical assessments (available for a subset).
  3. Predicted Liability: The output of the EDGAR model.

B. Model Architecture and Training

• Architecture: EDGAR adapts the AutoComplete architecture (a Multi-Layer Perceptron) to handle tabular EHR data.
• Input Features:
  • Counts of diagnosis codes (more informative than binary presence).
  • Preservation of original coding formats (Read2/CTV3) rather than mapping to ICD-10 to retain operational context.
  • Age at diagnosis (first and last occurrence).
  • Disease-relevant measures: Continuous clinical variables (e.g., blood pressure, spirometry, biomarkers) available in the EHR.
• Loss Function: Target-specific loss optimization, supervised exclusively by masked deep phenotype labels (not EHR codes), ensuring the model learns biological liability rather than coding patterns.
• Active Learning: To address the scarcity of deep phenotype labels, the authors employ deep batch active learning (using strategies like Conf, Coreset, and Badge). This prioritizes the most informative individuals for "call-back" to obtain deep labels, significantly improving label efficiency.

C. Bias Identification and Removal

• GWAS-by-Subtraction: The authors use a structural model to isolate a latent "Bias" factor ($Bias_k$) for each disease. This factor represents the genetic component of the EHR phenotype that is orthogonal to the true disease liability ($Liab_k$).
• Common Bias Factor: Using Genomic SEM, they extract a shared "Common Bias" factor across all nine diseases, which captures the systemic heritable confounding.
• External Bias Removal: They demonstrate that this Common Bias factor is generalizable. They apply a subtraction model to remove this bias from external EHR GWAS (e.g., FinnGen) using only summary statistics, without re-training on individual-level data.

3. Key Results

A. Prediction Accuracy

• Baseline: Raw EHR phenotypes show low predictiveness for deep phenotypes (Macro-AUC $\approx$ 0.64).
• EDGAR Performance:
  • Using only EHR counts improves Macro-AUC to 0.74.
  • Adding disease-relevant clinical measures (biomarkers, labs) dramatically boosts performance to Macro-AUC = 0.98 (Pearson's $r$ up to 0.97).
  • Active Learning: The Conf strategy achieves the same prediction accuracy ($R^2$) with only 41% of the labeling budget compared to random sampling (a 2.43-fold efficiency gain).
• Architecture: Tabular models (EDGAR) perform comparably to state-of-the-art Transformer models (e.g., Delphi-2M) when trained on deep phenotype labels, suggesting that sophisticated sequence modeling is not strictly necessary if the supervision signal is high-quality.

B. GWAS Power and Specificity

• Locus Discovery: GWAS performed on EDGAR liabilities identified more genome-wide significant loci than GWAS on raw EHR phenotypes for 7 out of 9 diseases.
• Genetic Correlation ($r_G$):
  • EDGAR liabilities show higher genetic correlation with external Deep Phenotype GWAS than raw EHR phenotypes do.
  • Conversely, raw EHR phenotypes show higher correlation with external EHR GWAS (FinnGen), suggesting EHR-based associations replicate due to shared bias rather than shared biology.
• PRS Specificity: Polygenic Risk Scores (PRS) derived from EDGAR liabilities show higher specificity (lower pleiotropy) and better cross-ancestry portability compared to EHR-based PRS.

C. Identification and Removal of Systemic Bias

• The Common Bias Factor: The authors identified a heritable factor shared across EHR phenotypes that correlates strongly with socioeconomic and behavioral traits (e.g., lower education, smoking, poor self-reported health, lower participation probability).
• Bias Removal Impact:
  • Removing the Common Bias factor from external EHR GWAS (FinnGen) significantly increased the genetic correlation with true disease liability (EDGAR/Deep GWAS).
  • It abolished spurious correlations between the EHR GWAS and socioeconomic/behavioral traits.
  • This proves that the "robustness" of EHR-based genetic associations is often an artifact of replicable bias.

4. Significance and Contributions

1. Methodological Innovation: EDGAR provides a scalable framework to convert noisy EHR codes into high-fidelity "lifetime disease liability" phenotypes by integrating deep phenotypes and active learning.
2. Disentangling Biology from Bias: The study provides empirical evidence that a significant portion of genetic signals in EHR-based GWAS are driven by heritable confounders (healthcare access, behavior) rather than disease biology.
3. Practical Solution for Existing Data: The authors demonstrate that systemic bias can be identified in one cohort (UK Biobank) and mathematically removed from summary statistics of other cohorts (FinnGen) without needing individual-level data. This offers a immediate pathway to clean up thousands of existing EHR-based GWAS.
4. Efficiency: The active learning component shows that high-quality genetic studies can be conducted with a fraction of the expensive "deep phenotyping" budget, making the approach feasible for future prospective studies.

In conclusion, this work shifts the paradigm from treating EHR codes as ground truth to treating them as noisy proxies that require statistical correction. It offers a robust pipeline to recover true disease genetics and eliminate the "circularity of bias" that has plagued population-scale genetic research.