Dual-Phase Cross-Modal Contrastive Learning for CMR-Guided ECG Representations for Cardiovascular Disease Assessment

1. Problem Statement

Cardiovascular diseases (CVD) are a leading cause of global mortality, often developing silently before symptoms appear. While Cardiac Magnetic Resonance (CMR) imaging provides the gold standard for assessing cardiac structure, function, and tissue properties, its high cost and complexity limit its use to specialized centers, making it unsuitable for large-scale screening. Conversely, the Electrocardiogram (ECG) is ubiquitous, inexpensive, and non-invasive but offers limited insight into underlying cardiac structure and mechanical function.

The core challenge is to bridge this gap: How can we extract rich, image-derived structural and functional cardiac phenotypes from ECG signals alone? Previous deep learning approaches using contrastive learning between ECG and CMR have been limited by:

• Using simplified 2D CMR representations (single slices or 2D temporal sequences).
• Failing to fully capture the 3D geometry of the heart or the spatial relationships between structures.
• Neglecting the distinct physiological differences between End-Diastole (ED) and End-Systole (ES), which are critical for assessing myocardial contractility and deformation.

2. Methodology

The authors propose a Dual-Phase Cross-Modal Contrastive Learning Framework that aligns ECG representations with comprehensive 3D CMR volumes at both ED and ES phases. The framework consists of three main stages:

A. Data Preparation

• Dataset: UK Biobank, utilizing 34,142 paired ECG-CMR records.
• ECG Data: 10-second, 12-lead recordings (500 Hz), preprocessed with filtering and normalization.
• CMR Data: Short-axis cine stacks (up to 12 slices, 50 temporal frames). Six central slices covering the left ventricle were selected. Two specific phases were extracted: End-Diastole (ED) and End-Systole (ES). Volumes were cropped and zero-padded for consistency.

B. Model Architecture

1. ECG Encoder (Self-Supervised):
   • A Vision Transformer (ViT) is trained using a Masked Autoencoding objective.
   • Random segments of the 12-lead signal are masked, and the model reconstructs them to learn robust morphological features.
   • Output: 576-dimensional embedding projected to a 128-dimensional latent space.
2. CMR Encoders (Supervised):
   • Two separate 3D ResNet-50 encoders are used: one for ED volumes and one for ES volumes.
   • They are trained in a supervised manner to regress specific CMR-derived phenotypes (volumes, mass, strain, etc.).
   • Output: 2048-dimensional embeddings projected to the same 128-dimensional latent space.
3. Dual-Phase Contrastive Alignment:
   • Objective: Align the ECG embedding ($z_{ECG}$) simultaneously with both the ED CMR embedding ($z_{ED}$) and the ES CMR embedding ($z_{ES}$) in a shared latent space.
   • Loss Function: A weighted sum of three contrastive losses (inspired by CLIP):
     $$L_{dual-phase} = \frac{1}{3} (L_{ECG \to CMR_{ED}} + L_{ECG \to CMR_{ES}} + L_{CMR_{ED} \to CMR_{ES}})$$
   • Training Strategy: The CMR encoders are frozen after pretraining to preserve phenotype-informed representations. Only the ECG encoder and projection heads are optimized during the alignment phase. Positive pairs are ECG-CMR from the same subject; negative pairs are from different subjects.

C. Downstream Prediction

• After alignment, the enriched ECG embeddings are fed into Multi-Layer Perceptrons (MLPs) to predict:
  • CMR Phenotypes: 12 metrics including ventricular volumes, myocardial mass, wall thickness, ejection fraction, and global strain (circumferential, longitudinal, radial).
  • Clinical Outcomes: 6 endpoints including Coronary Artery Disease (CAD), Atrial Fibrillation (AF), Sudden Cardiac Death (SCD), Heart Failure (HF), Myocardial Infarction (MI), and Cardiomyopathy (CMP).

3. Key Contributions

1. Dual-Phase 3D Alignment: Unlike prior works using 2D slices, this method explicitly aligns ECGs with 3D volumetric data at both ED and ES phases. This allows the model to disentangle structural (geometry) and functional (dynamics) cardiac properties.
2. Scalable Phenotype Extraction: Demonstrates a method to extract complex, image-derived cardiac traits from a simple, low-cost ECG signal at inference time.
3. Comprehensive Evaluation: Systematically compares 2D vs. 3D representations and single-phase vs. dual-phase approaches, showing that preserving 3D anatomy and phase-specific dynamics is crucial for functional assessment.
4. Open Source: The code is publicly available to facilitate reproducibility.

4. Results

The study evaluated performance using $R^2$ for phenotypes and AUROC for clinical outcomes, comparing the proposed model against baselines (ECG only, 2D CL, 2D+Time CL, 3D-ED CL).

• Functional Phenotypes (Major Improvement):
  • The proposed Dual-Phase 3D CL model achieved the best performance for functional metrics.
  • Average Improvement: +9.2% over the baseline ECG model.
  • Specific Gains: Significant improvements in Ejection Fraction (LVEF) and all Strain measurements (Global Circumferential, Longitudinal, and Radial). For example, LVEF prediction improved from 0.482 (baseline) to 0.572.
• Volumetric/Mass Phenotypes:
  • Performance was mixed; 2D models sometimes outperformed 3D models for volume prediction, but the dual-phase model remained competitive or superior for mass and wall thickness.
• Clinical Outcomes:
  • Improvements were modest (+0.7% average over baseline).
  • The best AUROC for Heart Failure increased from 76.8% to 79.7%.
  • The authors attribute this limitation to the multifactorial nature of clinical diseases (involving genetics, lifestyle, comorbidities) which cannot be fully captured by cardiac structure alone.
• Encoder Capacity:
  • A "Medium" sized ViT (48M parameters) was found to be the optimal trade-off, outperforming larger "Base" models on specific tasks like QRS duration and sex classification while using fewer parameters.

5. Significance and Future Directions

• Clinical Impact: This approach enables scalable, cost-effective screening for cardiac structural and functional abnormalities using only an ECG. This is particularly valuable for populations where CMR is inaccessible.
• Scientific Insight: The results confirm that 3D anatomical context and phase-specific dynamics (ED vs. ES) are essential for learning meaningful cardiac representations from ECGs. Simplified 2D representations fail to capture global shape changes and volumetric contraction patterns.
• Limitations:
  • Data Bias: The training data (UK Biobank) is heavily skewed toward healthy individuals, limiting generalizability to patients with advanced or heterogeneous heart disease.
  • Clinical Prediction Ceiling: The modest gains in predicting clinical outcomes suggest that ECG-derived structural traits alone are insufficient for complex risk stratification without integrating demographics, labs, or history.
• Future Work:
  • Validation on external datasets with diverse, high-risk patient populations.
  • Integration of additional data modalities (demographics, labs) to improve clinical outcome prediction.
  • Exploring temporally resolved CMR data (beyond just ED/ES) to capture the full cardiac cycle dynamics.

In conclusion, the paper establishes a new state-of-the-art for cross-modal cardiac learning, proving that aligning ECGs with full 3D, dual-phase CMR volumes significantly enhances the ability to infer cardiac function from electrical signals.