Opportunistic Cardiac Health Assessment: Estimating Phenotypes from Localizer MRI through Multi-Modal Representations

1. Problem Statement

Cardiovascular diseases (CVD) are the leading cause of global mortality. Accurate assessment relies on Cardiac Phenotypes (CPs) such as Left Ventricular Ejection Fraction (LVEF), Left Ventricular Mass (LVM), and ventricular volumes.

• Current Standard: The gold standard for extracting these phenotypes is Cardiac Cine Magnetic Resonance Imaging (CMR). However, CMR is expensive, time-consuming, requires high spatio-temporal resolution, and necessitates dedicated ECG gating hardware.
• The Gap: Routine MRI examinations begin with Localizer scans (rapid, coarse, static images used for planning) which are typically discarded after scan planning. While these lack temporal resolution and diagnostic quality, they contain structural information.
• The Opportunity: There is a need for a low-cost, accessible screening tool that can estimate CPs using the ubiquitous localizer MRI, augmented by other routinely available data sources like Electrocardiograms (ECG) and patient metadata (demographics, lifestyle), without requiring a full CMR protocol.

2. Methodology: The C-TRIP Framework

The authors propose C-TRIP (Cardiac Tri-modal Representations for Imaging Phenotypes), a multi-modal contrastive learning framework designed to align three distinct data modalities: Localizer MRI (L), ECG signals (E), and Tabular Metadata (T).

The framework operates in three sequential stages:

Stage I: Single-Modality Pre-training (Self-Supervised)

Three independent encoders are pre-trained using Masked Autoencoders (MAEs) with a self-supervised objective (reconstructing 75% of masked input from 25% visible input):

• Localizer Encoder: Uses a Vision Transformer (ViT) adapted for 2D images.
• ECG Encoder: Processes 1D multi-channel ECG signals divided into non-overlapping patches to capture temporal dynamics.
• Tabular Encoder: Processes discrete feature tokens (numerical and categorical) projected into a uniform embedding space.
• Loss: Mean Squared Error (MSE) for numerical reconstruction; Cross-Entropy for categorical features.

Stage II: Multi-Modal Contrastive Alignment

The pre-trained encoders are fused to create a unified latent space. The core innovation is a Localizer-Centric Contrastive Loss:

• Objective: Align the Localizer embeddings ($L$) with ECG ($E$) and Tabular ($T$) embeddings.
• Mechanism: Uses an InfoNCE-based loss to pull embeddings of the same patient closer and push different patients apart.
• Constraint: The model aligns $L \leftrightarrow E$ and $L \leftrightarrow T$, but intentionally avoids direct $E \leftrightarrow T$ alignment. This prevents the model from learning "shortcuts" (e.g., inferring heart structure solely from age/sex without imaging) and ensures the localizer remains the anchor for spatial information.
• Loss Function: $L_{Total} = \frac{1}{2}(L_{L \leftrightarrow E} + L_{L \leftrightarrow T})$.

Stage III: Inference (Single-Modality Prediction)

• Training: A lightweight Multi-Layer Perceptron (MLP) regression head is attached to the Localizer encoder only.
• Inference: CPs are predicted exclusively from Localizer MRI. The ECG and Tabular data are used only during the training/alignment phase to enrich the latent space representation of the localizer.

3. Key Contributions

1. Localizer-Centric Tri-Modal Alignment: Unlike previous studies that align all modalities equally, C-TRIP anchors on the localizer MRI, using ECG for functional/temporal context and tabular data for clinical context, without allowing non-imaging modalities to bypass the image data.
2. Multi-Modal Training, Single-Modality Inference: The framework leverages the richness of ECG and metadata during training to learn a robust representation, but requires only the cheap, rapid localizer MRI for deployment.
3. Robustness in Low-Data Regimes: The model demonstrates strong performance even when fine-tuning data is scarce (e.g., 1% of the dataset), making it suitable for scenarios with limited labeled data.
4. Opportunistic Screening Tool: It transforms discarded localizer scans into a viable screening tool for cardiac health, increasing accessibility without replacing the gold-standard CMR.

4. Experimental Results

The study utilized the UK Biobank dataset (N ≈ 20,000) with paired Localizer, ECG, Tabular, and CMR data.

• Performance Metrics: Evaluated using Mean Difference (MD), Limits of Agreement (LoA), and Pearson's Correlation Coefficient ($R$).
• Structural CPs (LVM, RVEDV):
  • C-TRIP achieved the smallest Mean Difference and tightest LoA among all non-CMR methods.
  • It significantly outperformed unimodal baselines (Localizer-only, ECG-only, Tabular-only).
  • Notably, adding explicit CPs to the tabular data ($L+E+T_p$) did not improve performance and sometimes degraded it, suggesting the model learns generalizable features rather than memorizing labels.
• Functional CPs (LVEF, RVEF):
  • While CMR (the reference) remained the best, C-TRIP was competitive, achieving the second-best performance after CMR.
  • ECG-only ($E_{sup}$) performed poorly on structural metrics, highlighting the necessity of the imaging modality.
• Low-Data Scaling:
  • In a 1% fine-tuning regime, C-TRIP maintained high correlation ($R > 0.70$) for structural CPs, whereas unimodal baselines ($L_{sup}$, $E_{sup}$) suffered significant performance drops.
• Attention Analysis:
  • Attention maps showed that C-TRIP focused on biologically relevant cardiac structures (LV/RV chambers), whereas the baseline Localizer-only model ($L_{sup}$) exhibited "leaked" attention to non-cardiac structures (lungs, chest wall).
  • UMAP visualizations confirmed that the alignment successfully merged Localizer and Tabular representations (due to strong anatomical correlations like sex/size) while keeping ECG distinct but aligned along phenotypic gradients.

5. Significance and Conclusion

• Clinical Impact: C-TRIP offers a low-cost, opportunistic screening solution. It allows clinicians to estimate critical cardiac phenotypes from routine, non-diagnostic localizer scans, potentially identifying at-risk patients who cannot undergo a full CMR protocol.
• Technical Insight: The study proves that multi-modal contrastive learning can effectively distill temporal (ECG) and contextual (Tabular) knowledge into a spatial (MRI) representation, enabling high-accuracy inference from a single, low-quality modality.
• Limitations & Future Work: The current dataset is homogeneous (single vendor/scanner). Future work requires validation on multi-site, multi-vendor data where localizer acquisition protocols vary. Additionally, the reference CMR used in the study was a subset of cine frames, not the full diagnostic stack.

In summary, C-TRIP successfully repurposes discarded MRI data into a powerful diagnostic aid by leveraging the synergy between imaging, electrophysiology, and patient history, bridging the gap between high-cost gold standards and accessible routine care.