Seeing Beyond the Image: ECG and Anatomical Knowledge-Guided Myocardial Scar Segmentation from Late Gadolinium-Enhanced Images

1. Problem Statement

Accurate segmentation of myocardial scar tissue from Late Gadolinium-Enhanced (LGE) Cardiac Magnetic Resonance (CMR) imaging is critical for assessing tissue viability and arrhythmic risk. However, automated segmentation remains challenging due to:

• Image Quality Issues: Low signal-to-noise ratios, heterogeneous contrast enhancement, and motion artifacts.
• Variability: Differences in scanner types and pathology presentation.
• Limitations of Image-Only Models: Purely image-based deep learning approaches (e.g., nnU-Net) struggle with high contrast variability and often fail to localize scars accurately without additional context.
• Data Misalignment: Electrocardiogram (ECG) signals, which provide crucial electrophysiological context for scar localization, are rarely acquired simultaneously with MRI. In clinical settings (e.g., UK NHS), ECGs are often recorded weeks or months prior to MRI, and simultaneous recordings suffer from magneto-hydrodynamic (MHD) artifacts.

2. Methodology

The authors propose a novel multimodal learning framework that integrates LGE-MRI, 12-lead ECG signals, and anatomical priors. The architecture consists of three core components:

A. Multi-modal Network Architecture

• Image Branch: Uses an image encoder ($E_{MRI}$) and decoder ($D_{MRI}$) based on a U-Net structure. It takes 2D LGE-MRI slices and an AHA-17 anatomical map as input.
• ECG Branch: Uses an ECG autoencoder ($E_{ECG}$ and $D_{ECG}$) to extract latent features ($z_{ECG}$) from 12-lead median waveforms. The decoder reconstructs the ECG signal to ensure the latent features retain physiological fidelity.
• Feature Fusion: The mid-level features from the MRI encoder are fused with the projected ECG features using the Temporal Aware Feature Fusion (TAFF) module.

B. Temporal Aware Feature Fusion (TAFF)

This is the core innovation designed to handle the non-simultaneous acquisition of ECG and MRI.

• Gating Mechanism: A Gating MLP learns modality-specific weights ($w_{MRI}, w_{ECG}$) to adaptively fuse the MRI features ($f$) and projected ECG features ($f_{ECG}$).
• Temporal Conditioning: A Temporal FiLM (Feature-wise Linear Modulation) layer takes the time interval ($t_{interval} = |t_{MRI} - t_{ECG}|$) as input. It generates affine scaling ($\gamma$) and shifting ($\beta$) parameters to modulate the fused features.
• Function: This allows the network to dynamically adjust the reliance on ECG data based on how "recent" or "reliable" the signal is relative to the MRI scan. If the time gap is large, the model can down-weight the ECG contribution.
• Residual Connection: A residual connection is added to the final fused feature ($f_{fused}$) to stabilize training, allowing the network to ignore the fusion if the ECG is uninformative.

C. Anatomical Prior (AHA-17)

• The 17-segment American Heart Association (AHA) model is used as a spatial prior.
• A 17-channel binary mask representing the myocardial segments is generated based on geometric partitioning.
• This mask is concatenated with the MRI input at the slice level to provide the network with explicit anatomical context, guiding feature alignment and constraining predictions to the myocardial wall.

D. Loss Function

The network is optimized using a composite loss:
$$L_{total} = L_{seg} + \lambda_{ECG} L_{ecg} + \lambda_{ent} L_{ent}$$

• $L_{seg}$: Segmentation loss (Dice + Cross-Entropy).
• $L_{ecg}$: Mean Squared Error (MSE) for ECG signal reconstruction.
• $L_{ent}$: Entropy regularization on gating weights to prevent premature "gating collapse" (where the model ignores one modality too early in training).
• Warm-up: The ECG contribution ($\lambda_{ECG}$) follows a quadratic warm-up schedule.

3. Key Contributions

1. TAFF Framework: A novel mechanism that adaptively fuses ECG and MRI features while explicitly accounting for the time difference between acquisitions, solving the temporal misalignment problem.
2. Anatomical Guidance: The integration of the 17-segment AHA model as a learnable anatomical prior to enforce physiological consistency and improve spatial alignment between electrical abnormalities and scar regions.
3. Bridging Domains: One of the first studies to successfully harness ECG signals to guide pixel-level MRI-based myocardial scar segmentation, effectively bridging electrophysiological and structural domains.

4. Experimental Results

The method was evaluated on a clinical dataset of 103 paired LGE-MRI and ECG cases (acquired within 90 days) from Nottingham University Hospitals.

• Performance Metrics:

  • Baseline (Image-only, nnU-Net): Dice Score = 0.6149, Precision = 0.7315, Sensitivity = 0.7072.
  • Proposed Multimodal Model: Dice Score = 0.8463, Precision = 0.9115, Sensitivity = 0.9043.
  • Improvement: A substantial ~37% increase in Dice score over the state-of-the-art image-only baseline.

• Ablation Studies:

  • Without Time Conditioning: Removing the temporal aspect of TAFF caused performance to drop significantly (Dice 0.5916), even lower than the image-only baseline, highlighting the difficulty of direct ECG-MRI fusion without temporal context.
  • Without Anatomical Prior: Removing the AHA-17 prior reduced the Dice score from 0.8234 to 0.5916 (in the no-time-conditioning scenario), proving the prior is essential for spatial regularization.
  • Time Interval Analysis: The model showed the most significant improvements for cases with time gaps of 1 week to 1 month, demonstrating robustness to temporal gaps when conditioning is applied.

5. Significance and Conclusion

This work demonstrates that integrating physiological (ECG), anatomical (AHA-17), and temporal information allows deep learning models to "see beyond the image."

• Clinical Impact: The approach significantly improves the accuracy of scar burden quantification, which is vital for patient risk stratification and treatment planning.
• Robustness: The TAFF mechanism effectively handles the real-world clinical challenge of non-simultaneous data acquisition, making the model applicable to routine hospital workflows where ECGs and MRIs are often recorded weeks apart.
• Future Direction: The study sets a new direction for physiologically grounded cardiac image analysis, suggesting that multimodal fusion with temporal awareness is a superior strategy for handling heterogeneous medical data.