A Unified Model for Highly Accurate ECG-Free Dynamic Coronary Roadmapping Using Spatio-Temporal Transformers

Technical Summary: A Unified Model for Highly Accurate ECG-Free Dynamic Coronary Roadmapping Using Spatio-Temporal Transformers

Problem Statement

Percutaneous Coronary Intervention (PCI) relies on repeated injections of iodine-based contrast agents to visualize coronary arteries and guide interventional devices. This practice increases radiation exposure and the risk of contrast-induced nephropathy, particularly in patients with renal impairment. Dynamic Coronary Roadmapping (DRM) offers a solution by overlaying a precomputed angiographic vessel map onto live fluoroscopy, reducing the need for repeated contrast injections.

However, accurate DRM requires precise cardiac phase matching between the angiography sequence and live fluoroscopy, as well as reliable catheter tip tracking for motion compensation. Existing methods face significant limitations:

• ECG Dependency: Many approaches rely on ECG signals for phase alignment, which are not always available or reliable in clinical settings.
• Annotation Burden: Image-based deep learning methods often require extensive, dense manual annotations (e.g., catheter masks) to learn motion cues, which is labor-intensive and costly.
• Modular Complexity: Prior works typically address phase matching and catheter tracking using separate models, increasing computational complexity and hindering real-time performance.
• Domain Mismatch: Models pretrained on natural images often fail to capture the subtle inter-frame dynamics specific to cardiac motion in X-ray imaging.

Methodology

The authors propose a unified DRM framework that simultaneously performs cardiac phase matching and catheter tip tracking without requiring ECG signals during inference. The approach is structured into three main stages:

1. Spatio-Temporal Pretraining:

   • A large-scale spatio-temporal transformer encoder is pretrained on 16 million frames from unlabeled angiography and fluoroscopy datasets.
   • This pretraining allows the model to learn fine-grained cardiac motion dynamics and robust feature representations specific to the medical domain, addressing the limitations of models trained on natural images.

2. Unified Training Pipeline:

   • Architecture: The model utilizes a shared spatio-temporal encoder to extract features from both angiography and fluoroscopy sequences. These features are processed by a lightweight CNN to produce frame-level embeddings, which are then combined with learnable positional encodings and passed through multi-headed self-attention (MHA) layers.
   • Auxiliary Tasks: To stabilize training and eliminate the need for extensive catheter mask annotations, the model employs two auxiliary tasks:
     • ECG R-Peak Detection: A binary classification task predicting R-peaks to provide a sparse but strong temporal cue for cardiac phase.
     • Catheter Tip Tracking: A heatmap regression task. Since full annotations are scarce, the authors use a pretrained HiFT (Historical Feature Guided Tracker) model to generate pseudo-labels for unannotated frames.
   • Loss Function: The total loss is a weighted sum of the R-peak detection loss (adaptive weighted binary cross-entropy), the cardiac phase matching loss (triplet loss), and the catheter tip tracking loss (Dice loss).

3. Inference Strategy:

   • Offline Phase: A contrast detection model identifies a cardiac cycle in the angiography sequence. A temporal window of 10 frames is used to precompute and store angiography embeddings and vessel roadmaps.
   • Online Phase: A sliding window of 10 fluoroscopy frames is processed in real-time. The model computes embeddings for the current live frame and matches them against all stored angiography embeddings using cosine similarity.
   • Majority Voting: To enhance robustness, a majority-voting post-processing strategy aggregates predictions from the temporal window. This generates a confidence score based on voting frequency, which correlates with the phase-matching error.

Key Contributions

• First Spatio-Temporal Pretraining for DRM: The work introduces the first use of a spatio-temporal encoder pretrained on 16 million medical frames for cardiac motion compensation in DRM, achieving superior phase-matching performance compared to existing methods.
• Annotation-Efficient Unified Framework: By integrating ECG R-peak detection and catheter tip tracking (using pseudo-labels) as auxiliary tasks, the model achieves stable convergence without requiring extensive manual catheter mask annotations.
• Real-Time Unified Architecture: The design jointly performs phase matching and catheter tracking in a single model, significantly reducing inference latency compared to sequential two-stage approaches.
• Confidence-Aware Post-Processing: The majority-voting strategy not only improves robustness but also provides a confidence score that correlates inversely with phase-matching error, offering a practical mechanism for uncertainty estimation in clinical settings.

Results

Comprehensive evaluations on clinical X-ray datasets demonstrate the following:

• Phase Matching Accuracy: The proposed method achieves state-of-the-art performance with a mean distance error of 56.31 ms (4.38% of the cardiac cycle) and a standard deviation of 0.66 frames. This outperforms baselines using ImageNet-pretrained CNNs, DINOv2, and previous triplet-loss approaches (AIT).
• Robustness: The model maintains a distance error below one frame for approximately 82% of test cases. The confidence score effectively predicts error magnitude; predictions with high confidence scores (>0.8) exhibit near-zero median errors.
• Catheter Tracking: Under automatic initialization, the unified model achieves a mean tracking error of 1.58 mm, which is statistically comparable to the HiFT baseline (1.45 mm) that requires manual initialization, despite the unified model's reliance on pseudo-labels.
• Efficiency: The unified single-stage formulation significantly improves throughput, operating at up to 84 fps for CNN-based backbones, compared to ~20 fps in sequential two-stage pipelines.

Significance and Claims

The paper claims that this work represents a significant step toward practical, real-time, and fully automated dynamic coronary roadmapping. By leveraging large-scale spatio-temporal pretraining and carefully designed auxiliary tasks, the framework successfully reduces dependence on extensive labeled data and external ECG signals.

The authors emphasize that their approach:

1. Generalizes better across diverse imaging conditions and cardiac motion patterns compared to models pretrained on natural images.
2. Enables clinical deployment by providing a confidence score that allows operators to assess the reliability of the system's output.
3. Balances accuracy and efficiency, offering a unified solution that is both computationally efficient and highly accurate, making it suitable for integration into live interventional workflows.

The paper concludes modestly, acknowledging that while the average error is low, outliers persist in challenging cases (e.g., extreme view angles or partial vessel cropping). Future work may require additional robustness strategies, such as further auxiliary tasks (e.g., guidewire detection) or more advanced multi-task optimization, to close the remaining gap to fully reliable clinical deployment.