Harmonic-Aware Transformer for Real-Time Catheter Localization in Interventional Procedures of Magnetic Particle Imaging

Technical Summary: Harmonic-Aware Transformer for Real-Time Catheter Localization in MPI

Problem Statement
Magnetic Particle Imaging (MPI) offers a radiation-free, real-time imaging modality suitable for interventional procedures by tracking magnetic nanoparticle-coated instruments. However, conventional MPI-guided interventions rely on image reconstruction pipelines (e.g., system matrix inversion or X-space methods) to determine instrument positions. These traditional approaches suffer from high computational latency, memory constraints, and potential image blurring, which hinder true real-time tracking. While deep learning has been applied to MPI for image reconstruction and denoising, few methods bypass the reconstruction step entirely to directly predict instrument coordinates from raw signals. This paper addresses the need for a low-latency, high-accuracy framework that maps raw MPI voltage signals directly to 3D catheter tip positions without intermediate image formation.

Methodology
The authors propose a Harmonic-Aware Transformer framework that operates directly in the frequency domain. The methodology consists of three primary stages:

1. Harmonic-Aware Preprocessing:

   • Raw complex-valued MPI signals from three receive coils ($x, y, z$) are transformed into the frequency domain using Fast Fourier Transform (FFT).
   • A band-pass filter isolates the 2nd to 8th harmonics of the drive-field frequencies. The fundamental frequency is excluded as it is dominated by modulation signals, while higher-order harmonics are often noise-dominated.
   • The filtered real and imaginary components are concatenated to form a six-channel frequency-domain feature vector, preserving motion-relevant spectral components while enhancing the signal-to-noise ratio (SNR).

2. Transformer Architecture:

   • The model, termed the "catheter transformer," utilizes a 1D convolutional layer to patch-embed the frequency-domain signals (kernel size $P=64$).
   • Sinusoidal positional encoding is added to the embedded patches to retain sequence order information.
   • The sequence is processed by six encoder layers, each containing eight multi-head self-attention (MHSA) heads. This architecture is designed to capture long-range temporal dependencies and cross-axis correlations ($x, y, z$) within the spectral data.
   • A position-wise feed-forward network (FFN) with GELU activation follows the attention mechanism.
   • The final encoded representation is flattened and passed through a two-layer Multilayer Perceptron (MLP) to regress the 3D catheter tip coordinates ($\hat{x}, \hat{y}, \hat{z}$).

3. Training Strategy:

   • The model is trained using a weighted Mean Absolute Error (MAE) loss function. Crucially, the $z$-axis is assigned a higher weight ($w_z > w_x = w_y$) to compensate for the inherently lower spatial resolution of MPI along the selection field gradient direction.
   • Training utilizes the AdamW optimizer with a cosine annealing schedule and mixed-precision acceleration.
   • Data augmentation includes amplitude scaling and Gaussian noise injection to improve generalization.

Key Contributions

• Reconstruction-Free Framework: The study introduces a signal-driven approach that directly maps raw MPI voltage signals to 3D coordinates, eliminating the computationally expensive system matrix inversion and image reconstruction steps.
• Harmonic-Aware Preprocessing: A novel strategy that selectively extracts the 2nd–8th harmonics to enhance SNR while retaining critical motion-sensitive spectral features.
• Transformer-Based Signal Modeling: The application of a transformer architecture specifically tailored for MPI signals to model spatio-temporal spectral patterns and cross-axis dependencies for accurate 3D localization.
• Comprehensive Validation: Extensive evaluation on both simulated datasets and real in vitro experimental data (standard, bending, and heartbeat motion profiles), demonstrating robustness across diverse motion conditions.

Results
The proposed method achieves sub-millimeter localization accuracy with significant improvements in inference speed:

• Accuracy: On the bending dataset, the model achieved a minimum $L_2$ error of 0.103 ± 0.092 mm and Mean Absolute Errors (MAE) of 0.039 ± 0.046 mm ($x$), 0.054 ± 0.049 mm ($y$), and 0.060 ± 0.044 mm ($z$). Across all datasets, the MAE ranged from 0.165 mm to 0.655 mm.
• Performance: The optimized inference model achieved a frame latency of 0.55 ms and a throughput of approximately 1,800 frames per second. This throughput significantly exceeds the MPI hardware acquisition rate (approx. 23.2 Hz), confirming real-time capability.
• Ablation Studies: Experiments confirmed that the 2nd–8th harmonic range offers the best trade-off between spatial information and noise robustness. The weighted loss function was shown to be critical for balancing accuracy across axes, particularly improving $z$-axis performance.

Significance
The paper claims that this framework represents a shift from image-reconstruction-dependent tracking to direct signal-to-position mapping. By eliminating the reconstruction pipeline, the method provides improved accuracy, reduced latency, and enhanced robustness under complex motion conditions compared to conventional MPI-guided approaches. The results highlight the potential of harmonic-aware transformer models as efficient, scalable solutions for real-time catheter localization in interventional MPI, enabling high-speed tracking without the computational bottlenecks of traditional methods. The authors note that while the current validation is limited to in vitro phantoms and simulated data, the approach lays the groundwork for future in vivo applications and clinical translation.