Temporally Consistent Mitral Annulus Measurements from Sparse Annotations in Echocardiographic Videos

Technical Summary: Temporally Consistent Mitral Annulus Measurements from Sparse Annotations in Echocardiographic Videos

Problem Statement
Accurate assessment of the mitral annulus is critical for diagnosing cardiac conditions, particularly mitral regurgitation and coronary heart disease. Key clinical metrics, such as Mitral Annular Plane Systolic Excursion (MAPSE) and annulus diameter, rely on the precise localization of annulus landmarks at end-systole (ES) and end-diastole (ED). While deep learning has enabled supervised landmark detection, existing methods face two primary challenges:

1. Temporal Inconsistency: Manual frame-by-frame annotation is labor-intensive and prone to inconsistencies. Existing automated methods often focus on replicating sparse human annotations without enforcing temporal smoothness, leading to erratic landmark trajectories that do not reflect physiological motion.
2. Missing Anatomical Landmarks: Due to the limited field of view in ultrasound imaging, target anatomy often disappears due to patient or operator movement. Current methods struggle to recognize when landmarks are out of view, potentially leading to false detections.

Methodology
The authors propose a novel self-supervised approach to achieve temporally consistent landmark localization using only sparse annotations. The method integrates three core components:

1. Network Architecture:

   • The system employs a ResNet-like fully convolutional neural network (CNN) without global pooling, utilizing GroupNorm for small batch training.
   • The network processes three consecutive frames stacked in the channel dimension as input.
   • It utilizes a dual-head output: a classification head that predicts the probability of a landmark being within a specific image patch, and a regression head that infers the distance from the patch center to the landmark.
   • Final landmark coordinates are calculated as the weighted mean of regressed locations, weighted by classification probabilities.

2. Temporal Consistency Loss:

   • To enforce smoothness, the network is trained to predict not only the landmark position for the current frame ($t$) but also the displacement vectors to neighboring frames ($t-1$ and $t+1$).
   • A self-supervised loss term ($L_{temp}$) is introduced, which minimizes the error between the directly predicted location at $t \pm 1$ and the location derived by propagating the prediction from frame $t$ using the predicted displacement vectors.
   • This loss is combined with standard supervised classification and regression losses, scaled by a factor $\beta = 0.5$.

3. Field-of-View Augmentation:

   • To handle missing landmarks, the training pipeline incorporates realistic sector-shaped cropping and rotation augmentations.
   • These augmentations simulate scenarios where landmarks move out of the ultrasound field of view.
   • During training, if a landmark is cropped out, its classification target is nullified (set to zero), and the regression output for that patch is excluded from back-propagation.
   • At inference, if the maximum classification probability for a landmark falls below a specific threshold, the prediction is withheld.

Key Contributions

• Self-Supervised Temporal Regularization: The introduction of a loss term that enforces consistency between neighboring frames without requiring dense annotations, effectively smoothing landmark trajectories.
• Robustness to Missing Data: A specific augmentation strategy that trains the model to recognize and suppress predictions for landmarks that are out of the field of view, rather than hallucinating positions.
• Sparse Annotation Efficiency: The method achieves high accuracy using only sparsely annotated frames (end-systole and end-diastole), reducing the annotation burden compared to dense labeling strategies.

Results
The method was evaluated on a combined dataset of the public CAMUS dataset and a private dataset from Amsterdam University Medical Center (AUMC), containing apical 2-, 3-, and 4-chamber views.

• Localization Accuracy: The proposed method achieved a Mean Absolute Error (MAE) of 2.48 ± 0.07 mm for landmark localization.
• MAPSE Calculation: The MAE for MAPSE was 1.81 ± 0.14 mm, a significant improvement over the baseline method (2.32 ± 0.14 mm, $p < 0.05$).
• Temporal Smoothness: The mean absolute jerk (a measure of trajectory smoothness) was reduced from 3.18 to 0.98 mm/frame³, indicating significantly more physiologically plausible motion tracking.
• Missing Landmark Recognition: The model achieved a 0.99 ROC-AUC for recognizing missing landmarks when tested with augmented data, compared to 0.91 without the specific field-of-view augmentations.
• Annulus Size: The error in measuring the distance between annulus landmarks was 2.46 ± 0.31 mm.

Significance and Claims
The paper claims that the proposed method provides a robust solution for automated mitral annulus tracking that aligns better with expected physiological behavior than previous approaches. By reducing the error in MAPSE calculations by approximately 0.5 mm, the authors suggest the method could improve risk stratification for patients near the clinically relevant cut-off value of 9 mm, which is associated with increased mortality risk.

The authors maintain a modest stance regarding broader comparisons, noting that while their results appear superior to previous work (e.g., Smistad et al.), direct comparison is challenging due to the use of different datasets. They acknowledge that while the method improves upon the state of the art for this specific task, further research is needed to determine its efficacy on other video-based landmark detection tasks. Additionally, they note that the missing landmark recognition was validated on augmented test data rather than a clinical dataset containing naturally occurring missing landmarks, identifying this as an area for future data collection.