Reducing Annotation Burden for Femoral Cartilage Segmentation in Knee MRI via Cross-Sequence Transfer Learning

Technical Summary: Reducing Annotation Burden for Femoral Cartilage Segmentation in Knee MRI via Cross-Sequence Transfer Learning

Problem Statement
Accurate segmentation of femoral cartilage in knee magnetic resonance imaging (MRI) is critical for quantifying osteoarthritis progression. While deep learning (DL) has improved segmentation efficiency, most existing models are trained and evaluated on a single MRI sequence, predominantly the dual-echo steady-state (DESS) sequence from the Osteoarthritis Initiative (OAI). Clinical practice, however, frequently utilizes other sequences, such as proton density–weighted 3D fast spin-echo (Cube), which offer different tissue contrasts and spatial resolutions. Directly applying models trained on one sequence to another often results in performance degradation. Retraining models from scratch on target sequences requires large volumes of manually annotated data, a process that is labor-intensive and often infeasible due to the scarcity of labeled clinical datasets. The core challenge addressed is determining whether cross-sequence transfer learning can achieve segmentation performance comparable to same-sequence training while significantly reducing the number of required target-domain annotations.

Methodology
The study employed a retrospective design utilizing two datasets: 507 DESS images from the OAI and 44 Cube images acquired at the IRCCS Istituto Ortopedico Rizzoli. The Cube cohort was matched to the OAI cohort based on age, sex, body mass index (BMI), laterality, and Kellgren–Lawrence grade (KLG), stratified into non-lesioned and lesioned subgroups.

• Architecture: A modified 2D U-Net was implemented. The architecture featured an encoder with four resolution levels (64 to 512 channels) and a bottleneck (1024 channels), followed by a decoder with skip connections. Modifications included replacing valid convolutions with same-padded convolutions to preserve spatial dimensions, adding batch normalization, and introducing dropout regularization.
• Training Strategy:
  1. Optimization: Hyperparameters (loss function, learning rate, batch size, dropout) and data augmentation parameters were optimized using a Tree-structured Parzen Estimator (Optuna) on the full OAI DESS dataset.
  2. Same-Sequence Baselines: Models were trained from scratch on subsets of DESS and Cube images using subject-level cross-validation (70% train, 20% validation, 10% test) to establish performance baselines.
  3. Cross-Sequence Transfer Learning: Pretrained models were fine-tuned on the target sequence. The encoder was frozen to preserve low-level features, while the bottleneck and decoder were updated. Fine-tuning utilized a reduced learning rate ($1 \times 10^{-5}$) and smaller batch sizes.
• Convergence Analysis: To determine the minimum annotation burden, the training set size for the target domain was incrementally increased (from 1 to the maximum available) while keeping validation and test sets fixed. Convergence was defined as an absolute improvement of less than 0.003 in the Dice Similarity Coefficient (DSC) over four consecutive increments.
• Evaluation: Performance was measured using DSC and Average Surface Distance (ASD). Statistical significance regarding lesion effects was assessed using two-sided Mann–Whitney U tests with Bonferroni correction.

Key Contributions and Results
The study demonstrates that cross-sequence transfer learning can substantially reduce annotation requirements, though performance is highly dependent on the direction of transfer and the specific MRI sequence characteristics.

• Same-Sequence Performance: Training on DESS yielded higher accuracy (DSC: 0.900) compared to Cube (DSC: 0.830), likely due to the superior cartilage-to-background contrast in DESS images.
• Transfer Directionality:
  • Cube-to-DESS: Transfer learning from Cube (source) to DESS (target) achieved a DSC of 0.903, matching the same-sequence DESS baseline (0.900). This performance plateau was reached with only 9 target-domain training subjects.
  • DESS-to-Cube: Transfer from DESS (source) to Cube (target) resulted in a lower DSC (0.802) compared to the same-sequence Cube baseline (0.830). This direction required 24 target-domain subjects to reach a performance plateau, indicating that knowledge transfer from high-contrast to low-contrast sequences is less efficient.
• Lesion Impact: The presence of femoral cartilage lesions did not significantly affect segmentation accuracy in DESS (P ≥ .39). However, lesions significantly reduced accuracy in Cube sequences (DSC 0.805 vs. 0.856 in non-lesioned; P < .001), suggesting that lesion-related performance degradation is sequence-dependent.
• Annotation Efficiency: The study quantified that fewer than 10 manually annotated target-domain subjects are sufficient to achieve performance comparable to same-sequence training when transferring from Cube to DESS.

Significance and Claims
The authors claim that cross-sequence transfer learning is a viable strategy to mitigate the annotation burden for developing femoral cartilage segmentation models across different institutional MRI protocols. The findings highlight that:

1. Direction matters: Transferring from a lower-contrast sequence (Cube) to a higher-contrast sequence (DESS) is more effective than the reverse, likely because the decoder and bottleneck layers can effectively refine features when fine-tuned on well-contrasted data.
2. Sequence dependency: The impact of cartilage lesions on segmentation accuracy varies by sequence, with DESS demonstrating robustness where Cube struggles.
3. Practical utility: By leveraging transfer learning, institutions can develop accurate segmentation models for their specific protocols using a minimal number of manually labeled cases (e.g., <10 subjects), rather than requiring large-scale retraining.

The paper concludes modestly, noting limitations such as the use of a 2D U-Net with 3D reconstruction, the focus solely on femoral cartilage, and the restriction to 3 T scanners, suggesting that generalizability to 1.5 T and 3D architectures requires further investigation.