Clinical utility of foundation models in musculoskeletal MRI for biomarker fidelity and predictive outcomes

1. Problem Statement

Musculoskeletal (MSK) disorders are a leading cause of global disability, yet routine MSK radiology workflows rarely produce standardized quantitative biomarkers. Current clinical practice relies on manual segmentation by experts to measure structures like cartilage, bone, and muscle across hundreds of MRI slices. This process is:

• Time-intensive and expensive: Requiring significant expert labor.
• Non-scalable: Difficult to implement across diverse patient populations.
• Variable: Prone to inter-reader variability, especially in low-contrast or degenerated anatomy.
• Protocol-dependent: Heterogeneity in MRI scanners, sequences, and sites complicates the creation of universal quantitative tools.

The paper addresses the need to convert routine MSK MRI scans into standardized, quantitative biomarkers that can drive clinical decision support, triage, and longitudinal risk prediction without relying on manual annotation.

2. Methodology

The authors developed a modular, open-source framework that bridges automated segmentation with clinical decision-making.

A. Dataset and Annotations

• Data: 12 heterogeneous MSK MRI datasets totaling 913 scans covering five anatomies: knee, hip, lumbar spine, shoulder, and thigh.
• Variety: Includes both 2D clinical protocols (spin-echo) and high-resolution 3D research protocols (CUBE, DESS, quantitative T1ρ/T2) at 1.5T and 3T (Siemens and GE).
• Labels: Expert-annotated masks for cartilage, bone, muscle, fat, nerves, and intervertebral discs.

B. Foundation Model Architecture

The system utilizes Promptable Foundation Models (Segment Anything Model family) coupled with an automated detection pipeline:

1. Models Evaluated: SAM (ViT-B), MedSAM (ViT-B), and SAM2 (sam2_hiera_base_plus).
2. Prompting Strategy: Instead of manual prompts, a YOLOv8m detector generates slice-wise bounding boxes to prompt the segmentation models (AutoLabel workflow).
3. Fine-Tuning: Models were fine-tuned using a unified pipeline with:
   • Data Efficiency: Training on small subsets (5–40 subjects) to full datasets.
   • Ablation Studies: Comparing frozen vs. unfrozen encoders and varying bounding-box shift augmentations (±20 px) to simulate detection variability.
   • Strategies: Single-dataset training, anatomy-grouped training, and a pooled "mskSAM" model trained across all datasets.

C. Biomarker Extraction

Quantitative measurements were derived directly from the segmentation masks:

• Structural: Cartilage thickness (medial-axis transform), disc height, muscle volume, bone volume.
• Compositional: Mean T1ρ and T2 relaxation times.
• Validation: Agreement between model-derived and expert measurements was assessed using Intraclass Correlation Coefficients (ICC), Bland-Altman analysis, and regression.

D. Clinical Applications

Two downstream applications were developed using the validated biomarkers:

1. Automated Triage Cascade (Knee): A three-stage pipeline (Stage A: Normal screen; Stage B: Cartilage/Bone filter; Stage C: Tissue localization) to route abnormal cases for review, aiming to reduce radiologist workload while maintaining sensitivity.
2. Longitudinal Risk Modeling: Using data from the Osteoarthritis Initiative (OAI), landmark models (48-month horizon) predicted Total Knee Replacement (TKR) and Incident Osteoarthritis (OA) using serial biomarker trajectories (thickness changes over time).

3. Key Contributions

1. Modular "Measurement-First" Architecture: The system decouples the segmentation backbone from the biomarker extraction and decision layers. This allows swapping segmentation models (e.g., from SAM to a future model) without re-engineering the clinical decision logic.
2. Scalable Automation (AutoLabel): Demonstrated that detector-generated bounding boxes can replace manual prompts, reducing the Dice score by only 0.05–0.11 while enabling population-scale deployment without annotation bottlenecks.
3. Clinical-Grade Biomarker Fidelity: Showed that fine-tuned foundation models achieve high concordance with expert measurements (ICCs up to 0.99 for cartilage thickness and muscle volume), validating their use as quantitative inputs for precision medicine.
4. Operational and Predictive Utility: Proved the framework's utility in two distinct timescales:
   • Immediate: Reducing triage verification time from ~133 hours to ~1.7 hours per 1,000 scans (at 90% specificity).
   • Long-term: Forecasting knee replacement and OA incidence with favorable net benefit in decision-curve analysis.

4. Key Results

• Segmentation Performance:
  • Fine-tuning Impact: Fine-tuning increased median Dice scores by +0.14 across all datasets.
  • Model Comparison: SAM2 generally outperformed SAM and MedSAM in zero-shot settings, particularly on complex anatomy (e.g., Shoulder CUBE: 0.68 vs. 0.55 for SAM). However, MedSAM showed the largest relative gains after fine-tuning due to its medical pre-training.
  • Robustness: The pooled fine-tuned model (mskSAM2) achieved Dice scores of 0.92–0.96 across diverse tissues and protocols, including low-contrast regions and undersampled sequences.
• Biomarker Agreement:
  • Cartilage Thickness: ICC ≥ 0.89 (up to 0.996); Bland-Altman limits of agreement ±0.18 mm (comparable to test-retest precision).
  • Muscle Volume: ICC 0.99–1.00.
  • Disc Height: ICC 0.97.
• Triage Performance:
  • The cascade successfully filtered ~90% of normal knees at 90% specificity, reducing the verification workload to 1.7 hours per 1,000 scans (assuming 2 min/check).
  • Sensitivity for pathology was maintained at preset thresholds.
• Risk Prediction:
  • TKR Prediction: Random Forest model achieved an AUC of 0.76 (95% CI 0.72–0.80) for predicting knee replacement 48–96 months out, significantly outperforming demographics-only baselines (AUC 0.60).
  • Key Features: Lateral tibial cartilage thickness and lateral meniscus features were the strongest predictors (SHAP analysis).

5. Significance and Impact

• Bridging the Gap to Clinical Practice: This work moves foundation models from "technical benchmarks" to "clinical utility" by prioritizing biomarker fidelity over raw segmentation metrics. It demonstrates that AI can generate measurements reliable enough to drive clinical decisions.
• Workflow Optimization: The proposed triage system offers a practical solution to radiologist burnout by automating the screening of normal cases, allowing experts to focus on complex pathology.
• Precision Medicine: By enabling the extraction of standardized longitudinal biomarkers, the framework supports proactive care, risk stratification, and personalized treatment planning for osteoarthritis.
• Sustainability: The model-agnostic design ensures that as new, better segmentation backbones emerge, the clinical pipeline (biomarker extraction and risk modeling) remains stable and reusable.
• Open Science: The authors released the code, configuration files, and fine-tuned weights, fostering independent validation and further development in the MSK imaging community.

Limitations Noted:

• Most data came from a single health system (UCSF), requiring external validation.
• The system was not stress-tested on severe motion artifacts, metal implants, or post-operative scans.
• Automation relies on detector-generated boxes, which may fail on anatomy outside the training distribution.

In conclusion, the paper validates a pathway where automated measurement leads to clinical decision support, proving that foundation models can be operationalized to improve both the efficiency of radiology workflows and the accuracy of patient risk stratification.