Comparing the Performance of Foundation Model Derived Embeddings with Traditional Approaches for Distant Metastasis Prediction in Head and Neck Cancer

Technical Summary: Comparing Foundation Model Derived Embeddings with Traditional Approaches for Distant Metastasis Prediction in Head and Neck Cancer

Problem Statement
Early prediction of distant metastasis (DM) in head and neck cancer (HNC) is critical for clinical management, enabling timely interventions that may improve survival. While machine learning approaches utilizing medical imaging are widely explored, traditional methods often rely on prior knowledge of the region of interest (ROI), specifically tumor segmentations. These traditional approaches, such as radiomics and deep learning, face significant limitations:

• Radiomics: Requires accurate manual or automated contouring of the ROI. Manual delineation is time-consuming and subjective, while automated segmentation can be unreliable if trained on limited or biased reference contours.
• Deep Learning: Generally requires a bounding box or ROI definition, necessitating knowledge of tumor location. Furthermore, the localized nature of these ROIs may ignore prognostic information present in surrounding tissues.
• General Constraints: Both approaches introduce user-dependent variability, increase the time to treatment due to the need for expert delineation, and often require extensive annotated datasets and significant computational resources for 3D network training.

Methodology
The study utilized preoperative CT images from 2,327 HNC patients in the RADCURE dataset (375 with distant failures within 2 years). The dataset was split into a training/validation set (1,819 patients) and a hold-out test set (508 patients).

Three distinct feature extraction strategies were compared:

1. CT Foundation Embeddings: Utilized the "CT Foundation" model, a foundation model trained on over 500,000 multi-institutional CT images. This model processes full CT volumes (with axial reduction for storage efficiency but without tumor cropping) and outputs a 1,408-dimensional embedding vector. It requires no ROI delineation or manual segmentation.
2. Radiomics: Extracted 1,316 handcrafted features (first-order, shape, texture matrices) from the Gross Tumor Volume (GTVp) using the pyradiomics library. This required manual GTVp contours provided in the dataset.
3. Deep Learning (Vision Transformer - ViT): A ViT model trained from scratch on 80x80x80 voxel crops centered on the GTVp. This required GTVp contours and standard preprocessing (resampling, HU truncation, normalization).

Model Architecture and Training

• Classifiers: Two classifier types were employed: a 4-layer Multi-Layer Perceptron (MLP) and a Support Vector Machine (SVM).
• Inputs: All models incorporated 14 clinical features (e.g., age, sex, T/N stage, HPV status) concatenated with the image-derived features.
• Training Protocol:
  • MLP: Trained using 5-fold cross-validation with an Adam optimizer and a plateau-based learning rate scheduler. An ensemble of the 5 folds was created by averaging probabilities.
  • SVM: Trained using grid search and 5-fold cross-validation, preceded by a minimum Redundancy Maximum Relevance (mRMR) feature selection algorithm.
• Evaluation: Models were evaluated on the hold-out test set using Receiver Operating Characteristic Area Under the Curve (AUC), Average Precision (AP), Sensitivity (SEN), and Specificity (SPE). Statistical significance was assessed using a one-sided DeLong test with False Discovery Rate (FDR) correction.

Key Results
The study compared the performance of foundation model embeddings against traditional radiomics and deep learning approaches:

• MLP Performance: The CT Foundation embedding model achieved the highest AUC among single-feature inputs at 0.791 (95% CI: 0.736–0.846). This outperformed the Radiomics model (AUC 0.772) and the ViT model (AUC 0.753).
• Comparison with Combined Features: A model combining Radiomics and ViT features achieved an AUC of 0.794. Statistical analysis showed no significant difference between the CT Foundation model and this combined model (corrected p-value = 0.558).
• SVM Performance: SVM models showed comparable AUCs to their MLP counterparts (Foundation: 0.791, Radiomics: 0.792). However, MLP models demonstrated a better balance between sensitivity and specificity (SEN/SPE ratio of ~0.9) compared to SVM models (ratio of ~1.8), which tended to favor specificity.
• Efficiency: The CT Foundation approach reduced the number of trainable parameters in the downstream task by nearly 5 orders of magnitude (from ~97 million in a 3D network to ~2,600), allowing efficient training on CPUs.

Significance and Claims
The paper claims that foundation model-derived embeddings offer a promising alternative to traditional radiomics and deep learning for DM prediction in HNC. The primary significance lies in the reduction of domain expertise and data annotation requirements:

• Automation: The CT Foundation model eliminates the need for time-consuming, expert-dependent tumor segmentation or ROI delineation.
• Scalability: By processing full volumes without cropping and outputting compact embeddings, the approach is more accessible to researchers with limited computational resources (e.g., no need for GPUs for feature extraction).
• Performance: The foundation model achieved performance comparable to, or slightly better than, traditional methods, and statistically indistinguishable from a complex combination of radiomics and deep learning features.

The authors conclude that while challenges regarding API accessibility and cloud-based data handling exist, the ability to generate high-performance predictive features with minimal preprocessing makes foundation models a scalable solution for medical imaging tasks, particularly in settings lacking extensive annotated datasets or expert curation.