Vision Transformers for Preoperative CT-Based Prediction of Histopathologic Chemotherapy Response Score in High-Grade Serous Ovarian Carcinoma

1. Problem Statement

High-Grade Serous Ovarian Carcinoma (HGSOC) is a heterogeneous and aggressive cancer often diagnosed at advanced stages. Treatment typically involves either primary cytoreductive surgery (PCS) or Neoadjuvant Chemotherapy (NACT) followed by delayed surgery. The Chemotherapy Response Score (CRS) is the gold-standard histopathological biomarker for assessing response to NACT, stratifying patients into:

• CRS 1: Minimal/no response.
• CRS 2: Partial response.
• CRS 3: Complete/near-complete response (associated with better survival).

The Challenge: CRS is only available post-operatively after the omentum is resected. This limits its utility in pre-treatment decision-making. Multidisciplinary Teams (MDTs) currently lack non-invasive, pre-treatment tools to predict which patients will achieve a complete response (CRS 3) versus a poor response (CRS 1–2). This uncertainty complicates the choice between NACT and immediate surgery. The authors aim to develop a pre-treatment predictive model using routine CT imaging and clinical data to estimate CRS, thereby aiding MDT discussions.

2. Methodology

The authors propose a 2.5D Multimodal Deep Learning Framework that integrates pre-treatment CT imaging with clinical variables.

A. Data Preprocessing & Feature Extraction

• Input Data: Pre-treatment contrast-enhanced CT scans (abdomen/pelvis) and clinical variables (Age, CA-125).
• Segmentation: Omental lesions were segmented using a deep learning model (OvSeg) or manual radiologist annotation.
• 2.5D Slice Selection:
  • The 3D CT volume was masked by the segmentation mask.
  • The three axial slices with the highest density of lesion pixels were extracted.
  • These slices were stacked along the channel dimension to create a 2.5D multi-channel input ($224 \times 224 \times 3$), preserving anatomical order.

B. Model Architecture

The architecture ($M$) consists of three main components:

1. Vision Encoder (ViT):
   • Uses a pre-trained DINOv3-small Vision Transformer (trained on natural images, not medical data).
   • Processes the 2.5D stacked slices to capture spatial relationships across the most informative lesion regions.
   • Extracts the [CLS] token as a compact volume-level embedding ($z_V \in \mathbb{R}^{384}$).
2. Clinical Encoder ($M_c$):
   • A neural network module encodes clinical features (Age, CA-125) into a learned representation ($z_{clin}$).
3. Intermediate Fusion & Classification:
   • The visual embedding ($z_V$) and clinical embedding ($z_{clin}$) are concatenated.
   • Passed through an MLP-based classification head with ReLU activations and Batch Normalization.
   • Output: A binary probability ($\hat{p}$) distinguishing Non-Complete Response (CRS 1–2) vs. Complete Response (CRS 3).

C. Training Strategy

• Transfer Learning: The ViT encoder weights were frozen; only the classification head and clinical encoder were fine-tuned.
• Loss Function: Weighted Binary Cross-Entropy (WBCE) to handle class imbalance.
• Optimization: AdamW optimizer with linear warm-up and decay.
• Thresholding: The decision threshold was optimized to prioritize Precision (minimizing False Positives) to avoid overestimating chemosensitivity, which could lead to suboptimal treatment choices.

3. Key Contributions

1. First DL-based Pre-treatment Predictor: This is the first study to predict CRS scores using only pre-treatment CT and clinical data for HGSOC.
2. Multimodal Fusion: Demonstrates that integrating clinical metadata (Age, CA-125) with Vision Transformer-derived imaging features significantly improves generalizability compared to imaging-only models.
3. 2.5D Approach: Introduces a specific 2.5D input strategy (stacking top 3 lesion-dense slices) that leverages pre-trained natural image transformers (DINOv3) effectively for medical volumetric data.
4. Clinical Utility Focus: The model is designed as a decision-support tool to assist MDTs in borderline cases, specifically by providing high-confidence estimates of "good responders" to support NACT selection.

4. Results

The model was evaluated on an internal cohort (IEO, $n=271$) and an external validation cohort (OV04, $n=70$).

• Internal Performance (IEO Test Set, $n=41$):

  • ROC-AUC: 0.95
  • Accuracy: 95% (at optimized threshold $\tau_{opt}=0.69$)
  • Precision: 80%
  • Recall: 80%
  • Note: At the default 0.5 threshold, recall was 100% but precision was low (36%). The optimized threshold successfully reduced False Positives from 25% to 3%.

• External Performance (OV04 Test Set, $n=70$):

  • ROC-AUC: 0.68
  • Accuracy: 67%
  • Precision: 75%
  • Observation: While AUC dropped on the external set (indicating domain shift), the Precision remained high (75%), confirming the model's ability to reliably identify complete responders without excessive false alarms.

• Ablation Study:

  • Imaging Only: AUC 0.89 (IEO) but dropped to 0.47 (OV04), showing poor generalizability.
  • Full Multimodal (CT + Age + CA-125): Achieved the best performance on both sets (AUC 0.95 on IEO, 0.68 on OV04).
  • Conclusion: Clinical variables act as stabilizing factors, mitigating the impact of morphological differences between datasets.

5. Significance and Future Directions

• Clinical Impact: The model offers a non-invasive, pre-treatment method to estimate treatment response. High precision in predicting CRS 3 can increase MDT confidence in selecting NACT for patients where complete upfront surgery is uncertain.
• Robustness: The study highlights the necessity of multimodal data to overcome domain shifts between institutions (e.g., different CT protocols or patient demographics).
• Future Work:
  • Expanding training to multi-center datasets.
  • Incorporating post-treatment dynamic biomarkers (e.g., CA-125 decline rates/KELIM index).
  • Extending imaging analysis to include ovarian and adnexal lesions, not just omental deposits.
  • Improving standardization in clinical and pathological reporting.

In summary, this paper presents a robust, multimodal deep learning framework that successfully leverages Vision Transformers and clinical data to predict chemotherapy response in ovarian cancer, offering a promising tool for personalized treatment planning.