Multimodal B-Mode and Contrast-Enhanced Ultrasound Radiomics With Two-Stage Supervised Contrastive Representation Learning for Differentiating Benign and Malignant Breast Lesions

Technical Summary: Multimodal B-Mode and Contrast-Enhanced Ultrasound Radiomics with Two-Stage Supervised Contrastive Representation Learning

Problem Statement
Accurate differentiation between benign and malignant breast lesions remains a critical clinical challenge. While B-mode ultrasound (BUS) is a routine first-line examination, its diagnostic accuracy is limited by reliance on visual assessment of morphological features (shape, margin, echogenicity), leading to potential unnecessary biopsies or missed malignancies. Contrast-enhanced ultrasound (CEUS) provides complementary information regarding lesion microvascular perfusion and enhancement patterns but is not always effectively integrated with BUS data. Conventional radiomics approaches often concatenate features from multiple modalities into a single high-dimensional pool, which may blur modality-specific distributions and fail to explicitly learn the agreement or divergence between BUS and CEUS signals. This study addresses the need for a robust method to fuse complementary BUS and CEUS radiomics features without reducing them to a simple feature pool, aiming to improve benign-malignant discrimination.

Methodology
This retrospective single-center study analyzed 486 women with pathologically confirmed breast lesions (310 benign, 176 malignant) who underwent both BUS and CEUS. The workflow involved manual 3D segmentation of regions of interest (ROIs) by experienced radiologists, followed by radiomics feature extraction using PyRadiomics. Features with high reproducibility (Intraclass Correlation Coefficient > 0.75) were retained.

The core of the study was the development and comparison of six models:

1. Conventional Radiomics Models: Support Vector Machine (SVM) classifiers trained on BUS-only, CEUS-only, and concatenated BUS+CEUS features.
2. Neural Network Fusion: A Multilayer Perceptron (BUS+CEUS-MLP) trained on concatenated features to learn nonlinear relationships.
3. Radiomics-TSMCR (Primary Model): A radiomics-based adaptation of Two-Stage Supervised Multimodal Contrastive Representation Learning (TSMCR). This architecture features:
   • Stage 1: Modality-specific encoders that transform BUS and CEUS features into separate latent representations, preserving distinct distributions (morphology vs. perfusion). A batch-wise supervised cross-modal contrastive objective aligns same-class representations and separates different classes using pathologic labels, without direct feature concatenation.
   • Stage 2: A shared encoder learns higher-level cross-modal features. The representations from both stages are concatenated into a multilevel vector and classified using a Deep Support Vector Machine (DSVM)-inspired margin-based classifier.
4. ResNet18-TSMCR: An image-level comparator using paired BUS and CEUS image patches (128x128) processed through dual ResNet18 branches within the same TSMCR framework, serving as an exploratory benchmark against handcrafted radiomics.

Model performance was evaluated using the Area Under the Receiver Operating Characteristic Curve (AUC), DeLong testing, calibration curves, Hosmer-Lemeshow tests, Decision Curve Analysis (DCA), and SHapley Additive exPlanations (SHAP) for interpretability. Post-hoc recalibration using Platt scaling and isotonic regression was applied to the primary model.

Key Results

• Performance: The Radiomics-TSMCR model achieved the highest internal validation AUC of 0.896 (95% CI, 0.838–0.945). It significantly outperformed BUS-SVM (0.833), CEUS-SVM (0.815), BUS+CEUS-SVM (0.825), and BUS+CEUS-MLP (0.863) (all P < 0.05).
• Comparison with Image-Level Learning: Radiomics-TSMCR showed comparable performance to the image-level ResNet18-TSMCR model (AUC 0.892), with no statistically significant difference (P = 0.9102).
• Calibration: While the model demonstrated strong discrimination, the raw output showed a significant lack of fit in the Hosmer-Lemeshow test. Isotonic regression improved calibration metrics (Expected Calibration Error, Brier score, Log loss) compared to Platt scaling, though the Hosmer-Lemeshow test remained significant, indicating the model functions best as a risk-ranking tool rather than a fully calibrated probability estimator.
• Interpretability: SHAP analysis revealed balanced contributions from both modalities, with BUS features contributing 54.95% and CEUS features 45.05% to the total mean absolute SHAP value. The top features included wavelet texture, shape, and gray-level dependence information, confirming the integration of morphological and enhancement heterogeneity data.

Significance and Claims
The paper claims that the multimodal Radiomics-TSMCR model demonstrates favorable performance for differentiating benign from malignant breast lesions in an internal validation setting. Its primary significance lies in the successful adaptation of a two-stage supervised contrastive learning framework to radiomics data. The authors posit that this approach effectively fuses BUS and CEUS information by preserving modality-specific distributions while learning a shared representation guided by pathology, outperforming conventional concatenation methods (SVM, MLP).

The comparable performance between the handcrafted radiomics model (Radiomics-TSMCR) and the image-level deep learning model (ResNet18-TSMCR) suggests that the TSMCR strategy is applicable to both feature-based and image-based representations. Furthermore, the balanced SHAP contributions support the premise that BUS and CEUS provide complementary diagnostic information. The authors conclude that while the model shows strong discrimination, its current limitations—specifically the lack of external validation and imperfect calibration—necessitate that it be viewed as a risk-ranking tool requiring further prospective, multicenter validation before clinical deployment as an absolute probability estimator.