Searching for Robust Augmentations to Improve Out-of-Domain Generalization in Dermoscopic Skin Cancer Classification

Technical Summary: Searching for Robust Augmentations to Improve Out-of-Domain Generalization in Dermoscopic Skin Cancer Classification

Problem Statement
Deep learning models for dermoscopic skin lesion classification often achieve high accuracy on internal test sets but suffer significant performance degradation when applied to data from new sources (out-of-domain, or OOD). This "domain shift" arises from variations in imaging devices, illumination, color processing, and capture artifacts across different clinics. While prior research has extensively studied architecture selection and internal validation, there is a lack of systematic analysis regarding which specific classes of data augmentations effectively improve robustness to these source-level shifts. Furthermore, existing augmentation studies often optimize for in-domain validation, failing to address the specific challenge of transferring models to entirely new acquisition environments.

Methodology
The study employs a data-centric approach to improve OOD generalization for a binary malignant-versus-non-malignant classifier.

• Datasets: The training and in-domain evaluation utilize a multi-source collection from the ISIC Archive (BCN20000, Derm12345, HIBA, BALD, MILK10k, ISIC 2016–2020) and Derm7pt. To ensure a rigorous OOD evaluation, the HAM10000 and ISIC 2019–2020 datasets were held out entirely as a source-disjoint test set. An independent, closed clinical dataset (Melanoscope) was used solely for final external validation without influencing model selection.
• Model Architecture: A ConvNeXt-Large backbone, initialized with ImageNet weights, was used with a binary classification head.
• Augmentation Search: The authors conducted a systematic search across three categories of augmentations:
  1. Single augmentations: Individual geometric and photometric transformations.
  2. Photometric combinations: Fixed pairings or triples of color-based operators (e.g., ColorJitter, PlanckianJitter, HEStain).
  3. Composite policies: Configurations inspired by competition solutions and the authors' own findings (e.g., mix, kaggle_strong_hestain).
• Evaluation Metrics: The primary metric was the Area Under the Receiver Operating Characteristic Curve (ROC-AUC). Evaluation included in-domain tests, the source-disjoint OOD test, and the independent Melanoscope set. Statistical significance was assessed using bootstrap confidence intervals and DeLong tests. The study also employed t-SNE visualizations and Grad-CAM to qualitatively assess feature space mixing and attention maps.

Key Contributions

1. Systematic Augmentation Search for OOD: Unlike previous studies that rank augmentations based on in-domain performance, this work explicitly searches for policies that maximize performance on source-level held-out test sets.
2. Identification of Photometric Dominance: The study identifies that photometric transformations (color and intensity adjustments) are the primary drivers of OOD robustness, outperforming purely geometric transformations in this specific domain.
3. The "Mix" Policy: The authors propose a specific composite augmentation policy named mix, which combines photometric and moderate geometric transformations. This policy is shown to yield the highest OOD gains while preserving in-domain accuracy.
4. Rigorous Validation Protocol: The paper distinguishes between screening results (single-run point estimates) and expanded evaluations (multiple seeds, confidence intervals, and independent external datasets), providing a transparent view of the variance and limitations of the findings.

Results

• Out-of-Domain Performance: The mix policy achieved the largest gain on the OOD test set, improving ROC-AUC by +0.0332 relative to the baseline in the screening phase. In the expanded evaluation (9,921 images from held-out sources), the improvement was +0.053 (95% CI: +0.045 to +0.061, p < 0.001).
• In-Domain Performance: The mix policy also maintained a positive effect on in-domain accuracy, improving ROC-AUC by +0.0061, demonstrating that robustness did not come at the cost of internal performance.
• Robustness Across Seeds: The OOD advantage of the mix policy persisted across four different random training seeds, with non-overlapping ROC-AUC ranges (Baseline: 0.761–0.775; Mix: 0.806–0.829).
• External Validation (Melanoscope): On the independent Melanoscope dataset, a single checkpoint showed a sensitivity increase from 0.591 to 0.818. However, the authors note this result is based on only 22 malignant cases, does not reach statistical significance in AUC (p = 0.22), and the advantage disappears when averaged across multiple training seeds.
• Feature Analysis: t-SNE visualizations indicated that the mix policy reduced source-specific clustering in the feature space compared to the baseline. Grad-CAM analysis suggested the augmented model focused more compactly on the lesion rather than peripheral artifacts.
• Diagnosis-Specific Breakdown: The OOD improvement was driven primarily by increased specificity on benign keratoses (reducing false positives) rather than a significant improvement in melanoma recall, which remained low (0.26–0.43) across configurations.

Significance and Limitations
The paper claims that augmentations modeling real sources of domain shift (specifically photometric variations) can be more critical for OOD generalization than maximizing in-domain accuracy or changing the model architecture. The mix policy is presented as a simple, reproducible baseline step that should be considered before clinical validation of decision-support systems.

However, the authors explicitly qualify their claims:

• Selection Bias: The policy was selected using data from the same sources used for the primary OOD evaluation (HAM10000 and ISIC 2019–2020). Therefore, the reported effect size may be optimistically biased, and the results do not constitute independent confirmation of generalization to entirely unseen sources.
• Limited External Evidence: The improvement on the independent Melanoscope dataset was not statistically significant and did not persist across training seeds, indicating that the robustness observed on the ISIC/HAM sources has not yet been proven to transfer to a completely different clinical setting.
• Clinical Utility: While specificity improved, the absolute recall for melanoma remains low, suggesting the model is currently better suited for prioritizing cases for review rather than autonomous diagnosis.

The authors conclude that while the mix policy offers a robust baseline, future work requires a source-disjoint selection protocol (e.g., leave-one-source-out) and validation on larger, diverse external cohorts to confirm unbiased generalization.