Technical Summary: Benchmarking Foundation Models for Mitotic Figure Classification

Problem Statement

In computational pathology, the performance of deep learning models is heavily constrained by the scarcity of high-quality, expert-annotated datasets. Annotating Whole Slide Images (WSIs) is labor-intensive, prone to inter-rater variability, and complicated by domain shifts arising from differences in staining protocols and scanning devices. While self-supervised learning (SSL) has enabled the training of large-scale "foundation models" on vast unlabeled datasets, there is a lack of in-depth analysis regarding how these models perform on specific, fine-grained tasks like mitotic figure classification under varying data regimes. Specifically, it remains unclear how different adaptation strategies (e.g., linear probing vs. parameter-efficient fine-tuning) scale with data availability and how robust these models are when applied to unseen tumor domains (out-of-distribution).

Methodology

The authors conducted a systematic benchmarking study evaluating six state-of-the-art pathology-specific foundation models against traditional baselines.

Datasets

Two publicly available datasets were utilized:

1. CCMCT: A large-scale, single-domain dataset of canine cutaneous mast cell tumors (32 WSIs, ~45k mitotic figures). Used for data scaling experiments.
2. MIDOG 2022: A highly diverse, multi-domain dataset spanning five tumor types (canine and human), multiple species, and various scanners. Used for cross-domain robustness experiments.

Models Evaluated

The study compared six pathology foundation models (Phikon, UNI, Virchow, Virchow2, H-optimus-0, Prov-GigaPath) and a general-purpose Vision Transformer (ViT-S DINOv3) against traditional baselines (ResNet50, ViT-S/B/H). The foundation models varied in backbone size (ViT-B to ViT-G), pretraining algorithms (iBOT, DINOv2), and pretraining data scale (up to 2 billion tiles).

Adaptation Strategies

Three primary adaptation approaches were tested:

1. Linear Probing: The backbone is frozen, and only a linear classifier is trained on top of the extracted features.
2. Low-Rank Adaptation (LoRA): A parameter-efficient fine-tuning technique where low-rank matrices are injected into attention and MLP layers, allowing for selective fine-tuning with minimal trainable parameters.
3. End-to-End Fine-Tuning: Traditional supervised training of the entire model (used for baselines and comparison).

Experimental Design

• Data Scaling: Models were trained on 0.1%, 1%, 10%, and 100% of the training data to assess performance in data-scarce versus data-rich scenarios.
• Cross-Domain: Using MIDOG 2022, models were trained on one tumor domain and tested on all others to evaluate generalization to unseen distributions.
• Metrics: Performance was evaluated using Balanced Accuracy, Weighted F1, and AUROC, with statistical significance tested via Wilcoxon signed-rank tests.

Key Contributions

1. Systematic Benchmarking: The paper provides a comprehensive evaluation of current pathology foundation models specifically for the task of mitotic figure classification, a critical prognostic marker.
2. Adaptation Strategy Comparison: It rigorously compares linear probing against LoRA, demonstrating that LoRA consistently outperforms linear probing, particularly in low-data regimes.
3. Data Scaling Laws: The study quantifies the relationship between training data size and model performance, showing that LoRA-adapted foundation models can achieve near-full-data performance using only 10% of the training data.
4. Cross-Domain Robustness: The work evaluates the ability of these models to generalize across different tumor types and species, highlighting the effectiveness of LoRA in closing the performance gap between in-domain and out-of-domain scenarios.

Results

• Superiority of LoRA: Across both datasets and all data scales, foundation models adapted with LoRA significantly outperformed those using linear probing. In the CCMCT dataset, LoRA-adapted models reached performance levels comparable to 100% data availability using only 10% of the data.
• Outperforming Baselines: At full data scale, LoRA-adapted foundation models (specifically Virchow2 and H-optimus-0) matched or exceeded the performance of fully fine-tuned traditional architectures (ResNet50). However, linear probing alone generally underperformed compared to end-to-end trained baselines.
• Cross-Domain Generalization: In the cross-domain experiment, LoRA-adapted models demonstrated remarkable robustness. Models like H-optimus-0 and Virchow2 achieved out-of-domain AUROC scores of 0.88 and 0.87 respectively, nearly closing the gap with their in-domain performance. In contrast, traditional baselines and linear probing approaches showed significant performance drops in unseen domains.
• Model Specifics: The most recent and largest foundation models (Virchow2, H-optimus-0) generally yielded the best results. The general-purpose ViT-S DINOv3 model underperformed compared to pathology-specific foundation models, indicating that domain-specific pretraining is crucial.

Significance and Claims

The authors claim that their work demonstrates the practical value of parameter-efficient adaptation strategies (LoRA) for clinical applications where annotated data is scarce and heterogeneous. The findings suggest that:

• Foundation models, when adapted via LoRA, offer a superior trade-off between computational cost and performance, enabling robust solutions that generalize across diverse clinical settings.
• The "data scaling laws" observed indicate that high-performance mitotic figure classification is achievable with significantly less labeled data than previously thought necessary for traditional supervised learning.
• While traditional architectures with full fine-tuning remain competitive when ample data is available, LoRA-adapted foundation models provide a more scalable and robust approach for real-world deployment where domain shifts are common.

The paper concludes that while foundation models are poised to transform computational pathology, future work must address their integration into full detection pipelines and further explore hyperparameter optimization for parameter-efficient methods.