Technical Summary: Generalization of Self-Supervised Vision Transformers for Protein Localization Across Microscopy Domains

Problem Statement

Task-specific microscopy datasets are frequently small, posing significant challenges for training deep learning models capable of extracting robust and meaningful features. While self-supervised learning (SSL) has shown promise through pretraining on large, domain-specific datasets, the generalizability of these models across datasets with varying staining protocols, imaging modalities, and channel configurations remains largely unexplored. This limitation is critical for small-scale studies that cannot generate large annotated datasets. Specifically, it is unclear whether SSL models pretrained on natural images or distinct microscopy domains can effectively transfer learned representations to downstream tasks involving different biological contexts and technical specifications.

Methodology

The study investigates the transferability of DINO (self-distillation with no labels) pretrained Vision Transformers (ViT) to a downstream protein localization task on the OpenCell dataset. The experimental design involves three primary components:

1. Datasets and Pretraining Backbones

Three distinct DINO backbones were evaluated:

• ImageNet-1k: A general-purpose backbone pretrained on 1.2 million natural images (RGB).
• HPA (Human Protein Atlas): A domain-specific backbone pretrained on a subset of the HPA dataset, which utilizes four-channel fluorescence microscopy (protein, microtubules, nucleus, endoplasmic reticulum) across 35 cell lines.
• OpenCell: A backbone trained from scratch on the OpenCell dataset, which contains two-channel fluorescence microscopy images (target protein and nucleus) of HEK293T cells with CRISPR-mediated endogenous tagging.

2. Embedding Strategies

To address the mismatch in channel composition between the pretraining data and the target OpenCell data (2 channels vs. 3 or 4 channels), two embedding strategies were employed:

• Channel Replication: Features were extracted independently for each channel and concatenated. This approach requires no additional training but scales linearly with channel count.
• Channel Mapping: Corresponding channels were aligned (e.g., protein to red, nucleus to green), and missing channels were padded with zeros. This allows for the reuse of channel-specific features but relies on semantic compatibility between datasets.

3. Evaluation Protocol

The performance of the extracted embeddings was assessed via a two-stage process:

1. Feature Extraction: Embeddings were generated for the OpenCell dataset using the three pretrained backbones.
2. Supervised Classification: A multi-layer perceptron (MLP) classifier head was trained on these embeddings to predict protein localization across 17 subcellular compartments.
   The evaluation utilized a 5-fold cross-validation on 90% of the data, with a held-out 10% test set. The primary metric was the mean macro F1 score, accounting for the multi-label nature of the classification problem.

Key Results

The study yielded the following quantitative findings (reported as mean macro F1 ± standard deviation):

• Domain-Specific Pretraining Superiority: The model pretrained on the HPA dataset using the channel mapping strategy achieved the highest performance, with a mean macro F1 score of 0.8221 (±0.0062).
• Comparison with Task-Specific Training: Notably, the HPA-pretrained model outperformed the model trained from scratch directly on the OpenCell dataset, which achieved 0.8057 (±0.0090).
• General-Purpose Pretraining: The ImageNet-1k pretrained model using channel mapping performed competitively with a score of 0.7986 (±0.0089) (channel replication) and 0.8057 (±0.0090) (channel mapping), demonstrating that natural image pretraining can still yield robust features for microscopy tasks.
• Embedding Strategy Impact: Channel mapping generally outperformed channel replication, particularly for the HPA backbone (0.8221 vs. 0.7681), suggesting that early fusion of spatial information across channels is more effective than concatenating independent feature vectors.

Significance and Claims

The authors claim that self-supervised methods like DINO, when pretrained on large, domain-relevant datasets (such as HPA) or even general-purpose datasets (ImageNet-1k), can effectively generalize to distinct, smaller microscopy datasets. The key contributions include:

1. Cross-Domain Transferability: The study demonstrates that models pretrained on the HPA dataset generalize well to the OpenCell dataset, even surpassing models trained specifically on the smaller target dataset. This suggests that large-scale domain-specific pretraining can compensate for the lack of task-specific data.
2. Robustness of Natural Image Features: The competitive performance of ImageNet-1k pretrained models indicates that robust morphological features learned from natural images are transferable to fluorescence microscopy, reducing the absolute necessity for domain-specific pretraining in all scenarios.
3. Methodological Insight: The work highlights the importance of input channel handling, showing that channel mapping is a superior strategy for transferring models between microscopy datasets with differing channel counts compared to simple channel replication.

The paper concludes that these findings support the broader adoption of pretrained self-supervised models in bioimage analysis, particularly for studies with limited data availability, and suggests that domain-specific models can generalize effectively to related but distinct microscopy datasets.