Adaptation of Weakly Supervised Localization in Histopathology by Debiasing Predictions

1. Problem Statement

The paper addresses the challenge of Weakly Supervised Object Localization (WSOL) in digital pathology when models are deployed across different domains (e.g., different organs, institutions, or staining protocols).

• The Core Issue: When a WSOL model trained on a source domain is applied to a target domain without retraining (Source-Free Domain Adaptation or SFDA), performance degrades significantly due to domain shift.
• The Specific Failure Mode: Under severe shifts (e.g., cross-organ or cross-center), the source model develops a prediction bias toward dominant classes. This leads to highly skewed pseudo-label distributions in the target domain.
• The SFDA Trap: Traditional SFDA methods rely on self-training using these pseudo-labels. Because the initial predictions are biased, self-training reinforces this bias over iterations, causing the model to collapse into predicting a single dominant class. This degrades both image-level classification and pixel-level localization (Region of Interest detection).

2. Methodology: SFDA-DeP

The authors propose SFDA-DeP (Source-Free Domain Adaptation with Debiasing Predictions), a method inspired by Machine Unlearning. Instead of simply reinforcing confident predictions, the method iteratively identifies and "forgets" biased assignments to reshape decision boundaries.

Key Components:

A. Iterative Bias Correction (Forget & Retain Strategy)
The method treats adaptation as an iterative process to correct class imbalance:

1. Identification of Dominant Classes: The system identifies samples predicted as belonging to over-represented (dominant) classes in the target domain.
2. Set Partitioning:
   • Forget Set ($B_f$): Contains the top $\rho$ most uncertain samples (high normalized entropy) from the dominant classes. These are samples near the decision boundary that are likely misclassified due to bias.
   • Retain Set ($B_r$): Contains the remaining samples, primarily those with confident and reliable predictions.
3. Loss Functions:
   • Retain Loss ($L_{retain}$): Standard cross-entropy applied to the Retain Set to preserve reliable predictions.
   • Forget Loss ($L_{forget}$): A novel loss applied to the Forget Set that maximizes the cross-entropy for the currently predicted dominant class. This effectively penalizes the model for predicting the dominant class on these uncertain samples, forcing it to "forget" the old bias and explore other classes.
   • Dynamic Resampling: The Forget and Retain sets are rebuilt every $m$ epochs based on the current model state to prevent overfitting to incorrect pseudo-labels and adapt to shifting boundaries.

B. Pixel-Level Localization Supervision
To ensure the model maintains spatial discrimination despite domain shifts:

• A lightweight pixel-level classifier is jointly optimized alongside the image classifier.
• Pseudo-label Generation: Class Activation Maps (CAMs) from the source model are used to generate partial pixel-wise pseudo-labels (foreground vs. background).
• Selection: Only the top-$\rho_{loc}$ fraction of samples with the lowest entropy are selected to generate these masks.
• Loss: A binary cross-entropy loss ($L_{loc}$) is applied to train the pixel classifier, anchoring the localization features and preventing them from drifting during adaptation.

C. Total Objective
The total loss function combines these elements:
$$L = \lambda_{retain}L_{retain} + \lambda_{forget}L_{forget} + \lambda_{loc}L_{loc}$$

3. Key Contributions

1. Identification of Bias Amplification: The paper explicitly identifies that the failure of existing SFDA methods in histopathology is due to the amplification of initial prediction biases, rather than just feature distribution shifts.
2. SFDA-DeP Framework: Introduces a novel SFDA approach that integrates Machine Unlearning principles. By selectively penalizing uncertain predictions from dominant classes, it breaks the cycle of bias reinforcement.
3. Joint Optimization: Combines class-level debiasing with pixel-level localization supervision (using CAM-derived priors) to restore discriminative features for both tasks simultaneously.
4. Dynamic Adaptation: The iterative rebuilding of the "Forget" and "Retain" sets ensures the model adapts to the evolving decision boundaries of the target domain.

4. Experimental Results

The method was evaluated on three histopathology benchmarks involving cross-organ and cross-center shifts:

• Datasets: GlaS (Colon), CAMELYON16, and CAMELYON17 (Breast lymph-node metastasis).
• Baselines: Compared against Source-only, SFDA-DE, CDCL, ERL, and RGV.
• Backbones: Tested on ResNet-50 (CNN) and DeiT-Tiny (Transformer) with WSOL models like PixelCAM, SAT, and DeepMIL.

Key Findings:

• Classification Accuracy: SFDA-DeP consistently outperformed state-of-the-art baselines. For example, on the PixelCAM model, it achieved an average classification accuracy of 67.1% compared to 54.6% for SFDA-DE.
• Localization Performance: Significant improvements were seen in Pixel-level Average Precision (PxAP). PixelCAM improved from 28.0 (SFDA-DE) to 44.1 (SFDA-DeP) on average.
• Bias Mitigation: Qualitative analysis (CAMs) showed that SFDA-DeP produced coherent activations focused on tumor tissue, whereas baselines often collapsed to highlighting background or spurious areas due to class bias.
• Robustness: The method was particularly effective in scenarios where baselines failed (e.g., CAMELYON17 centers where baselines hovered around 50% accuracy due to class collapse).

5. Significance

• Clinical Reliability: By preventing the model from collapsing into a single dominant class, SFDA-DeP ensures that rare but critical conditions (e.g., specific tumor types) are not missed during cross-center deployment.
• Interpretability: The restoration of accurate pixel-level localization is crucial for pathologists to trust AI-assisted diagnostics. The method ensures that the "highlighted" regions actually correspond to pathology, not artifacts of domain shift.
• Generalizability: The approach offers a practical solution for deploying deep learning models in digital pathology without requiring access to source data or new labeled target data, addressing a major bottleneck in clinical AI adoption.

In summary, SFDA-DeP transforms the SFDA process from a passive reinforcement of existing biases into an active, iterative correction mechanism, significantly improving the robustness of weakly supervised models in real-world histopathology applications.