💻 computer science

A Comprehensive Benchmark of Histopathology Foundation Models for Kidney Histopathology

This paper systematically benchmarks 11 histopathology foundation models on kidney-specific tasks, revealing that while they perform moderately well on coarse meso-scale morphological features, they struggle with fine-grained microstructural discrimination and prognostic inference, thereby highlighting the urgent need for kidney-specific, multi-stain foundation models.

Original authors: Harishwar Reddy Kasireddy (University of Florida), Patricio S. La Rosa (University of Florida, Bayer Company), Akshita Gupta (University of Florida), Anindya S. Paul (University of Florida), Jamie L.

Published 2026-03-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Harishwar Reddy Kasireddy (University of Florida), Patricio S. La Rosa (University of Florida, Bayer Company), Akshita Gupta (University of Florida), Anindya S. Paul (University of Florida), Jamie L. Fermin (University of Florida), William L. Clapp (University of Florida), Meryl A. Waldman (National Institutes of Health), Tarek M. El-Ashkar (Indiana University School of Medicine), Sanjay Jain (Washington University School of Medicine), Luis Rodrigues (Universidade de Coimbra), Kuang Yu Jen (University of California Davis), Avi Z. Rosenberg (Johns Hopkins University), Michael T. Eadon (Indiana University School of Medicine), Jeffrey B. Hodgin (University of Michigan), Pinaki Sarder (University of Florida)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart student who has spent years studying millions of pictures of cancer. This student is so good at spotting tumors that they can identify them in a split second. In the world of medical AI, this student is called a Foundation Model.

Now, imagine a doctor asks this student: "Great job with cancer, but can you also help me diagnose kidney disease?"

Kidney disease is tricky. It's not just about finding a tumor; it's about spotting tiny changes in the kidney's plumbing, checking for inflammation, and predicting how a patient's health will change over years. The doctor doesn't know if the student, who only studied cancer, actually knows enough about kidneys to be helpful.

This paper is essentially a report card for 11 of these "super-students." The researchers put them through a series of tests to see how well they can handle kidney problems, even though they were trained mostly on cancer.

Here is the breakdown of what they found, using some everyday analogies:

1. The Setup: The "Cancer Expert" vs. The "Kidney Doctor"

The researchers took 11 different AI models (the students) that had been trained on huge databases of cancer slides. They then tested them on 11 different kidney tasks.

The Tasks: Some were easy (like spotting a big, obvious blockage in a pipe). Some were hard (like spotting a tiny scratch on a pipe wall). Some were very complex (like predicting if a patient will get better or worse in the future).
The Colors: Kidney doctors use different colored dyes (stains) to see different things. The AI was tested on slides with different colors, even though it was mostly trained on one specific color (pink and purple).

2. The Results: Where They Shined

The Good News: When the task was about spotting big, obvious changes, the AI did surprisingly well.

The Analogy: Imagine you are looking at a forest. If you ask the AI, "Is there a giant fallen tree blocking the path?" it says, "Yes, absolutely!" with high confidence.
The Reality: The AI was great at spotting large-scale kidney damage, like when the kidney's filtering units (glomeruli) are completely scarred over or when there is a massive amount of inflammation. It didn't matter if the slide was stained with a different color dye; the AI recognized the "shape" of the damage.

3. The Results: Where They Stumbled

The Bad News: When the task required fine details or future predictions, the AI struggled.

The Analogy: Now, imagine asking the AI, "Can you see the tiny scratch on the bark of that specific tree?" or "Can you predict if this forest will survive a drought next year based on the leaves?" The AI gets confused.
The Reality:
- Tiny Details: The AI was bad at spotting very subtle changes, like tiny spikes on the kidney's basement membrane or slight narrowing of blood vessels. These are like looking for a needle in a haystack.
- Molecular Mysteries: The AI couldn't tell the difference between a "healthy" kidney tube and a "sick" one if the change was based on invisible molecular signals rather than visible shape changes.
- Predicting the Future: The AI was terrible at predicting how a patient's kidney function would drop over the next few years. It's like trying to predict the stock market crash just by looking at a picture of a factory floor.

4. The "Color Blindness" Test

The researchers also tested if the AI was "color blind" to the different dyes used by pathologists.

The Analogy: If you show a picture of a red apple to someone who only studied green apples, can they still recognize it as an apple?
The Reality: The AI was surprisingly good at this! It learned the shape of the cells so well that it didn't get confused when the colors changed. However, if you added "noise" (like static on an old TV), the AI got confused.

5. The Big Conclusion

The paper concludes that these "Cancer Experts" are great for a quick, rough check of kidney health. They can tell a doctor, "Hey, this kidney looks pretty damaged," which is a good starting point.

But, they are not ready to be the final decision-maker for complex kidney cases. They can't yet replace the human pathologist when it comes to:

Spotting the tiniest, most subtle signs of disease.
Predicting how a patient will respond to treatment.
Understanding the invisible molecular changes happening inside the cells.

The Takeaway

The authors are saying: "We have built a powerful engine, but we need to tune it specifically for kidneys. We need to teach it to look at kidney-specific colors, kidney-specific tiny details, and combine its vision with other data (like blood tests and patient history) before we can trust it to make life-or-death decisions for kidney patients."

They also released a free toolkit (like a recipe book) so other scientists can run these same tests to make sure their own AI models are ready for the real world.

1. Problem Statement

While Histopathology Foundation Models (HFMs) trained on large-scale cancer datasets (e.g., TCGA, CPTAC) have revolutionized computational pathology, their applicability to non-cancerous chronic kidney diseases (CKD) remains underexplored.

The Gap: CKD often coexists with renal cancers (RCC, urothelial carcinoma), suggesting potential feature overlap. However, kidney histology relies on multiple staining protocols (PAS, PASM, H&E, IHC) to visualize distinct nephron structures, whereas most HFMs are trained primarily on H&E-stained cancer slides.
The Challenge: It is unclear if current HFMs can:
1. Generalize from cancer to non-cancer kidney pathologies.
2. Transfer effectively across different staining protocols (cross-stain generalization).
3. Capture fine-grained microstructural details or complex molecular phenotypes required for prognosis, or if they are limited to coarse meso-scale features.

2. Methodology

Models Evaluated

The study benchmarks 11 publicly available HFMs pre-trained on >100,000 Whole Slide Images (WSIs). These models utilize Vision Transformer (ViT) backbones and Self-Supervised Learning (SSL) strategies (mostly DINO/DINOv2).

Models: UNI, UNI2-h, Virchow, Virchow2, H-optimus-0/1, Hibou-B/L, SP22M/SP85M, and Prov-Gigapath.
Pre-training: Primarily on H&E and IHC cancer datasets, with varying scales (up to 3.1M slides) and magnifications.

Evaluation Framework

The authors evaluated these models across 11 kidney-specific downstream tasks at two scales: Tile-level and Slide-level.

A. Tile-Level Tasks (Local Morphology)

Approach: Used Linear Probing (Logistic Regression) and k-Nearest Neighbor (kNN) classifiers on frozen embeddings. Regression tasks used Ridge Regression.
Tasks:
1. Globally vs. Non-globally Sclerotic Glomeruli (PAS stain).
2. GBM Spike Detection (PASM stain).
3. Reference vs. Abnormal Tubule (H&E stain, labeled via spatial transcriptomics).
4. Inflammation vs. Non-inflammation (H&E stain, aligned with CD45-IHC).
5. Arterial Stenosis Classification (Mild/Moderate/Severe, PAS stain).
6. Cell Type Proportion Estimation (Regression of 16 cell types from H&E).
7. Copy Detection (Robustness test against geometric, color, noise, and deformation augmentations).

B. Slide-Level Tasks (Global Context)

Approach: Used Attention-Based Multiple Instance Learning (ABMIL) to aggregate tile embeddings into slide-level representations.
Validation: Repeated Nested Stratified K-Fold Cross-Validation (5 outer folds, 3 seeds) with hyperparameter tuning (learning rate, attention dim, FC dim) to ensure unbiased performance estimation.
Tasks:
1. Diabetic Nephropathy (DN) vs. Reference (PAS).
2. Membranous Nephropathy (MN) Treatment Response (Responder vs. Non-responder, PAS).
3. eGFR Decline Prediction (Post-transplant, PAS).
4. eGFR Prediction (1-year post-transplant, PAS).

Statistical Analysis

Metrics: Matthews Correlation Coefficient (MCC) for classification, Pearson Correlation Coefficient (PCC) for regression, and $R^2$ for slide-level regression.
Significance Testing: Friedman test followed by pairwise Wilcoxon signed-rank tests with Holm-Bonferroni correction. Results visualized using Compact Letter Display (CLD).
Reproducibility: An open-source Python package, kidney-hfm-eval, was released to standardize these pipelines.

3. Key Results

Tile-Level Performance

High Performance (Coarse Morphology): HFMs excelled at tasks involving clear, coarse structural changes.
- Globally Sclerotic Glomeruli: Top models (Prov-Gigapath, UNI) achieved median MCC > 0.96.
- Inflammation: Moderate to strong performance (Median MCC ~0.47), suggesting models capture immune cell clustering.
Moderate/Low Performance (Fine/Micro-Structural):
- GBM Spikes (PASM): Performance dropped significantly (Median MCC ~0.38), indicating difficulty in detecting subtle basement membrane irregularities.
- Arterial Stenosis: Moderate performance (Median MCC ~0.39), struggling with continuous gradations of vascular injury.
- Tubular Abnormalities: Near-random performance (MCC ~0), failing to distinguish molecularly defined tubular injury from reference tissue.
- Cell Type Estimation: Weak correlation (PCC ~0.36), indicating embeddings cannot fully capture transcriptomic-level cellular heterogeneity.
Cross-Stain Generalization: Models trained on H&E/IHC successfully transferred to PAS and PASM tasks, confirming stain invariance for meso-scale features. However, performance degraded on fine-grained tasks regardless of stain type.
Robustness: Models showed near-perfect robustness to geometric and color augmentations but varied significantly in handling noise perturbations.

Slide-Level Performance

Diagnostic Classification (DN vs. Reference): HFMs achieved near-perfect separation (Median MCC > 0.92), demonstrating strong ability to recognize disease-defining global morphology.
Prognostic/Functional Prediction:
- MN Treatment Response: Near-random performance (MCC ~0), failing to capture immunologic drivers of treatment response.
- eGFR Decline/Prediction: Poor performance (MCC < 0.1, $R^2$ < -0.08), indicating that static histological morphology alone is insufficient to predict functional decline or transplant outcomes.

4. Key Contributions

First Comprehensive Kidney Benchmark: Systematically evaluates 11 HFMs across 11 diverse kidney tasks, covering multiple stains (PAS, PASM, H&E, IHC), scales (tile/slide), and outcome types (detection, diagnosis, prognosis).
Rigorous Statistical Framework: Introduces a robust evaluation pipeline using repeated nested stratified cross-validation and rigorous post-hoc statistical testing (Friedman/Wilcoxon/CLD) to ensure fair model comparison.
Open-Source Tooling: Releases kidney-hfm-eval, a standardized Python package to facilitate reproducibility and future benchmarking in nephrology.
Empirical Insights: Provides evidence that current HFMs are stain-agnostic for coarse features but limited in capturing fine-grained microstructures, molecular phenotypes, and prognostic signals.

5. Significance and Conclusion

Clinical Implications: Current HFMs are suitable for coarse diagnostic screening (e.g., identifying diabetic nephropathy or global sclerosis) but are not yet ready for complex clinical decision-making involving treatment response prediction, prognosis, or fine-grained molecular phenotyping in nephrology.
Scientific Insight: The study confirms that while self-supervised learning (DINO) induces robustness to stain variations, it does not inherently solve the "signal availability" problem for subtle or molecularly defined kidney pathologies.
Future Directions: The authors advocate for the development of kidney-specific, multi-stain, and multi-modal foundation models that integrate clinical, omics, and longitudinal data to bridge the gap between static morphology and dynamic disease progression.

In summary, the paper establishes that while HFMs have successfully generalized to kidney histology for macroscopic structural tasks, they currently lack the capacity to decode the subtle, molecular, and prognostic signals essential for advanced nephrology care.