Weakly Supervised Multicenter Nancy Index Scoring in Ulcerative Colitis Using Foundation Models

1. Problem Statement

The paper addresses the challenge of automating the Nancy Histological Index (NHI) scoring for Ulcerative Colitis (UC) activity.

• Clinical Context: NHI is a standard 5-grade scale (0–4) used to assess histologic activity in UC, crucial for clinical trials and routine care. Manual grading is time-consuming and suffers from significant inter- and intra-observer variability.
• Technical Challenge: Existing computational pathology solutions often rely on dense region-level or cell-level annotations, which are expensive to generate and difficult to standardize across different institutions.
• Specific Goal: Develop a robust, automated scoring system that operates under weak supervision (using only case-level or slide-level labels) across multicenter datasets with heterogeneous staining protocols, scanner types, and patient populations.

2. Methodology

The authors propose a Weakly Supervised Multiple Instance Learning (MIL) pipeline tailored for Whole Slide Images (WSIs).

A. Data Preparation

• Dataset: Retrospective collection of H&E-stained colon biopsies from three Czech institutions (IKEM, TUH, KNL) spanning 2019–2025.
• Labels: Only slide/case-level NHI grades (0–4) were available. No region-level annotations were used.
• Preprocessing:
  • Tissue detection and quality control (removing blur/artifacts).
  • Tiling: Slides are divided into tiles at three resolutions (T0, T1, T2).
  • Data Split: Case-level splitting to ensure no patient data leakage between training, preliminary test, and final test sets.

B. Model Architecture

The pipeline utilizes a Foundation Model approach combined with a specialized MIL strategy:

1. Foundation Model Encoders: Frozen pre-trained visual encoders (UNI2-h, Prov-GigaPath, Virchow2) extract tile embeddings ($h_i$).
2. Specialized Inference Models: Instead of a single monolithic model, three distinct MIL models are trained for specific diagnostic sub-tasks:
   • Neutrophil Activity Model: Binary classification (Low vs. High activity).
   • Nancy-Low Model: Multi-class classification for grades {0, 1, High}.
   • Nancy-High Model: Multi-class classification for grades {Low, 2, 3, 4}.
3. Attention-Based Aggregation (Instance-First Variant):
   • Standard MIL aggregates embeddings first, then classifies.
   • This paper's variant: Applies a task-specific classifier ($c_\psi$) to each tile embedding to generate tile-level logits ($s_i$).
   • These tile logits are then pooled using attention weights ($a_i$) to generate slide-level logits: $s = \sum a_i s_i$.
   • This "instance-first" approach allows for direct tile-level scoring and better interpretability.

C. Post-Processing & Ensembling

• Ensemble Strategy: At inference, the three models vote to determine the activity group (Low vs. High).
• Final Prediction: The final 5-grade NHI is derived by selecting the appropriate specialist model (Low or High) and taking the argmax over its specific grade classes, ignoring the "other" group class.

3. Key Contributions

1. Weak Supervision Framework: Successfully tackles NHI prediction using only coarse slide-level labels, eliminating the need for costly dense annotations.
2. Specialized MIL Architecture: Introduces a "hierarchical gating" via an ensemble of task-specific models (Neutrophil, Low, High) rather than a single flat classifier, improving performance on specific clinical endpoints.
3. Foundation Model Integration: Demonstrates the efficacy of modern pathology foundation models (specifically Virchow2) in handling multicenter heterogeneity.
4. Multicenter Validation: Rigorous evaluation across three distinct institutions with varying protocols, proving robustness to domain shift.
5. Interpretability: Utilizes attention weights and instance-first classification to visualize specific tissue regions (e.g., cryptitis, abscesses) driving the decision.

4. Results

The model was evaluated on a final test set comprising data from all three centers.

• Performance Metrics:
  • Weighted Cohen's $\kappa$: 0.85 (compared to PathAI's reference $\kappa$ of 0.91).
  • Spearman Correlation ($\rho$): 0.85 (compared to PathAI's 0.89).
  • The performance is comparable to strong reference systems despite using only weak supervision.
• Ablation Studies:
  • Encoder Choice: Virchow2 provided the most consistent gains. Using a lower resolution (T2) with Virchow2 improved accuracy for Neutrophil and Nancy-Low tasks while reducing computation time by ~10x.
  • Ensemble vs. Hierarchical Gating: The proposed ensemble approach (weighted $\kappa$ = 0.89) outperformed a strict hierarchical gating baseline (weighted $\kappa$ = 0.87), suggesting that independent task-specific views yield more robust decisions.
  • Neutrophil Detection: The learned neutrophil model significantly outperformed an external pretrained neutrophil detector (Precision: 0.91 vs. 0.74).
• Interpretability: Attention heatmaps successfully highlighted biologically relevant features such as neutrophilic cryptitis, crypt abscesses, and surface epithelial injury.

5. Significance and Conclusion

• Clinical Impact: This work demonstrates that weakly supervised MIL with foundation models can provide robust, interpretable histology scoring in realistic, heterogeneous multicenter settings.
• Efficiency: By relying on slide-level labels, the approach drastically reduces the annotation burden on pathologists, making large-scale deployment feasible.
• Decision Support: The system acts as a viable decision-support tool that can standardize grading across centers and reduce inter-observer variability, particularly for distinguishing between inactive (NHI 0) and highly active (NHI 4) cases.
• Future Direction: The study validates the potential of "foundation models + weak supervision" as a standard paradigm for computational pathology in complex, real-world clinical environments.