How Far Has AI Come in Liver Fibrosis Staging? A Large-Scale Real-World Dataset and Benchmark

Technical Summary: LiFS Benchmark for Liver Fibrosis Staging

Problem Statement
Despite significant methodological progress in deep learning for medical image analysis, the clinical readiness of Artificial Intelligence (AI) for liver fibrosis staging remains unquantified under real-world conditions. Existing benchmarks are typically limited to single-center, controlled environments with curated data, failing to capture the heterogeneity of multi-scanner clinical practice. Furthermore, there is a lack of large-scale datasets featuring complete gadoxetic acid-enhanced (GED) sequences (including the hepatobiliary phase, HBP) paired with histopathology-confirmed staging labels. Consequently, it is unclear how far current AI has progressed toward matching the performance of practicing radiologists or how robust these systems are to cross-center domain shifts, label imbalances, and missing modalities.

Methodology and Dataset Construction
To address these gaps, the authors introduce LiFS (Liver Fibrosis Staging), a large-scale, multi-center dataset and benchmark derived from the CARE-Liver challenge held with MICCAI 2025.

• Dataset Composition: LiFS comprises 610 patients from three clinical centers and four different MRI scanners (varying vendors and field strengths: 1.5T and 3.0T). The dataset includes 4,174 3D MRI volumes and over 218,000 slices.
• Imaging Protocols: The data covers multi-sequence MRI: T1-weighted (T1W), T2-weighted (T2W), Diffusion-Weighted Imaging (DWI), and four-phase GED imaging (arterial, portal venous, delayed, and hepatobiliary phases).
• Ground Truth: Staging labels are based on the Scheuer grading system (S1–S4), confirmed by histopathology from biopsies or surgical resections performed within three months of the MRI scan.
• Real-World Imperfections: The dataset intentionally includes spatial misalignment between sequences, missing modalities due to acquisition failures, and significant cross-center heterogeneity.
• Experimental Design: The benchmark evaluates 9 independently developed methods selected from 96 registered teams. Evaluation is conducted on:
  • In-Distribution (ID) Test Set: Data from the same distribution as training.
  • Out-of-Distribution (OOD) Test Set: Data from a completely unseen scanner and center (Vendor C) to assess generalization.
  • Subtasks: Cirrhosis Detection (S4 vs. S1–3) and Significant Fibrosis Detection (S1 vs. S2–4).
  • Settings: Contrast-Enhanced (all sequences) and Non-Contrast (T1W, T2W, DWI only).
• Clinical Reference: Two board-certified radiologists (3 and 8 years of experience) independently scored the same cases using a 7-point qualitative MRI scoring scheme, providing an in-cohort reference for AI-versus-human comparison.

Key Contributions

1. LiFS Dataset: The first large-scale, multi-center dataset featuring complete GED sequences (including HBP) and histopathology-confirmed fibrosis staging from diverse real-world scanners.
2. Comprehensive Benchmark: A unified evaluation framework comparing AI methods against radiologist performance under both matched (ID) and shifted (OOD) conditions.
3. Empirical Analysis: A systematic analysis of nine top-performing methods to identify factors influencing clinical readiness, including input dimensionality, fusion strategies, and backbone architectures.

Results

• AI vs. Radiologists:
  • Under ID conditions, the best AI methods achieved AUCs comparable to the senior radiologist (Expert-8Y) and significantly outperformed the junior radiologist (Expert-3Y). For example, in the contrast-enhanced ID setting, the best AI reached an AUC of 86.16% (Subtask 1), exceeding Expert-8Y's 81.83%.
  • Under OOD conditions, performance became heterogeneous. While the best AI methods remained comparable to the senior radiologist in several settings, median AI performance generally dropped to junior-radiologist levels or below.
  • Significance: Top-tier AI can approach senior radiologist performance in matched settings, but clinical-level consistency is not yet achieved across distribution shifts.
• Data-Perspective Challenges:
  • Cross-Center Heterogeneity: Most methods suffered substantial performance degradation when moving from ID to OOD sets.
  • Label Imbalance: In Subtask 2 (S1 vs. S2–4), severe class imbalance (S2–4 constituting ~93% of OOD data) led to "model collapse" where many methods achieved high Accuracy (>85%) by predicting the majority class but failed to achieve high AUC.
  • Contrast-Enhanced Sequences: The inclusion of GED sequences (especially HBP) yielded mixed results. While some methods improved significantly under OOD conditions (suggesting HBP captures functional information invariant to scanner shifts), others degraded, indicating a practical dilemma in utilizing HBP without standardized acquisition.
• Technical-Perspective Challenges:
  • Pre-registration: Methods incorporating rigid spatial registration achieved higher ID performance, suggesting alignment is crucial for multi-modal fusion.
  • Input Dimensionality: 3D volumetric approaches achieved higher ID performance but suffered larger OOD drops compared to 2D patch-based methods, likely due to overfitting to vendor-specific spatial priors.
  • Fusion Strategy: Early fusion (joint representation learning) showed the smallest OOD drop, whereas late fusion (decision-level aggregation) suffered the largest degradation.
  • Backbone Architecture: Transformer-based models demonstrated slightly better OOD robustness compared to CNNs, potentially due to their ability to capture long-range dependencies.

Significance and Claims
The paper claims that LiFS provides the first rigorous, real-world benchmark to measure the current state of AI in liver fibrosis staging. The authors conclude that while AI has reached a clinically meaningful level in selected, matched settings—potentially supporting non-invasive staging and reducing biopsy reliance in specific scenarios—its robustness remains insufficient for reliable deployment across heterogeneous clinical environments.

The benchmark exposes a critical gap between high performance on curated test data and clinically reliable AI. The authors emphasize that future research must prioritize cross-center generalization, handling label imbalance, and developing robust multi-modal modeling strategies that can navigate the variability of contrast-enhanced sequences and scanner differences. LiFS serves as a platform to guide this next stage of research toward clinically deployable solutions.