Deep Learning Based Automated Assessment of Ulcerative Colitis Endoscopic Severity Using Local Global Transformer and Graph Convolutional Networks

Technical Summary: L-GTACNet for Automated Ulcerative Colitis Severity Assessment

Problem Statement
Accurate assessment of Ulcerative Colitis (UC) severity is critical for treatment selection and prognosis, typically relying on the Mayo Endoscopic Score (MES), which ranges from 0 (normal) to 3 (severe). However, manual assessment via colonoscopy is subjective, suffers from inter-observer variability, and is complicated by subtle morphological differences between adjacent severity grades. Existing deep learning approaches face specific limitations: Convolutional Neural Networks (CNNs) often fail to capture long-range spatial dependencies and global contextual patterns, while standalone Transformer models may miss fine-grained local details such as early erythema or superficial erosions. Furthermore, current methods often treat image enhancement, feature extraction, and relational reasoning as disjointed components, lacking a unified framework that models the spatial relationships between discriminative mucosal regions while ensuring clinical interpretability.

Methodology
The authors propose L-GTACNet (Local–Global Transformer Attention-based Graph Convolutional Network), a hybrid deep learning framework designed to address these gaps through an integrated pipeline:

1. Data Preprocessing and Enhancement:

   • Illumination Correction: A Retinex-based algorithm is applied to separate tissue reflectance from non-uniform illumination, correcting brightness gradients without altering pathological color patterns.
   • Contrast Enhancement: Contrast-Limited Adaptive Histogram Equalization (CLAHE) is used to improve local contrast in regions of interest without amplifying noise.
   • Standardization: Images undergo normalization and adaptive data augmentation (geometric and mild photometric transformations) to ensure robustness.
   • Dataset: The study utilizes a publicly available dataset of 564 patient records and 11,276 colonoscopy images, annotated by expert gastroenterologists with MES labels (0–3). Crucially, data partitioning is performed at the patient level to prevent data leakage and ensure generalization.

2. Architecture Design:

   • Local–Global Transformer: The core visual encoder converts the image into a sequence of tokens. It employs a dual-stream approach:
     • Local Branch: Uses attention mechanisms to identify and weight fine-grained discriminative regions (e.g., specific lesions, vascular changes).
     • Global Branch: Utilizes self-attention to model long-range dependencies and capture the overall inflammatory context (e.g., diffuse erythema).
   • Attention-Guided Graph Convolutional Network (GCN): The most discriminative local tokens are transformed into graph nodes. Edges are constructed based on both spatial proximity and semantic similarity. A graph convolution module propagates information across these nodes, modeling the relational dependencies between spatially distributed mucosal regions.
   • Feature Fusion: A learned gating mechanism dynamically fuses the local, global, and graph-based representations, allowing the model to adapt its decision strategy based on image content (e.g., relying more on local features for small ulcers vs. global features for diffuse inflammation).
   • Classification: The fused representation is passed to a classifier head to predict the four MES classes (0–3).

3. Training and Evaluation:

   • The model is trained using categorical cross-entropy loss with class weighting to handle imbalance.
   • Performance is evaluated using accuracy, precision, recall, F1-score, specificity, AUROC, and Cohen's Kappa.
   • Explainability is assessed via attention maps to verify that the model focuses on clinically relevant mucosal abnormalities.

Key Contributions

• Hybrid Architecture: The integration of Local–Global Transformers with Graph Convolutional Networks allows for the simultaneous capture of fine-grained local lesion details and global contextual patterns, while explicitly modeling the spatial relationships between inflammatory regions.
• Unified Pipeline: The framework combines image enhancement (Retinex/CLAHE), hierarchical feature extraction, and relational reasoning into a single end-to-end system, addressing the fragmentation seen in previous works.
• Clinical Interpretability: The model provides disease-relevant explanations by highlighting specific mucosal areas (e.g., ulceration, bleeding) that contribute to the severity prediction, aligning with expert diagnostic criteria.
• Robust Patient-Level Validation: The study employs strict patient-level data partitioning to ensure that performance metrics reflect true generalization capabilities rather than overfitting to specific patient frames.

Results
The proposed L-GTACNet was evaluated on the test set and demonstrated superior performance compared to baseline CNNs, standard Transformers (ViT, Swin), and hybrid architectures:

• Overall Performance: Achieved 94.80% accuracy, 94.67% precision, 94.51% recall, 94.58% F1-score, and an AUROC of 0.986.
• Class-wise Performance: The model maintained high performance across all MES classes, with the highest F1-score of 97.66% for MES-0 (along with 97.31% precision, 98.02% recall, and 98.74% specificity) and consistent results for active disease stages (MES-1 to MES-3).
• Boundary Discrimination: The model effectively distinguished adjacent severity grades, with the most significant errors occurring between MES-1 and MES-2, reflecting the inherent visual ambiguity of intermediate inflammation.
• Ablation Study: Progressive addition of components (Image Enhancement → Local-Global Transformer → Local Attention → Graph Convolution) confirmed that each module contributed incrementally to the final performance.
• Robustness: The model maintained high accuracy (>90%) under simulated degradations including reduced illumination, contrast loss, noise, and motion blur.
• Computational Efficiency: With 18.7 million parameters and an inference time of 21.4 ms/image, the model offers a favorable trade-off between accuracy and computational cost compared to heavier Transformer models.

Significance and Claims
The paper claims that L-GTACNet provides a robust, objective, and reproducible tool for assessing UC severity, potentially reducing inter-observer variability in clinical practice. The authors emphasize that the model's ability to localize specific pathological features (erythema, vascular loss, friability, ulceration) and model their spatial relationships makes it clinically interpretable.

However, the authors maintain a modest stance regarding clinical deployment. They explicitly state that L-GTACNet is a prospective tool designed to assist gastroenterologists, not replace them. The paper concludes that before routine clinical adoption, the framework requires external validation across different institutions, endoscopy devices, and patient populations, as well as prospective clinical studies to confirm its utility in real-world settings.