Joint Segmentation and Grading with Iterative Optimization for Multimodal Glaucoma Diagnosis

1. Problem Statement

Accurate glaucoma diagnosis is hindered by the subtlety of early-stage structural changes, which often lack clear visual cues. Existing approaches suffer from two main limitations:

• Single-Modality Reliance: Most methods rely solely on Fundus images or Optical Coherence Tomography (OCT), capturing only partial pathological information and missing early disease progression.
• Task Decoupling: Current multimodal methods typically treat segmentation (identifying the optic disc and cup) and grading (classifying disease severity) as independent tasks. This fails to exploit the inherent correlation between these tasks and misses the potential for joint optimization to improve overall performance.

2. Methodology: The IMO Framework

The authors propose the Iterative Multimodal Optimization (IMO) model, a joint framework designed to process Fundus and OCT data simultaneously. The architecture consists of three core components:

A. Mid-Level Fusion with Dual Encoders

• Input Processing: The model accepts paired Fundus images and OCT volumes.
• Feature Extraction:
  • A Feature Pyramid Encoder processes Fundus images using progressive downsampling and pointwise convolution to extract multi-scale features.
  • A Volume Encoder processes OCT data to capture complementary 3D structural cues.
• Fusion Strategy: Instead of simple concatenation, the model employs a mid-level fusion strategy where features from both modalities are integrated before task-specific heads.

B. Cross-Modal Feature Alignment (CMFA) Module

To address the discrepancies between Fundus (2D appearance) and OCT (3D structure) data, the CMFA module aligns representations in a shared feature space:

1. Channel Attention: Applies Global Average Pooling (GAP) and a two-layer fully connected network to generate channel-wise attention weights ($W_1, W_2$). This emphasizes informative channels and suppresses redundancy for each modality.
2. Spatial Attention: Computes average and max pooling on the channel-refined features to generate a spatial attention map ($S$), highlighting salient regions across modalities.
3. Integration: The final fused feature is obtained by applying a pointwise convolution and ReLU activation, ensuring the output contains refined, complementary information.

C. Iterative Refinement Decoder (IRD)

Inspired by Denoising Diffusion Probabilistic Models (DDPM), the segmentation branch moves away from single-step prediction:

• Forward Process: Standard noise injection is applied to the ground truth segmentation mask ($X_0$) to create noisy states ($X_t$).
• Reverse Process: The model uses a DDIM (Denoising Diffusion Implicit Models) sampling strategy to progressively denoise and refine the segmentation mask from a noisy state back to a clean prediction.
• Benefit: This iterative process allows the model to progressively refine boundaries, improving robustness against uncertainty and noise compared to static, single-pass segmentation.

D. Classification Head

An efficient grading head processes the fused features using Depthwise Separable Convolutions (DSC), Batch Normalization, ReLU, and an SE (Squeeze-and-Excitation) block to model channel dependencies, followed by a fully connected layer for stage probability output.

3. Key Contributions

1. Joint Optimization Framework: Introduction of IMO, a novel model that concurrently performs glaucoma grading and optic disc/cup segmentation, leveraging mutual supervision between tasks.
2. Cross-Modal Alignment: Development of the CMFA module, which effectively aligns heterogeneous Fundus and OCT features through channel and spatial attention mechanisms.
3. Iterative Segmentation: Implementation of a diffusion-based iterative decoder that progressively refines segmentation masks, achieving finer-grained boundary delineation than traditional methods.

4. Experimental Results

Experiments were conducted on the GAMMA dataset (100 Fundus/OCT pairs) with an 80/20 split.

• Segmentation Performance:

  • IMO achieved a mean Dice coefficient (mDice) of 93.33%, outperforming state-of-the-art baselines like SegFormer (90.81%) and SegNext (90.57%).
  • Specific Dice scores were 89.93% for the Optic Disc and 90.09% for the Optic Cup.
  • Visual results showed smoother, more precise boundaries and better capture of subtle structural details compared to other methods.

• Grading Performance:

  • The model achieved 81.48% Precision, 75.00% Accuracy, 79.17% Recall, and 72.03% F1-score.
  • It tied for the highest accuracy with ResNet-50 (75.00%) while significantly outperforming other architectures like ViT and ConvNeXt in recall and F1-score.

• Ablation Studies:

  • Multimodal Necessity: Removing OCT data ("w/o OCT") caused a significant drop in grading precision (from 81.48% to 70.12%), proving the value of multimodal fusion.
  • CMFA Importance: Removing the alignment module ("w/o CMFA") reduced mDice to 92.72% and precision by 7.24%, confirming that feature alignment is critical.
  • Iterative Refinement: Replacing the IRD with a single-pass decoder ("w/o IRD") resulted in a slight but consistent drop in performance, validating the benefit of progressive refinement.

5. Significance

This paper presents a clinically significant advancement in glaucoma diagnosis by:

• Bridging the Gap: Successfully integrating complementary 2D and 3D imaging modalities to capture a more comprehensive view of glaucomatous changes.
• Task Synergy: Demonstrating that joint optimization of segmentation and grading yields superior results compared to treating them as isolated problems.
• Robustness: Introducing a diffusion-based iterative approach that produces more reliable and precise segmentation boundaries, which is crucial for early detection where structural changes are subtle.

The source code is publicly available, facilitating further research and clinical application of multimodal glaucoma assessment.