Performance of Frangi-Hessian Pseudo-Labels for Retinal Vessel Segmentation in AI-Assisted Retinopathy of Prematurity Screening

Technical Summary: Performance of Frangi–Hessian Pseudo-Labels for Retinal Vessel Segmentation in AI-Assisted Retinopathy of Prematurity Screening

Problem Statement
Retinopathy of Prematurity (ROP) is a leading cause of preventable blindness, particularly in low-resource settings where screening resources are scarce. Accurate diagnosis relies heavily on assessing vascular dilation and tortuosity ("Plus disease"), which requires precise retinal vessel segmentation. However, the development of AI-driven screening tools is hindered by the "labeling bottleneck": manual annotation of retinal vessel masks is labor-intensive, prone to inter-observer variability, and difficult to scale for large datasets. While supervised deep learning models perform well with abundant labeled data, the scarcity of expert-labeled neonatal vascular images necessitates alternative, label-efficient strategies.

Methodology
The study proposes a hybrid learning framework that integrates classical image processing with modern deep learning to address the scarcity of ground truth (GT) annotations. The methodology consists of three primary components:

1. Automatic Self-Annotation Pipeline:
   A nine-step pipeline generates vessel pseudo-masks for unlabeled images using a dual Frangi–Hessian approach. The process includes:

   • Preprocessing: Extraction of the green channel (highest vessel contrast), resizing to 512×512, edge-preserving denoising (bilateral filter), and local contrast enhancement (CLAHE).
   • Normalization & Sharpening: Rolling-ball background subtraction to correct illumination and unsharp masking to accentuate vessel ridges.
   • Vesselness Enhancement: Application of Frangi and Hessian filters based on the eigenvalues of the Hessian matrix to quantify the likelihood of tubular structures.
   • Refinement: Adaptive thresholding, morphological operations to remove artifacts, and skeletonization to generate binary masks representing both vessel lumens and centerlines.

2. Model Architectures:
   Five distinct segmentation architectures were evaluated: BioSwinFuseNet (a custom SwinV2-ConvNeXt fusion), UNet, Feature Pyramid Network (FPN), LinkNet, and SegFormer (a transformer-based encoder).

3. Training Regimes:
   Each architecture was trained under three distinct supervision conditions:

   • GT-only: Trained exclusively on expert-labeled masks.
   • Self-only: Trained exclusively on the generated pseudo-masks.
   • Hybrid: Trained on a combination of GT and pseudo-masks using a weighted loss function ($L = 0.7L_{GT} + 0.3L_{Self}$).
     All models utilized a topology-aware loss combining binary cross-entropy, Dice loss, and continuity penalties.

Key Results

• Pseudo-Label Characteristics: The self-generated masks exhibited systematic differences compared to expert GT. They showed lower vessel fraction (0.021 vs. 0.082) and significantly thinner mean vessel diameters (2.01 px vs. 7.47 px), indicating they captured the vessel core rather than full boundaries. However, they preserved global topological features, with branch density and fractal dimension remaining comparable to GT.
• Performance of Training Regimes:
  • Self-only models performed poorly (Dice ≈ 0.005), confirming that pseudo-masks alone lack the precision required for clinical deployment.
  • Hybrid models consistently outperformed both GT-only and Self-only approaches across all architectures.
  • Top Performers: The SegFormer-Hybrid model achieved the highest overall sensitivity (0.70) and AUC (0.94). The BioSwinFuseNet-Hybrid model demonstrated the most significant relative improvement, showing a 122% increase in Dice score compared to its GT-only counterpart.
  • Stability: FPN-Hybrid demonstrated the lowest variance, while UNet-Hybrid maintained strong generalization with high specificity (0.97).

Significance and Claims
The paper claims that the hybrid paradigm offers a scalable solution for developing automated ROP screening tools in environments where expert labeling is limited. By integrating the topological priors derived from classical Frangi–Hessian filters with the representational power of deep neural networks, the study demonstrates that self-annotated masks can meaningfully augment supervised learning.

The authors conclude that while the pseudo-labels are imperfect (under-segmenting small vessels and smoothing curvature), they provide sufficient structural information to bootstrap learning. This approach allows models to learn generalized vascular patterns without relying solely on scarce, expensive expert annotations. The study positions this hybrid framework as a practical pathway for scaling AI-assisted ROP screening, particularly in low-resource clinical settings, without inventing new applications beyond the scope of vessel segmentation and Plus disease detection.