Dual-Part Multi-Lateral Branched Network for Multi-Class Segmentation in Cardiovascular Catheterization Angiograms

Technical Summary: Dual-Part Multi-Lateral Branched Network (MLBNet) for Multi-Class Segmentation in Cardiovascular Catheterization

Problem Statement
Cardiovascular catheterization relies on X-ray angiography for intraoperative visualization, yet these images are characterized by low contrast, noise, motion artifacts, and severe class imbalance. While existing deep learning approaches have achieved success in binary segmentation (e.g., separating background from a single tool like a guidewire or catheter), there is a critical gap in the simultaneous, multi-class segmentation of multiple structures (background, vessels, and various endovascular tools) within cluttered scenes. Traditional methods often struggle with class overlaps, geometric coupling between tools and vessels, and the dominance of background pixels, which can lead to suboptimal performance when attempting to parse complex, multi-structure scenes.

Methodology
The authors propose a Dual-Part Multi-Lateral Branched Network (MLBNet) designed specifically for class-aware segmentation in catheterization angiograms. The architecture is built upon an Xception backbone and introduces distinct innovations in both the encoder and decoder sub-networks:

• Encoder Sub-Network: The encoder utilizes a multi-flow structure (entry, middle, and exit flows) with lateral branching. Each lateral branch employs three parallel convolutional paths (including depthwise and separable convolutions) to extract diverse, complementary feature maps. These branches are fused via channel concatenation and residual updates to learn shared representations while preserving local edge responses crucial for thin, elongated structures.
• Decoder Sub-Network: Instead of a single shared segmentation head, the decoder employs class-specific lateral branching. Three distinct branches are dedicated to specific classes: one for tools (guidewire/catheter), one for vessels, and one for the background. This design addresses the geometric coupling of tools within vessels, allowing each branch to specialize in the structural properties of its target class without interference from a shared softmax head.
• Class-Aware Scene (CAS) Objective Function: To mitigate background dominance and class imbalance, the authors propose a hybrid loss function. This includes:
  • Structured Focal Cross-Entropy (SFCE): Optimizes each branch as a one-vs-rest classifier, adaptively weighting non-target classes.
  • Branch-Specific Losses: The tool branch incorporates skeleton-consistency and structural containment losses (ensuring tools lie within vessels). The vessel branch uses boundary-aware constraints. The background branch is weighted to prevent overwhelming other classes.
  • Fused Multiclass Supervision: A unified scene head is trained with class-balanced cross-entropy and Dice loss to learn global context and inter-class relationships.
  • Total Loss: A weighted sum of branch-specific and fused losses, utilizing learnable uncertainty-based weights to balance gradients across tasks.

Key Contributions
The paper identifies three primary contributions:

1. Architecture: The proposal of a dual-part MLBNet with encoder-decoder lateral branching, enabling simultaneous background-vessel-tool pixel-level segmentation. This addresses limitations in prior works regarding sequential multi-head branching and shared decoder sub-optimality for geometrically similar structures.
2. Loss Function: The development of a hybrid, class-aware objective function with structure-specific loss terms designed to prevent vulnerability to class dominance and imbalance in angiographic datasets.
3. Comprehensive Evaluation: Extensive validation across multiple datasets, including in-vitro phantom models (silicone-based), a public dataset with 2,000 frames extracted from fluoroscopy video recorded in a silicone-based aortic phantom, and animal models, demonstrating the model's ability to distinguish dominant background classes from foreground structures.

Experimental Results
The model was evaluated on three datasets (Dataset 1a/1b: camera-based phantom; Dataset 2: public dataset with 2,000 frames extracted from fluoroscopy video recorded in a silicone-based aortic phantom; Dataset 3: CathAction phantom and animal).

• Performance: The dual-part MLBNet achieved high overall accuracy (99.20%–99.51%) and strong background segmentation (F1-scores of 99.67%–99.90%).
• Foreground Segmentation: The model demonstrated high macro F1-scores for simpler datasets (96.94% for Dataset 1a, 97.28% for Dataset 2). However, performance decreased on more complex, heterogeneous datasets (66.32% for Dataset 3a phantom, 77.29% for Dataset 3b animal), particularly in distinguishing between catheters and guidewires, which have similar appearances.
• Comparative Analysis: The proposed architecture outperformed existing models (MLBNet, JSR-UNet, FAR-Net, DeepLabV3+) in mean Intersection over Union (mIoU) and tool error rates across all datasets.
• Ablation Studies: Results confirmed that the multi-head architecture combined with branch-specific loss terms yielded more stable foreground segmentation than single-decoder configurations or standard cross-entropy losses.
• Domain Adaptation: The model showed strong transferability between visually similar domains (e.g., between different phantom setups) but struggled with significant domain shifts (e.g., between animal and phantom data), indicating sensitivity to differences in vessel visibility and tool contrast.

Significance and Claims
The authors claim that the dual-part MLBNet effectively resolves the challenge of simultaneous multi-class segmentation in cardiovascular catheterization, a task previously underexplored. The study demonstrates that decoupling decoder heads and employing class-specific loss functions allows the network to handle the geometric coupling of tools and vessels better than unified or sequential approaches. While the model achieves high accuracy in separating background from foreground, the authors modestly note that distinguishing between tools with similar appearances (catheter vs. guidewire) remains a challenge, particularly in animal models. The paper concludes that while the model is fast and accurate, future work must address the "black box" nature of deep learning to ensure explainability for clinical and robot-assisted interventions, and expand the scope to include other tools like stents and balloons.