💻 computer science

Polygon-mamba: Retinal vessel segmentation using polygon scanning mamba and space-frequency collaborative attention

The paper proposes Polygon-mamba, a hybrid CNN-Mamba network that integrates a polygon scanning visual state space model and a space-frequency collaborative attention mechanism to effectively preserve topological integrity and enhance feature extraction for accurate small retinal vessel segmentation across multiple datasets.

Original authors: Yuanyuan Peng, Wen Li

Published 2026-05-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuanyuan Peng, Wen Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to trace a delicate, winding river system on a map, but the ink is faint, the paper is crumpled, and the river splits into tiny, almost invisible streams. This is essentially what doctors face when trying to map the blood vessels in the back of the eye (the retina). These vessels are crucial for diagnosing eye diseases, but they are incredibly hard to trace because they are thin, twisty, and often blend into the background.

This paper introduces a new computer program called Polygon-mamba designed to solve this tracing problem automatically. Here is how it works, broken down into simple concepts:

1. The Problem with Old Maps (The "Grid" Issue)

Previous computer programs tried to trace these vessels by scanning the image like a lawnmower: moving strictly left-to-right, then right-to-left, and up-and-down.

The Analogy: Imagine trying to trace a curved river using only a ruler that can only move in straight horizontal or vertical lines. You would miss the curves, break the river into disconnected pieces, and fail to see the tiny tributaries.
The Result: The old methods often broke the continuity of the vessels or missed the smallest ones entirely.

2. The New Strategy: "Polygon Scanning" (The "Spiderweb" Approach)

The authors created a new way to scan the image called Polygon Scanning. Instead of a rigid grid, the program scans the image in expanding shapes, like a spider spinning a web or ripples spreading out from a stone dropped in water.

The Analogy: Instead of walking in a grid, imagine standing in the center of a room and walking outward in a circle, then a hexagon, then an octagon. You cover every angle and curve naturally.
The Benefit: This "Polygon Mamba" method follows the natural, winding shape of the blood vessels. It keeps the "river" connected and ensures that even the tiniest, most fragile branches are detected without breaking the line.

3. The "Two-Eyed" View (Space and Frequency)

The paper argues that looking at an image only one way isn't enough. The new model uses a Space-Frequency Collaborative Attention mechanism, which is like having two different pairs of glasses to look at the same picture.

The Spatial Glasses: These focus on where things are and their shape (the structure of the vessel).
The Frequency Glasses: These focus on the details and the "texture" (the sharp edges and fine lines).
The Collaboration: The model combines these two views. It uses the "Spatial" view to understand the big picture and the "Frequency" view to sharpen the blurry edges and catch the faintest details. This prevents the computer from getting confused by background noise or clutter.

4. The Results: A Better Map

The researchers tested this new "Polygon-mamba" on three standard datasets (collections of retinal images) that are like the "standard tests" for this field.

The Outcome: Compared to other top-tier computer models, their new method was better at finding the small, tricky vessels. It produced maps that looked much closer to what a human expert would draw by hand.
The Numbers: It achieved higher scores in accuracy and sensitivity (finding the vessels that are actually there) across all three tests.

Summary

In short, the authors built a smarter "digital tracer" for eye blood vessels. Instead of using a rigid, grid-like scanner that breaks delicate lines, they used a flexible, shape-shifting scanner (Polygon Scanning) that follows the curves of the vessels. They also gave the computer a "dual-vision" system to see both the big picture and the tiny details at the same time. The result is a more accurate, complete map of the eye's blood vessels, which is a significant step forward in helping computers assist doctors in diagnosing eye diseases.

Technical Summary: Polygon-mamba for Retinal Vessel Segmentation

Problem Statement

Retinal vessel segmentation is a critical task for diagnosing and assessing ocular diseases. However, accurately segmenting small retinal vessels remains a significant challenge due to several factors:

Low Contrast: Weak contrast between vessels and their surroundings makes differentiation difficult.
Complex Topology: Vessels exhibit intricate, interwoven patterns with significant scale variations.
Limitations of Existing Models:
- CNNs: While proficient at capturing local details, they often struggle with long-range dependencies.
- Transformers: Effective at global semantics but computationally expensive.
- Mamba-based Models: While offering linear time complexity for long-range dependencies, traditional Mamba architectures relying on horizontal and vertical scanning can compromise the topological integrity of anatomical structures. This often leads to local discontinuities and missed detections in slender, small retinal capillaries.
- Attention Mechanisms: Many existing methods rely solely on spatial attention, potentially overlooking critical frequency-domain information necessary for capturing fine details and global correlations.

Methodology

The authors propose Polygon-mamba, a hybrid CNN-Mamba fusion network designed to address the specific challenges of small vessel detection. The architecture integrates two novel modules into a UNet framework:

1. Polygon Scanning Visual State Space Model (PS-VSS)

To overcome the topological fragmentation caused by standard horizontal/vertical scanning in Mamba models, the authors introduce a Polygon Scanning strategy within the Visual State Space Model (SS2D).

Mechanism: Instead of linear scans, the model initiates from a central point and expands outward along polygonal paths (e.g., pentagonal, hexagonal).
Multi-layer Reverse Scanning: The scanning direction alternates between layers (e.g., Layer $M$ vs. Layer $M+1$ ), creating a reverse scanning effect that systematically covers the feature map.
Goal: This approach preserves pixel-level connectivity and better models the curved, branching, and slender topological features of retinal vessels, ensuring continuous structural representation.

2. Space-Frequency Collaborative Attention Mechanism (SFCAM)

Integrated into the skip connections, this module leverages the complementary nature of spatial and frequency domains.

Three-Branch Architecture:
- Local Branch: Extracts multi-scale features (using $1\times1$ , $3\times3$ , $5\times5$ convolutions) to capture coarse and fine vessel details.
- Global Branch: Utilizes the PS-VSS module to capture long-range dependencies and global context.
- Wavelet Transformer (WT) Branch: Decomposes features into low-frequency (morphology, topology) and high-frequency (boundaries, small branches, texture) components using Discrete Wavelet Transform.
Bidirectional Cross-Attention Fusion (BCFM): This sub-module aligns semantic discrepancies between spatial and frequency features. It employs a query-exchange strategy to construct bidirectional cross-attention, allowing the model to dynamically select and fuse features from local/high-frequency and global/low-frequency branches.
Goal: To adaptively enhance prominent vessel features while suppressing clutter, effectively recovering small vessels and edge contours.

Key Contributions

The paper outlines four primary contributions:

Polygon-mamba Model: A novel segmentation framework specifically designed for the accurate detection of small retinal vessels, substantially boosting segmentation accuracy.
PS-VSS Module: An innovative scanning technology that uses polygonal paths to efficiently extract semantic information and capture global features, addressing the continuity issues of traditional Mamba scans.
SFCAM Module: A mechanism that extracts efficient features from both spatial and frequency domains, enabling the model to highlight target objects and suppress noise.
Integrated UNet Scheme: The successful integration of PS-VSS and SFCAM into a UNet architecture to generate an efficient segmentation scheme that balances global modeling with local detail preservation.

Experimental Results

The model was evaluated on three public datasets: DRIVE, STARE, and CHASE DB1.

Quantitative Performance

The proposed method outperformed state-of-the-art models (including Mamba-UNet, FSE-Mamba, Rolling-UNet, and KM-UNet) in key metrics:

DRIVE: F1 Score: 0.8288, AUC: 0.9808, Sensitivity (SE): 0.8356.
STARE: F1 Score: 0.8282, AUC: 0.9840, SE: 0.8314.
CHASE DB1: F1 Score: 0.8251, AUC: 0.9866, SE: 0.8484.

Notably, the model achieved the highest SE values across all datasets, indicating superior capability in detecting positive samples (vessels), particularly small and faint ones. While the Specificity (SP) was slightly lower than some competitors in certain cases, the authors attribute this to the model's heightened sensitivity to fine vessels, which may increase background misclassifications but significantly improves recall.

Ablation and Comparative Studies

Module Efficacy: Ablation studies confirmed that adding PS-VSS and SFCAM sequentially improved IOU, F1, and SE across all datasets. For instance, on CHASE DB1, SFCAM increased SE by 9.45% compared to the baseline plus PS-VSS.
Scanning Strategy: Experiments comparing different polygon shapes (Triangle, Quadrilateral, Pentagon, Hexagon, Octagon) revealed that the Pentagon scanning method yielded the best overall performance on DRIVE and CHASE DB1, while Hexagon performed well on STARE. The Pentagon strategy was selected as the optimal choice for the final model.
Complexity: The model has 68.32M parameters and 2.53G FLOPs. While higher than some lightweight models, it is significantly more efficient than high-complexity methods like MCDAU-Net and MDF-Net, achieving a favorable trade-off between performance and computational cost.

Significance and Claims

The authors claim that Polygon-mamba represents a significant advancement in retinal vessel segmentation by directly addressing the topological discontinuities inherent in standard Mamba architectures.

Topological Integrity: By utilizing polygon scanning, the model successfully preserves the continuity of complex vascular structures, a common failure point in previous Mamba-based approaches.
Dual-Domain Feature Extraction: The integration of space-frequency attention allows the model to capture both the global structural context and the fine-grained local details (edges, small branches) that are often lost in spatial-only models.
Small Vessel Detection: The primary significance lies in the model's ability to detect small, low-contrast vessels with high sensitivity, outperforming existing methods in visual inspection and quantitative metrics.

The paper concludes that while the model achieves superior performance, future work will focus on improving Specificity (noise suppression) and exploring traditional pattern recognition methods to further refine noise handling without compromising the detection of small vessels.