🤖 machine learning

Evaluation of Blood Vessel Segmentation Methods on Hard-to-Detect Vascular Structures

This paper introduces a Local Vessel Salience (LVS) index and a corresponding mean Low-Salience Recall (mLSR) metric to reveal that standard segmentation methods systematically fail to detect hard-to-detect vascular structures, while also proposing a data augmentation strategy to improve algorithmic sensitivity and preserve vascular connectivity.

Original authors: João Pedro Parella, Matheus Viana da Silva, Cesar Henrique Comin

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: João Pedro Parella, Matheus Viana da Silva, Cesar Henrique Comin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a vast, tangled forest. In this forest, the trees are actually blood vessels, and your job is to map out every single branch, from the thick, sturdy trunks to the tiniest, almost invisible twigs. This is the world of blood vessel segmentation, a crucial task in medical imaging. Doctors and researchers use computers to draw these maps so they can spot diseases, measure how twisted the vessels are, or even simulate how blood flows through the body.

But here's the catch: not all trees in this forest are easy to see. Some are bright and bold against the dark background, like neon signs. Others are faint, ghostly, and blend right into the shadows. For a long time, scientists have used a "scorecard" to grade how well their computer programs did at mapping the forest. These scorecards, however, are a bit like grading a student based only on how well they answered the easy questions. If a program gets 90% of the bright, easy-to-see vessels right, it gets a high score, even if it completely misses the faint, tricky ones. This is a problem because missing a tiny, faint vessel can break the whole map, making the forest look disconnected when it's actually a continuous network. The big question is: How do we build a better test that forces our computer detectives to pay attention to the hardest-to-find clues?

This paper introduces a new way to look at the problem, acting like a magnifying glass for the "hard-to-see" parts of the map. The authors, a team from the Federal University of São Carlos, realized that standard tests were hiding a major flaw: computers were great at finding the obvious vessels but terrible at finding the faint ones. To fix this, they invented a new tool called the Local Vessel Salience (LVS) index. Think of "salience" as how much a vessel "pops out" from its background. The LVS index measures this for every single tiny piece of a vessel. If a piece of a vessel looks very different from the tissue around it, it has high salience (easy to find). If it looks almost the same as the background, it has low salience (hard to find).

Using this new index, the researchers created a special test called mean Low-Salience Recall (mLSR). Instead of just asking, "How many vessels did you find?", this test asks, "How many of the really hard-to-see vessels did you find?" They put 16 different computer programs (mostly variations of a popular AI design called U-Net) through the wringer on six different types of medical images, ranging from eye scans to heart angiograms.

The results were a wake-up call. While the programs looked impressive on the standard tests, scoring high marks for overall accuracy, they fell apart when the researchers looked at the faint vessels. The authors found that for the best-performing methods, the standard "recall" (how much of the vessel was found) was around 85% to 86%. However, when they switched to the new mLSR test to focus only on the difficult, low-salience parts, the scores dropped dramatically—by about 20 percentage points. In the toughest datasets, like OCTA-500, the top methods only managed to find about 58% of the hard-to-detect vessels. This suggests that current AI models are essentially "cheating" by relying on the easy, high-contrast parts of the image and ignoring the tricky bits.

The paper also tried to teach the AI a better lesson. The researchers developed a training trick where they artificially made the bright vessels look fainter and even added fake "gaps" in the lines during training. The idea was to force the AI to learn how to connect the dots even when the lines were barely visible. The results of this experiment were mixed but promising. When the AI had a huge amount of data to learn from, this trick didn't help much. However, when the researchers gave the AI very little data to work with (as few as 4 samples), the trick worked wonders. In these "data-starved" scenarios, the augmented training helped the model improve its performance significantly, suggesting that this method could be a lifesaver when doctors don't have many labeled images to teach the computer.

In short, the paper doesn't claim to have solved the problem of finding every single blood vessel. Instead, it provides a new, more honest ruler to measure how good our current tools really are. It reveals that while our AI is getting better at seeing the obvious, it is still struggling with the subtle, faint details that are critical for a complete picture. The authors suggest that future designs need to focus less on just getting the easy parts right and more on learning to see the invisible, perhaps by using these new "salience" tools to guide their training.

Technical Summary: Evaluation of Blood Vessel Segmentation Methods on Hard-to-Detect Vascular Structures

Problem Statement
Blood vessel segmentation is critical for clinical diagnosis and vascular research, yet minor errors in segmentation can significantly alter connectivity patterns and morphological measurements. Standard evaluation metrics, such as the Dice coefficient, precision, and recall, aggregate performance across an entire image. Consequently, these global metrics often overlook inaccuracies in specific regions, particularly where vessels exhibit low contrast against the background. In such "hard-to-detect" areas, small discontinuities can fundamentally alter the vascular network's topology, yet their impact is diluted in standard scores because high-contrast vessels dominate the calculation. Existing metrics fail to isolate algorithmic performance on these low-salience structures, masking systematic failures in preserving vascular connectivity.

Methodology
To address these limitations, the authors propose a framework centered on the Local Vessel Salience (LVS) index and a derived performance metric, mean Low-Salience Recall (mLSR).

Local Vessel Salience (LVS) Index:
- The LVS index quantifies the difficulty of identifying a specific vessel segment by comparing local vessel intensity against the immediate background.
- The calculation utilizes a ground-truth manual annotation mask, which is first converted into a graph representation (nodes at bifurcations/terminations, edges as vessel segments) using the Pyvane framework.
- For each pixel on the Medial Axis Segment (MAS), the method identifies the two closest contour points on opposite sides of the vessel to define a cross-section.
- It samples vessel pixels ( $S_v$ ) along the cross-section and background pixels ( $S_b$ ) within a defined radius ( $r_b$ ) from the contour points.
- The salience is calculated as the relative intensity difference ( $\Delta I$ ) between the average vessel intensity vector ( $I_v$ ) and the background intensity vector ( $I_b$ ).
- These values are smoothed along the MAS and propagated to all adjacent vessel pixels to create a continuous LVS map.
Mean Low-Salience Recall (mLSR):
- Using the LVS map, the authors define a threshold $t$ to isolate "challenging" vessel pixels (those with LVS $\le t$ ).
- A specific recall metric, LSRecall, is calculated for these low-salience pixels.
- To generate a single, threshold-independent score, the mLSR is computed as the mean of LSRecall values across a range of thresholds ( $T$ ), weighted equally to penalize errors in the most difficult regions.
Salience-Based Data Augmentation:
- As a proof-of-concept, the authors propose a data augmentation technique that systematically reduces the salience of specific vessel segments during training.
- This involves attenuating vessel intensity to match the local background and introducing synthetic discontinuities. The goal is to force neural networks to rely on global continuity cues rather than local texture contrast.

Experimental Setup
The study evaluated 16 segmentation methods (including UNet, UNet++, ResUNet, HRNet, and FSGNet variants) across six widely used 2D datasets: DRIVE, STARE, CHASEDB1, VessMAP, DCA1, and OCTA-500. These datasets encompass fundus photography, fluorescence microscopy, coronary angiography, and OCTA. The experiments utilized standard training protocols with deep supervision and extensive data augmentation (CutMix, flipping, blurring) to ensure model convergence.

Key Results

Correlation with Difficulty: The study demonstrates a strong correlation between segmentation performance and the LVS index. Algorithms consistently fail to recover vessels with low salience.
Performance Gap: The mLSR values for all evaluated methods were significantly lower than their standard recall values. A typical performance decrease of 20 percentage points was observed when focusing specifically on low-salience vessels.
- For top-performing methods on benchmark datasets like DRIVE and OCTA-500, mLSR values were approximately 58%, whereas standard recall was significantly higher (e.g., ~86% for DRIVE).
- The OCTA-500 dataset proved the most challenging, followed closely by DRIVE.
Metric Divergence: While global recall and mLSR are correlated, they diverge significantly at the image level (Pearson correlation of 0.46). Images with high global recall (~90%) can exhibit mLSR values as low as 30%, confirming that standard metrics mask failures in low-contrast regions.
Augmentation Efficacy: The proposed salience augmentation did not yield significant improvements when training on full datasets (DRIVE and VessMAP), likely because the original data already contained sufficient low-salience examples or the datasets had uniform salience distributions. However, in scenarios with extremely scarce training data (e.g., 2–4 samples), the augmentation significantly improved performance (e.g., Dice score increased from 75.3% to 82.5% with 4 samples), suggesting it helps models generalize from limited examples.

Significance and Claims
The paper claims that the developed methodology provides a quantitative basis for designing segmentation algorithms with improved sensitivity to hard-to-detect vessels. By introducing the LVS index and mLSR, the authors offer a tool to:

Reveal systematic errors in vascular connectivity preservation that global metrics miss.
Guide future algorithmic development toward preserving topological integrity in low-contrast regions.
Identify whether performance gaps stem from architectural limitations or inconsistencies in manual annotations (as low-salience vessels are also difficult for humans to annotate).

The authors conclude that while current state-of-the-art methods perform well on high-contrast structures, they exhibit consistent deficiencies in recovering low-salience vessels, a limitation that must be addressed to ensure reliable vascular network analysis. The salience augmentation technique is presented as a promising direction specifically for data-scarce scenarios.

More like this