💻 computer science

Rethinking Pulmonary Embolism Segmentation: A Study of Current Approaches and Challenges with an Open Weight Model

This study addresses critical limitations in pulmonary embolism segmentation research by curating a new 490-scan dataset, systematically evaluating nine 2D and 3D architectures to demonstrate that 3D U-Nets with ResNet encoders outperform other models, and releasing an open-weight best-performing model to advance reproducibility and highlight ongoing challenges in detecting distal emboli.

Original authors: Yixin Zhang, Ryan Chamberlain, Lawrence Ngo, Kevin Kramer, Maciej A. Mazurowski

Published 2026-05-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yixin Zhang, Ryan Chamberlain, Lawrence Ngo, Kevin Kramer, Maciej A. Mazurowski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the human lungs as a vast, intricate city of highways (blood vessels). Sometimes, a traffic jam called a Pulmonary Embolism (PE)—a blood clot—blocks these highways. If not found quickly, it can be fatal. Doctors use special X-ray maps called CTPA scans to find these blockages.

For years, scientists have been trying to build "smart robots" (AI models) to automatically spot and outline these clots on the maps. However, this paper argues that the current state of these robots is a bit like a group of drivers trying to navigate a city using three different, poorly drawn maps and a set of confusing traffic rules.

Here is the story of what the researchers found, explained simply:

1. The Problem: A Messy Map Room

The researchers looked at all the previous attempts to build these AI robots. They found three major headaches:

Tiny, Inconsistent Maps: The public maps (datasets) everyone uses are very small. Some only show big traffic jams in the main highways, ignoring the tiny side streets. Others accidentally mark a shadow on the road as a traffic jam.
No Standard Rules: Some researchers measure success by how many slices of the map they get right, while others look at the whole 3D volume. It's like one driver saying, "I got the street name right," and another saying, "I got the whole neighborhood right," making it impossible to compare who is actually the best driver.
Cheating: Some studies accidentally let the robots "peek" at the test answers during training (using slices from the same patient in both training and testing), making them look smarter than they really are.

2. The Solution: Building a Better Map and a New Robot

To fix this, the team from Duke University and Minnesota Health Solutions did two things:

Created a "Gold Standard" Map: They gathered 490 unique patient scans and had expert radiologists carefully draw the clots on every single one, from the biggest main highways down to the tiniest side streets. They split this into a "training set" (430 scans) and a "test set" (60 scans) that no one had seen before.
Released a Free Robot: They built a new AI model using this high-quality data and made the "brain" (the weights) of the best robot available for everyone to download for free.

3. The Race: Which Robot Wins?

They tested nine different types of robots (some based on 2D flat images, some on 3D volumes; some using traditional math, some using modern "Transformer" tech).

The Winners:

3D is King: Robots that looked at the whole 3D volume of the lung at once were much better than those that looked at it slice-by-slice (2D).
- Analogy: Imagine trying to find a thin, flat piece of paper in a stack of books. If you only look at the edge of the stack (2D), you might miss it. If you look at the whole stack (3D), you can see the paper clearly.
Old School vs. New School: Surprisingly, the older, traditional "CNN" style robots beat the newer, flashier "Transformer" style robots.
- Analogy: It's like a seasoned mechanic (CNN) beating a high-tech self-driving car prototype (Transformer) at fixing a specific, tiny engine part because the mechanic has better "instincts" for small details when there isn't a massive amount of data to learn from.
The Best Model: The winner was a 3D U-Net with ResNet blocks. It was the most accurate at outlining the clots.

4. The "Uncanny Valley" of Errors

Here is a strange finding: All the robots made the exact same mistakes.
Even though the robots were built differently, when they failed, they failed on the same patients.

Analogy: Imagine a group of different chefs trying to bake a cake. Even though they use different recipes, they all burn the same specific layer of the cake. This suggests the problem isn't the recipe (the model architecture); the problem is the ingredients (the data). The "hard" cases are just too difficult for any current recipe to handle perfectly.

5. The "Distal" Problem: The Tiny Clots

The hardest clots to find are the tiny ones in the farthest reaches of the lungs (distal emboli).

The Issue: These clots are so small they are almost invisible on standard scans.
The Result: The robots (and even human doctors) struggle here. The study found that many "missed" clots were so tiny they were smaller than the resolution of the scan itself.
The Lesson: We need better maps that focus specifically on these tiny, hard-to-see clots.

6. The "Pre-Training" Surprise

The researchers tried a common trick: teaching the robot to first recognize "Is there a clot or not?" (a simple yes/no question) on thousands of other scans, and then teaching it to draw the clot.

The Result: It didn't help.
Analogy: It's like teaching a student to identify a "red car" from a photo, and then expecting them to be better at drawing the exact outline of a specific red car. The skills didn't transfer. The robot learned to spot the context of a clot (the surrounding blood vessels) rather than the clot itself.

7. Testing on Public Maps (The Reality Check)

When they took their best robot and tested it on the old, public maps (FUMPE, CAD-PE, READ), the results were messy.

Why? The public maps had different rules. One map might say "this whole vessel is a clot," while another says "only the tiny speck inside is a clot."
The Takeaway: The robot wasn't necessarily "bad"; it was just confused by the inconsistent instructions. This proves that we can't just compare models on different datasets; we need a single, high-quality standard.

Summary

This paper is a call to action for the AI community. It says: "Stop arguing over which robot is best when we are all using broken maps."

They provided a new, high-quality map (the dataset) and a top-tier robot (the open-weight model) to help everyone start from the same fair ground. Their main conclusion is that to get better at finding these life-threatening clots, we need to focus on better data (especially for tiny clots) and 3D thinking, rather than just inventing new, complex robot designs.

The "Open Weight" Model: The researchers have made their best-performing robot available for free at a GitHub link, allowing other scientists to use it immediately without having to rebuild it from scratch.

1. Problem Statement

Pulmonary Embolism (PE) is a life-threatening cardiovascular condition where timely detection is critical. While Computed Tomography Pulmonary Angiography (CTPA) is the clinical gold standard, automated voxel-level segmentation of PE remains challenging. The authors identify three major bottlenecks hindering progress in the field:

Data Limitations: Existing public datasets (FUMPE, CAD-PE, READ) are small, lack consistency in annotation criteria (e.g., some omit distal/sub-segmental PEs, while others annotate artifacts as PEs), and suffer from data leakage issues in prior studies.
Evaluation Flaws: Many studies use inappropriate metrics (e.g., slice-level ROC-AUC instead of volumetric Dice) or fail to report segmentation-specific metrics like Average Symmetric Surface Distance (ASSD).
Reproducibility: There is a lack of open-weight models and standardized baselines, making it difficult to compare new architectures fairly.

2. Methodology

Dataset Curation

The authors curated a high-quality, in-house dataset consisting of 490 unique CTPA scans (430 for training/validation, 60 for testing).

Annotations: Derived from board-certified radiologists' reports, covering acute and chronic PE across all anatomical levels (central, lobar, segmental, and sub-segmental/distal).
Preprocessing: Scans were converted to NIfTI format. 2D models used native axial slices; 3D models were resampled to a consistent voxel spacing of $(0.7373, 0.7373, 1.0)$ mm.
Augmentation: Included random affine scaling, rotations, cropping, and flips.

Experimental Design

The study employed three core experiments:

Architecture Benchmarking: Nine representative segmentation architectures (mix of CNNs and Vision Transformers, in both 2D and 3D) were trained and evaluated under a unified protocol.
Pretraining Analysis: The best 2D model (Inception U-Net) was tested with weights pretrained on the RSNA-PE classification dataset (7,929 volumes) versus random initialization to determine if classification pretraining aids segmentation.
Generalization Study: The best-performing model was evaluated on three public datasets (FUMPE, CAD-PE, READ) without fine-tuning to assess out-of-the-box generalizability.

Evaluation Metrics

Primary: Mean Dice Similarity Coefficient (mDSC) and Average Symmetric Surface Distance (ASSD).
Secondary: Embolus-level detection analysis using a strict volume-aware threshold (X% overlap) rather than simple pixel overlap, to distinguish true detection from accidental overlaps.

3. Key Contributions

Open-Weight Model & Dataset: The authors released the first open-weight PE segmentation model trained on a large, densely annotated dataset of 490 unique patients. The code and weights are publicly available.
Systematic Benchmark: A comprehensive comparison of 9 state-of-the-art architectures (including nnUNet, MedNeXt, Swin-UNETR, SegFormer, etc.) establishes a new baseline for the field.
Critical Analysis of Public Data: The study exposes significant inconsistencies in public datasets (e.g., CAD-PE annotating vessel interfaces, FUMPE missing distal PEs), explaining why models often fail to generalize to them.
Reproducibility Framework: By standardizing the training pipeline and evaluation metrics, the paper provides a reproducible framework for future PE segmentation research.

4. Key Results

Architectural Performance

3D vs. 2D: 3D models consistently outperformed 2D models. The best model was nnUNet3D with ResNet encoding blocks, achieving an mDSC of 0.713 and ASSD of 1.03 mm.
- Reasoning: 3D context helps models distinguish thin emboli perpendicular to the axial plane and filter out artifacts.
CNN vs. ViT: Within the same dimensionality, CNN-based models outperformed Vision Transformers (ViTs).
- Reasoning: Transformers lack strong inductive biases for small objects when training data is modest (n=430).
Top Performers:
- 3D: nnUNet3D-ResEncXL (mDSC 0.713)
- 2D: Inception-UNet (mDSC 0.652)

Pretraining Findings

Classification Pretraining Failed: Pretraining on the RSNA-PE classification dataset did not improve segmentation performance compared to random initialization.
- Insight: Features learned for slice-level classification (relying on anatomical context) do not transfer well to voxel-level boundary delineation required for segmentation.

Error Analysis & Generalization

Consistent Error Patterns: Despite different architectures, models exhibited highly correlated performance spectra across patients. If a model failed on a specific patient, others likely did too, suggesting intrinsic data difficulty rather than architectural failure.
Distal PE Challenge: The primary source of error (False Negatives and False Positives) was small, distal emboli and artifacts. The median size of error fragments was an order of magnitude smaller than true positives.
Public Dataset Discrepancies:
- FUMPE: High False Positive rate because the model correctly identified distal PEs that FUMPE annotations omitted.
- CAD-PE: High False Negative rate due to annotations covering vessel interfaces or artifacts rather than true emboli.
- READ: High False Negative rate for very small fragments (<0.01 mL), likely due to the high resolution of READ annotations capturing details lost during model resampling.

5. Significance and Future Directions

Paradigm Shift: The paper argues that future progress depends less on novel architecture search and more on data quality and quantity, specifically focusing on difficult cases (distal PEs).
Clinical Utility: The study highlights that even experienced radiologists struggle with sub-segmental PEs, suggesting that current "ground truth" is imperfect.
Recommendations:
- Future datasets must include comprehensive, consistent annotations for distal PEs.
- Evaluation should move away from slice-level metrics toward volumetric and detection-based metrics.
- The released model and weights serve as a robust baseline for the community to build upon, shifting the focus from "inventing new models" to "solving data scarcity and annotation noise."

In conclusion, this work provides a critical audit of the PE segmentation landscape, demonstrating that 3D CNNs (specifically nnUNet) remain the state-of-the-art, but the field's primary limitation is data inconsistency rather than algorithmic capability. The authors' open resources aim to catalyze reproducible and clinically relevant advancements.