⚡ electrical engineering

Promise and challenges of heart chamber segmentation from non-contrast CT scans using contrastive unpaired image translation: a feasibility study

This feasibility study introduces ChameleonNet, a framework combining contrastive unpaired image translation and deep learning to enable heart chamber segmentation from non-contrast CT scans without manual annotations, achieving high segmentation accuracy on synthesized data while demonstrating promising but imperfect volume agreement on real clinical scans that requires further refinement.

Original authors: Jing Wang, Tong Yu, Hao-En Lu, Zixue Zeng, Joseph K. Leader, Xin Meng, Jianbing Zhu, Jiantao Pu

Published 2026-06-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jing Wang, Tong Yu, Hao-En Lu, Zixue Zeng, Joseph K. Leader, Xin Meng, Jianbing Zhu, Jiantao Pu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Turning "Gray" into "Color" to Find the Heart

Imagine you are trying to find a specific room inside a house, but you only have a black-and-white photo where the walls, furniture, and floors all look like shades of gray. It's incredibly hard to tell where the bedroom ends and the bathroom begins.

Now, imagine you have a second photo of the same house, but this one is in full, vibrant color. The walls are blue, the floor is wood, and the furniture pops out. It's easy to see the rooms.

The Problem:
Doctors often have the "black-and-white" photos (Non-Contrast CT scans) of patients' chests. These are cheap, easy to get, and safe. However, because the heart chambers (the four rooms of the heart) look so similar in gray, it is very difficult for computers (or even humans) to draw a line around them accurately.

Usually, to get the "color" photo (Contrast CT), doctors have to inject dye into the patient. This is expensive and carries small risks. The researchers wanted to know: Can we use the "color" photos to teach a computer how to find the rooms in the "black-and-white" photos, without ever needing to manually draw lines on the gray photos?

The Solution: "ChameleonNet"

The team built a two-step robot system they call ChameleonNet. Think of it as a two-person team working together.

Step 1: The Master Painter (Image Translation)

The first robot is a "Master Painter." Its job is to take a "color" photo (Contrast CT) and paint over it to look exactly like a "black-and-white" photo (Non-Contrast CT).

The Challenge: If the painter just changes the colors randomly, the shape of the heart might get distorted. If the heart looks squished or stretched, the next step won't work.
The Innovation: The researchers gave this painter a special new tool called Decoupled Contrastive Learning.
- The Analogy: Imagine a teacher trying to teach a student to recognize a cat.
  - Old Way (Standard Learning): The teacher shows a cat and says, "This is a cat. That dog is not a cat. That car is not a cat." The student gets confused because they are trying to learn what a cat is and what it isn't at the same time.
  - New Way (Decoupled Learning): The teacher says, "Focus only on what makes this cat a cat. Ignore the other things for a moment."
- By separating these lessons, the painter learns much faster (about 39% faster!) and creates "fake" black-and-white photos that look so real, they fool the experts. Crucially, the shape of the heart stays exactly the same as the original "color" photo.

Step 2: The Detective (Segmentation)

The second robot is a "Detective." It looks at the "fake" black-and-white photos created by the painter.

Because the painter kept the shapes perfect, the Detective can use the "color" photo's labels (the known locations of the heart rooms) to learn how to find those same rooms in the "fake" black-and-white photo.
The researchers added a special rule to the Detective: "Watch the edges closely." (This is the Hausdorff Distance Loss).
- The Analogy: If you are tracing a map, you don't just want to be in the right neighborhood; you want to trace the border of the street perfectly. This rule forces the computer to be very precise about the boundaries of the heart chambers.

What Did They Find?

The team tested this system on real patients. Here is the verdict:

The Painter is Great: The "fake" black-and-white photos looked almost identical to real ones. The heart shapes were preserved perfectly.
The Detective is Good, But Not Perfect:
- When the computer looked at the "fake" photos, it was excellent at finding the four heart chambers (Left/Right Atrium and Left/Right Ventricle).
- When they tested it on real black-and-white photos (which the computer had never seen before), it did a very good job, but not a perfect one.
- The Results: The computer's measurements of the heart rooms matched the "color" photo measurements very closely (correlation scores between 0.82 and 0.93).
- The Glitch: The computer sometimes guessed the size of the Left Ventricle (the main pumping chamber) a bit too small or the Right Ventricle a bit too big. The error rates were around 9% to 20%.

Why This Matters (According to the Paper)

The paper concludes that this method is feasible. It proves that you can take easy-to-get "black-and-white" scans, use a "color" scan to teach the computer what to look for, and get a decent estimate of the heart's size without needing to manually draw lines on the difficult gray images.

However, the paper is very clear about the limits:

It is not ready for clinical use yet.
The errors in measuring the size of the heart chambers (especially the ventricles) are still too high for a doctor to rely on it 100% for diagnosis.
The main reason for the errors is that the "color" and "black-and-white" scans were taken at different times. The heart beats and changes size, so the two photos aren't perfectly aligned in time, which confuses the computer slightly.

Summary

Think of this study as building a translator that can read a difficult language (gray heart scans) by first learning from an easy language (color heart scans). The translator works well enough to understand the general story, but it still stumbles on the specific details of the sentence structure. It's a promising start, but it needs more practice before it can be trusted to write the final report for a doctor.

Technical Summary: ChameleonNet for Heart Chamber Segmentation from Non-Contrast CT

Problem Statement
Accurate segmentation of heart chambers (left atrium, left ventricle, right atrium, and right ventricle) from non-contrast CT scans is highly desirable for opportunistic screening, cost reduction, and radiation therapy planning. However, manual annotation on non-contrast CT is extremely difficult due to poor soft-tissue contrast, leading to high interobserver variability. While deep learning models like nnU-Net perform well on contrast-enhanced CT, they struggle on non-contrast data. Existing solutions face significant hurdles: multi-atlas methods suffer from limited generalizability, and supervised deep learning requires extensive manual labeling on non-contrast scans, which is scarce. Furthermore, utilizing annotations from contrast CT is complicated by the fact that contrast and non-contrast scans from the same patient are rarely perfectly aligned (paired) due to differences in acquisition timing and lack of ECG gating.

Methodology: ChameleonNet
The authors propose ChameleonNet, a two-stage framework designed to segment heart chambers from non-contrast CT without requiring manual annotations on those specific scans. The approach leverages unpaired image translation to synthesize non-contrast images from contrast-enhanced scans, thereby transferring existing contrast annotations to the non-contrast domain.

Stage I: Contrastive Unpaired Image Translation (CUT) with Decoupled Learning
- Architecture: The framework utilizes a generator ( $G$ ) and discriminator ( $D$ ) based on the Contrastive Unpaired Translation (CUT) architecture.
- Innovation (DCE Loss): The authors introduce a Decoupled Contrastive Learning (DCE) loss to replace the standard InfoNCE loss used in CUT. In standard contrastive learning, the "negative-positive coupling" (NPC) effect can cause vanishing gradients when positive pairs are too similar or negative pairs are too distinct. The DCE loss removes the positive pair term from the denominator of the loss function, effectively decoupling the gradients. This accelerates training and improves efficiency.
- Process: The generator translates contrast-enhanced images ( $I_c$ ) into synthetic non-contrast images ( $I_{nc}$ ) while preserving anatomical structure. The training uses multilayer patch-wise contrastive loss to maintain feature consistency across scales.
- Data: The translation model was trained on 35,538 contrast and 37,197 non-contrast CT slices from unpaired datasets.
Stage II: Segmentation with Hausdorff Distance Loss
- Architecture: A modified nnU-Net is trained to segment the four heart chambers using the synthetic non-contrast images generated in Stage I.
- Loss Function: The segmentation model employs a composite loss function including Cross-Entropy, Dice loss, and a Hausdorff Distance (HD) loss. The HD loss is specifically added to regularize the segmentation boundaries, addressing the challenge of precise delineation in low-contrast regions.
- Training: The model was trained on 292 synthesized non-contrast scans using annotations derived from the original contrast scans.

Key Contributions

Decoupled Contrastive Learning: The integration of DCE loss into the CUT framework accelerates training by approximately 36–39% compared to CycleGAN and standard CUT, while improving image translation quality (higher SSIM, lower FID).
Unpaired Domain Adaptation: The method successfully bridges the gap between contrast and non-contrast CT without requiring paired, ECG-gated data, which is rare in clinical practice.
Boundary Regularization: The inclusion of Hausdorff distance loss in the segmentation stage demonstrated specific improvements in volume agreement for real non-contrast scans, particularly for the left ventricle.
Large-Scale Validation: The study utilizes a large dataset for translation training and includes an independent test set of 36 real, non-contrast CT scans for rigorous evaluation, moving beyond synthetic-only validation.

Results

Image Translation: ChameleonNet achieved a mean Structural Similarity Index Measure (SSIM) of 0.96 and a Fréchet Inception Distance (FID) of 13.29, outperforming both CycleGAN and standard CUT. Qualitative analysis showed better preservation of structural edges and lung intensity characteristics compared to baselines.
Segmentation on Synthetic Data: On synthesized non-contrast images, the model achieved high Dice Similarity Coefficients (DSC): LA (0.94), LV (0.91), RA (0.92), and RV (0.93). The addition of Hausdorff loss showed minimal difference on synthetic data.
Segmentation on Real Non-Contrast Data: When applied to real non-contrast CT scans, the model showed strong volume agreement with contrast-derived ground truth:
- Pearson Correlations: LA (0.93), LV (0.82), RA (0.87), RV (0.89).
- Error Metrics: Mean Absolute Percentage Error (MAPE) ranged from 9.22% (LA) to 20.79% (LV).
- Impact of HD Loss: The Hausdorff distance loss significantly improved Left Ventricle (LV) volume agreement on real scans, increasing the Pearson correlation from 0.73 to 0.82 and reducing MAPE from 34.75% to 20.79%.
- Limitations: Systematic errors persisted, particularly an underestimation of the Right Ventricle (MPE = -12.52%) and higher errors in the LV, likely due to cardiac phase misalignment between the unpaired scans.

Significance and Claims
The authors position this work as a feasibility study demonstrating that heart chamber segmentation from non-contrast CT is achievable without manual non-contrast annotations. By synthesizing non-contrast images from contrast scans, the framework effectively transfers valuable annotations from the contrast domain to the non-contrast domain.

The paper claims that ChameleonNet offers a practical solution for clinical scenarios where paired data is unavailable, preserving native CT resolution and intensity. However, the authors remain modest regarding clinical deployment. They explicitly state that while the method is feasible, residual volume errors (especially for LV and RV) indicate that further refinement and validation are necessary before clinical use. The study highlights that the lack of ECG-gated paired data introduces unavoidable variability, preventing direct voxel-level evaluation and suggesting that future work must address cardiac phase alignment to improve accuracy.