DM4CT: Benchmarking Diffusion Models for Computed Tomography Reconstruction

This paper introduces DM4CT, a comprehensive benchmark that systematically evaluates ten diffusion-based methods against seven strong baselines for computed tomography reconstruction across diverse medical and industrial datasets, including a newly acquired high-resolution real-world dataset, to analyze the practical challenges and performance of diffusion models in this domain.

The Gist

Imagine you are trying to solve a giant, complex jigsaw puzzle, but someone has thrown away 80% of the pieces, smudged the remaining ones with coffee, and glued a few fake pieces onto the board. This is essentially what Computed Tomography (CT) does. It tries to build a 3D picture of the inside of your body (or a rock, or a machine part) using X-rays, but the data it collects is often incomplete, noisy, and messy.

For decades, scientists have used mathematical "rules of thumb" to guess what the missing pieces look like. But recently, a new type of AI called Diffusion Models has become famous for creating stunningly realistic images from scratch (like generating pictures of cats or landscapes from text).

The paper "DM4CT" asks a big question: Can these powerful AI image generators be used to fix our broken CT puzzles?

Here is the breakdown of the paper using simple analogies:

1. The Problem: The "Broken Puzzle"

CT scans are like taking a photo of an object from the outside, but you only get to see it from a few angles.

• The Challenge: If you only have 20 photos of a car instead of 360, a computer has to guess what the back looks like.
• The Mess: Real-world X-rays aren't clean. They have "static" (noise) and weird rings (artifacts) caused by the machine itself.
• The Old Way: Traditional methods try to solve this with strict math. They are like a rigid robot trying to force the puzzle pieces together. They work okay, but the result often looks blurry or has strange streaks.

2. The New Contender: The "Imaginative Artist"

Diffusion models are like highly trained artists. They have studied millions of images and learned what "real" things look like.

• How they work: Imagine an artist who starts with a canvas covered in static (noise) and slowly wipes it away, revealing a clear image.
• The Idea: If we tell this artist, "Hey, the X-rays we have must match this blurry pattern," can they use their knowledge of what a "real" liver or a "real" rock looks like to fill in the missing gaps better than the rigid robot?

3. The Experiment: The "DM4CT" Arena

The authors created a massive testing ground called DM4CT (Diffusion Models for CT). Think of this as an Olympic Games for AI reconstruction.

• The Athletes: They tested 10 different "Diffusion" strategies (different ways the AI tries to solve the puzzle) against 7 "Old School" methods (the rigid robots).
• The Obstacle Course: They didn't just test on perfect pictures. They tested on:
  • Medical Scans: Human lungs and organs (with privacy-safe data).
  • Industrial Scans: A tube filled with weird materials like pine nuts and coriander.
  • The "Real World" Test: They went to a giant particle accelerator (a Synchrotron) to scan real rocks. This is the "final boss" level because real rocks are messy, and the data is imperfect.

4. The Results: Who Won?

The results were surprising and nuanced:

• The "Artistic" AI (Diffusion Models):

  • Pros: They are amazing at filling in the blanks. When the data is very sparse (few angles) or very noisy, the AI often produces images that look sharper and more detailed than the old methods. They can "hallucinate" (guess) fine textures that make the image look realistic.
  • Cons: Sometimes they get too creative. They might invent a texture that looks real but isn't actually there. It's like an artist who paints a beautiful flower on a rock because they think it should be there, even if it's not.
  • The Catch: They are computationally heavy. Running them takes a lot of time and powerful computers, like trying to paint a masterpiece in real-time.

• The "Rigid Robot" (Traditional Methods):

  • Pros: They are fast and reliable. They never invent fake details; they just stick strictly to the data they have.
  • Cons: The images often look blurry or have "streaks" when the data is bad.

• The "Supervised Learner" (SwinIR):

  • This was a special AI trained specifically on thousands of perfect CT pairs. It often won the "score" game (mathematical accuracy) but sometimes produced images that looked a bit too smooth, like a plastic mannequin, losing the tiny, gritty details.

5. The Big Takeaways

The paper concludes that while Diffusion Models are incredibly powerful, they aren't a "magic wand" yet.

• The Balancing Act: The hardest part is telling the AI, "Use your imagination to fill in the gaps, BUT don't make up things that contradict the X-ray data." If you push the AI too hard to follow the data, the image gets noisy. If you let it imagine too much, it invents fake structures.
• The Real-World Gap: The AI performed great on computer simulations but struggled a bit more on the real rock scans. This is because real-world data is messier than the training data the AI learned from.
• The Future: We need better ways to train these AIs on real-world data and to make them faster.

Summary Analogy

Imagine you are trying to restore an old, torn, and stained photograph of your grandmother.

• Traditional Methods are like a conservative restorer who only fills in the missing parts with the exact same color as the surrounding pixels. The result is safe but looks flat and blurry.
• Diffusion Models are like a talented artist who knows what your grandmother usually looks like. They can fill in the missing eyes and smile beautifully. But sometimes, they might accidentally give her a different hairstyle because they are "guessing" based on what they've seen in other photos.

DM4CT is the report card that tells us: "The artist is very talented and can fix the photo better than the conservative restorer in many cases, but we need to teach them to be careful not to change the story too much."

Technical Summary

1. Problem Statement

Computed Tomography (CT) is a classic inverse problem where an unknown object is reconstructed from indirect projection measurements. While theoretically linear, practical CT reconstruction faces significant challenges:

• Ill-posedness: Measurements are often sparse (limited angles) or noisy, leading to ambiguity where multiple solutions fit the data.
• Complex Artifacts: Real-world data suffers from correlated noise, ring artifacts, and non-linear preprocessing steps (e.g., log transformations) that deviate from idealized linear models.
• Domain Gap: Diffusion models, highly successful in natural image generation, struggle when applied directly to CT due to mismatched value ranges, specific noise characteristics (Poisson vs. Gaussian), and the need for strict data consistency.

There is currently no systematic benchmark to evaluate how well diffusion models handle these specific CT challenges compared to established reconstruction methods.

2. Methodology: The DM4CT Benchmark

The authors introduce DM4CT, a comprehensive framework designed to rigorously evaluate and compare diffusion-based reconstruction methods against strong baselines.

A. Datasets and Configurations

The benchmark utilizes three distinct data sources to ensure robustness:

1. Medical CT: The 2016 Low Dose CT Grand Challenge dataset (patient volumes).
2. Industrial CT: The LoDoInd dataset (tube with diverse materials).
3. Real-World Synchrotron CT: A newly acquired, high-resolution dataset of rock samples scanned at a synchrotron facility (24 keV, 1200 projections). This provides a realistic testbed with high spatial resolution and minimal preprocessing.

Simulation Configurations: To test robustness, the authors simulate five scenarios varying in:

• View Sparsity: 20, 40, and 80 projection angles.
• Noise Levels: Mild to high Poisson noise.
• Artifacts: Inclusion of ring artifacts.

B. Method Taxonomy

The paper categorizes 10 recent diffusion-based methods based on how they incorporate data consistency (ensuring the reconstruction matches measurements) and prior knowledge (the learned diffusion model). The strategies include:

• Data Consistency Gradient (DC-grad): Steers the denoising process using gradients of the data fidelity term (e.g., DPS, MCG).
• Data Consistency Optimization (DC-step): Inserts full optimization steps between denoising iterations (e.g., ReSample).
• Plug-and-Play: Alternates between data fidelity subproblems and unconditional denoising.
• Pseudo-Inverse: Uses approximate inverse solutions (e.g., FBP) to guide the diffusion process.
• Variational Bayesian: Approximates the posterior distribution directly without iterative sampling.

C. Baselines and Implementation

The diffusion methods are compared against:

• Classical: FBP, SIRT.
• Model-Based Iterative Reconstruction (MBIR): ADMM-PDTV, FISTA-SBTV.
• Unsupervised/Implicit Priors: Deep Image Prior (DIP), Implicit Neural Representations (INR).
• Supervised Learning: SwinIR (Transformer-based).

All methods are implemented using the diffusers library to ensure fair comparison, with shared backbones (Pixel-space and Latent-space diffusion models) trained on the respective datasets.

3. Key Contributions

1. First Systematic Benchmark: DM4CT is the first comprehensive benchmark specifically for diffusion models in CT reconstruction.
2. Real-World Dataset Release: The authors acquired and released a high-energy synchrotron CT dataset, offering a rare resource for evaluating methods under realistic, high-resolution conditions.
3. Unified Taxonomy: A classification system for diffusion methods based on their data consistency strategies (Table 1 in the paper).
4. Open-Source Codebase: Full implementation of all benchmarked methods is open-sourced.
5. Comprehensive Analysis: Extensive experiments covering quantitative metrics (PSNR, SSIM), qualitative visual inspection, computational efficiency, and uncertainty quantification.

4. Key Results and Findings

A. Performance vs. Baselines

• Diffusion vs. Classical: Diffusion models generally outperform classical methods (FBP, SIRT) and MBIR in terms of PSNR/SSIM, particularly in sparse-view and high-noise scenarios.
• Diffusion vs. Supervised: Fully supervised methods (SwinIR) often achieve the highest quantitative scores but tend to produce overly smooth images, losing high-frequency structural details.
• Diffusion vs. INR: Implicit Neural Representations (INR) perform competitively, often matching diffusion models in noiseless scenarios and real-world datasets, sometimes outperforming them in quantitative metrics.

B. The Trade-off: Prior vs. Data Consistency

• Gradient vs. Optimization: Methods using strict data consistency optimization steps (e.g., ReSample) perform well in noise-free settings but tend to overfit to noise in noisy scenarios, degrading image quality. Conversely, gradient-based methods (e.g., DPS) are more robust to noise but may introduce structural distortions.
• Latent Space Challenges: Latent diffusion models (LDMs) face difficulties in latent space. Enforcing data consistency via gradients in the latent space often leads to discontinuities and artifacts because gradients must propagate through the VQ-VAE decoder. Explicit optimization steps help but are computationally expensive.

C. Real-World Limitations

• Performance Drop: Diffusion models show a significant performance drop on the real-world synchrotron dataset compared to simulated data. This is attributed to distribution shifts, limited training data quality, and the difficulty of modeling complex physical noise/artifacts.
• Hallucination: While diffusion models recover fine details, these details can be "hallucinated" (plausible but incorrect), leading to a divergence from the ground truth despite high visual appeal.
• Value Range Mismatch: A practical challenge identified is the misalignment of value ranges (e.g., Hounsfield Units) between training and test data, requiring empirical linear mapping corrections.

D. Computational Efficiency

• Training Cost: Latent diffusion requires training a VQ-VAE first, making total training time comparable to or higher than pixel-space diffusion, despite using less memory during inference.
• Inference: Pixel diffusion is generally more memory-efficient than latent diffusion. SwinIR is the fastest at inference but requires massive memory for training.

5. Significance and Future Directions

• Bridging the Gap: DM4CT highlights that while diffusion models are powerful priors, their direct application to CT is hindered by the gap between idealized generative assumptions and the physical realities of CT imaging (noise models, geometry, artifacts).
• Guidance for Future Work: The paper suggests several promising directions:
  • Integrating flow-based models (e.g., FlowDPS).
  • Combining INRs with diffusion priors to leverage structural fidelity.
  • Adapting natural-image autoencoders specifically for scientific imaging (the paper found VQ-VAE outperformed fine-tuned SDXL autoencoders for CT).
  • Developing better strategies for value range alignment and handling non-Gaussian noise.

In conclusion, DM4CT establishes that diffusion models are a viable and powerful tool for CT reconstruction, particularly in challenging sparse-view regimes, but their practical deployment requires careful tuning of data consistency strategies and further research into domain adaptation.