💻 computer science

Towards reconstructing experimental sparse-view X-ray CT data with diffusion models

This paper investigates the application of diffusion models to experimental sparse-view X-ray CT reconstruction, revealing that while severe domain shifts cause hallucinations, diverse synthetic priors outperform narrow matched ones and annealed likelihood schedules can effectively mitigate forward model mismatches, ultimately demonstrating that performance gains from synthetic data do not automatically translate to real-world benchmarks.

Original authors: Nelas J. Thomsen, Xinyuan Wang, Felix Lucka, Ezgi Demircan-Tureyen

Published 2026-04-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Nelas J. Thomsen, Xinyuan Wang, Felix Lucka, Ezgi Demircan-Tureyen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, 3D jigsaw puzzle, but someone has thrown away 90% of the pieces. This is what Sparse-View X-ray CT is like. Doctors want to see inside your body (or an engineer wants to see inside a jet engine) using X-rays, but they want to do it quickly and with as little radiation as possible. To do this, they only take a few "snapshots" (projections) from different angles instead of a full 360-degree scan.

The problem? With so few snapshots, the computer has to guess what the missing pieces look like. If it guesses wrong, you get a blurry, distorted, or even fake image.

This paper is about teaching a super-smart AI (called a Diffusion Model) to be a better guesser, and figuring out why it sometimes fails when moving from "practice mode" to "real life."

Here is the breakdown of their journey, using some everyday analogies:

1. The Two Forces: The Artist vs. The Detective

To solve this puzzle, the AI has to balance two competing forces:

The Prior (The Artist): This is the AI's "common sense." It has seen thousands of images of what objects usually look like. It says, "I know this is a brain; brains usually look like this."
The Likelihood (The Detective): This is the actual data from the X-ray machine. It says, "The measurements I took show a line here and a curve there."

The AI has to listen to both. If it listens too much to the Artist, it might ignore the real data and "hallucinate" (imagine) details that aren't there. If it listens too much to the Detective, it might get confused by the noise in the data and create a messy, streaky mess.

2. The "Practice vs. Reality" Problem

The researchers wanted to see if AI trained on perfect, computer-generated puzzles could handle real, messy physical objects.

The Setup: They built a physical plastic object that looked like the famous "Shepp-Logan" test pattern (a standard math puzzle used in CT scans). They used a laser cutter to make it, then scanned it with a real X-ray machine.
The Training: They trained three different AIs:
1. The Purist: Trained only on perfect, computer-generated math puzzles.
2. The Realist: Trained only on the specific laser-cut plastic object.
3. The Generalist: Trained on a mix of both.

The Surprise Finding:
You might think the "Realist" (trained on the exact object) would win. But it didn't. In fact, it often failed miserably.

Why? The "Realist" was too rigid. It was like a chef who only knows how to cook one specific recipe. If the real object had a tiny scratch or a slight imperfection the AI hadn't seen, the chef panicked and ruined the dish.
The Winner: The Generalist (the mix) won. By seeing many different types of objects, it learned the concept of shapes and edges, not just the specific details of one object. It was flexible enough to adapt to the real world.

3. The "Physics Gap" (The Mismatch)

Even with a good AI, there was a second problem: The Physics Gap.

Imagine you are trying to navigate a city using a map.

The Simulation: The map is perfect. Every street is exactly where it should be.
The Real World: The map is slightly off. There's a new construction site, a detour, or a pothole that the map doesn't show.

In the lab, the computer simulates the X-ray process perfectly. But in the real world, the X-ray beam bends slightly, the machine vibrates, and the detector has noise.

The Result: When the AI tried to force the real, messy data to fit its perfect map, it started to hallucinate. It would shift the position of holes or distort shapes just to make the math work. It was like trying to force a square peg into a round hole, and the AI just squished the peg until it looked round, even though it was wrong.

4. The Solution: The "Annealing" Schedule

How do you fix this? The researchers found a clever trick called Annealing.

Think of this like tuning a radio or adjusting a camera lens.

Early Stage (High Noise): When the AI starts guessing, the image is just static noise. At this point, the AI needs to listen hard to the Detective (the real X-ray data) to get the general shape right. It needs to say, "Okay, I see a blob here, I'll lock onto that."
Late Stage (Clear Image): As the image becomes clearer, the Detective's instructions become noisy and unreliable because of the "Physics Gap" (the potholes and detours). If the AI keeps listening too hard to the Detective now, it will start squishing the image to fit the errors.
The Fix: The researchers told the AI to slowly turn down the volume on the Detective's instructions as the image gets clearer.
- Start: "Listen to the data! Get the shape right!"
- End: "Trust your common sense (the Artist) to fill in the details and smooth out the noise."

This "schedule" allowed them to get great results even with very few calculation steps, making the process faster and more accurate.

The Big Takeaway

The paper teaches us that you can't just copy-paste a solution from a computer simulation to the real world.

Don't be too specific: An AI trained on a narrow, perfect dataset is brittle. An AI trained on a diverse mix is more robust.
Respect the mess: Real-world physics is messy. You can't just force the data to fit the model; you have to let the model "relax" and ignore the small errors in the data.
Timing is everything: You need to listen to the data early on, but trust the AI's "gut feeling" more as the picture comes into focus.

In short: To see the world clearly through a foggy window, you need a smart guide who knows what things usually look like, but who also knows when to stop guessing and just accept the blurry reality.

1. Problem Statement

The paper addresses the challenge of applying diffusion-based image generators to sparse-view X-ray Computed Tomography (CT) reconstruction. While diffusion models have shown promise as priors for ill-posed inverse problems in synthetic settings, their application to experimental (real-world) data remains unproven.

Two primary sources of mismatch hinder the transition from simulation to reality:

Prior Mismatch (Domain Shift): Diffusion models are typically trained on idealized, synthetic datasets (e.g., standard Shepp-Logan phantoms). Real-world objects possess geometric inaccuracies, textural variations, and low-contrast features not present in training data, leading to "brittle" priors that fail on unseen objects.
Forward Model Mismatch: Standard reconstruction assumes a perfect mathematical projector. Experimental scans, however, suffer from beam hardening, scatter, geometric calibration errors, and complex noise textures. This discrepancy forces solvers to either amplify artifacts to fit incorrect data or rely too heavily on the prior, causing hallucinations.

The core question is: Do performance gains observed in synthetic data translate to experimental settings, and how can inference strategies be adapted to bridge the "sim-to-real" gap?

2. Methodology

A. Experimental Setup & Data Generation

To create a controlled environment for testing, the authors designed a physical phantom laser-cut from a PMMA plate, mimicking the standard Shepp-Logan (SL) phantom but with physical imperfections.

Scanner: Custom-built cone-beam CT system (70 kVp, 600 µA).
Data: 901 projections over 180°.
Test Domains: The authors evaluated performance across four progressively difficult domains:
1. Standard Simulation ( $y_{sim(std)}$ ): Idealized SL phantom.
2. CAD Simulation ( $y_{sim(cad)}$ ): Simulated projections from the laser-cut design (introduces geometric shift).
3. Reconstruction Simulation ( $y_{sim(recon)}$ ): Simulated projections from a full-view reconstruction of the physical phantom (introduces realistic textures).
4. Experimental Measurement ( $y_{exp}$ ): Actual raw data from the physical scanner (introduces full physics mismatch).

B. Training Priors

Three distinct diffusion priors were trained on 10,000 images each to test the impact of training distribution:

$f_{std}$ (Standard): Trained on perturbed standard SL ellipses (narrow, high-contrast distribution).
$f_{exp}$ (Experimental): Trained on perturbed ellipses matching the physical laser-cut phantom (tailored to specific geometry).
$f_{mix}$ (Mixed): Trained on a combination of standard and experimental ellipses (diverse, broad distribution).

C. Reconstruction Pipeline

The authors employed the Decomposed Diffusion Sampler (DDS):

Mechanism: A plug-and-play solver that combines a learned prior direction ( $\epsilon_\theta$ ) with a physics-based data direction (Conjugate Gradient steps).
Forward Operator: Modeled as a fan-beam geometry with sparse projections.
Inference Strategy: The paper investigates the impact of likelihood weighting ( $\gamma$ ). They compare a constant weight schedule against a linear decay (annealed) schedule, where $\gamma$ starts high (enforcing data consistency early) and decreases (allowing the prior to refine details later).

3. Key Contributions

Analysis of Prior Diversity vs. Specificity: The study demonstrates that while a prior specialized to the exact experimental geometry ( $f_{exp}$ ) performs well in simulation, it can fail or underperform on real data. Conversely, a diverse prior ( $f_{mix}$ ) trained on both standard and experimental variations offers superior robustness, outperforming specialized priors even on specific targets.
Characterization of Forward Model Mismatch: The authors identify that forward model mismatch does not just cause streak artifacts but leads to structural hallucinations (spatial shifts and feature distortion) as the solver attempts to force the image to fit physically inconsistent projections.
Annealed Likelihood Scheduling: The paper proposes and validates a linear decay schedule for the likelihood weight ( $\gamma$ ). This strategy mitigates the sim-to-real gap by prioritizing global geometry alignment in high-noise early steps and relaxing constraints in later steps to prevent the model from overfitting to experimental noise and artifacts.
Computational Efficiency: The study shows that the annealed scheduling strategy maintains high reconstruction quality even with drastically reduced sampling steps (NFE=100), making diffusion-based CT feasible for practical applications.

4. Key Results

Impact of Training Distribution:
- In idealized simulations, the specialized prior ( $f_{std}$ ) performed best.
- In experimental domains, the specialized prior ( $f_{exp}$ ) suffered from geometric mismatch.
- Surprising Finding: The mixed prior ( $f_{mix}$ ) consistently outperformed the specialized priors in experimental settings. The exposure to high-contrast standard structures provided better edge-learning signals that transferred to the low-contrast experimental phantom, smoothing out residual streaks.
Impact of Forward Model Mismatch:
- Performance dropped significantly when moving from simulated projections ( $y_{sim(recon)}$ ) to real measurements ( $y_{exp}$ ), even at high resolutions (512x512).
- This confirmed that physical factors (beam hardening, scatter) are a dominant source of error that cannot be solved by resolution alone.
Inference Strategy (Scheduling):
- Constant Weighting: High constant weights ( $\gamma=5$ ) worked well in simulation (26.82 dB) but caused catastrophic failure in experimental data (14.07 dB) due to global intensity drift and artifact amplification.
- Linear Decay: The annealed schedule ( $\gamma_t$ decaying from 150 to 0) recovered performance, achieving 25.69 dB on experimental data (24 views).
- Efficiency: With only 100 sampling steps (NFE), the linear decay schedule achieved the highest SSIM (0.851) on real data, proving that fewer steps are sufficient if the weighting schedule is optimized.

5. Significance and Conclusion

This paper provides a critical reality check for the application of diffusion models in medical and industrial imaging. It highlights that performance gains in synthetic benchmarks do not automatically translate to experimental success.

Robustness over Specialization: The findings suggest that training on diverse, mixed distributions is more effective than training on narrow, perfectly matched datasets for real-world applications.
Inference is Key: The success of diffusion-based CT in real settings depends heavily on inference strategies (specifically, annealed likelihood weighting) that balance the tension between the imperfect forward model and the generative prior.
Future Directions: The authors note that while 2D phantoms provide a controlled "best-case" scenario, future work must address 3D complexities, extreme noise regimes, and the development of evaluation metrics that better capture perceptual fidelity beyond PSNR/SSIM.

In summary, the paper demonstrates that with the right prior diversity and a carefully scheduled inference process, diffusion models can be made robust enough for sparse-view experimental CT, offering a practical path toward reducing radiation dose and scan times.