⚡ electrical engineering

Gaussian splatting for sparse-view X-ray phase-contrast reconstruction in MHz tomoscopy

This paper presents an adapted 3D Gaussian Splatting framework that enables rapid, high-quality 3D reconstruction of phase-dominant volumes from sparse-view X-ray projections by integrating absorption and phase-shift primitives with differentiable Fresnel propagation.

Original authors: Daniel Moško, Peter Szeles, Igli Draci, Patrik Vagovič, Jozef Uličný

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daniel Moško, Peter Szeles, Igli Draci, Patrik Vagovič, Jozef Uličný

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Taking a 3D Photo with Only Four Angles

Imagine you want to build a 3D model of a moving object, like a red blood cell rushing through a vein. Usually, to build a 3D model, you need to walk all the way around the object, taking hundreds of photos from every angle (like a CT scan).

But this paper is about MHz Tomoscopy, a super-fast X-ray camera. It takes a "snapshot" of a scene so fast that the object doesn't have time to move. The catch? Because it's so fast, the camera can only take four photos at once from four different angles.

Trying to build a full 3D model from just four photos is like trying to guess the shape of a hidden sculpture by only looking at four tiny shadows. It's a huge puzzle with missing pieces. Furthermore, soft tissues (like blood cells) don't block X-rays well; they mostly shift them slightly. Standard X-ray cameras see this as "nothing," resulting in a blank image.

The Solution: 3D "Gaussian Splatting"

The authors took a technique called 3D Gaussian Splatting (originally designed for making realistic 3D video games) and adapted it for X-rays.

The Analogy: The Cloud of Glowing Dust
Imagine the 3D object isn't made of solid blocks, but of millions of tiny, glowing, fuzzy clouds (Gaussians).

In Video Games: Each cloud has a color (Red, Green, Blue) and a transparency (how see-through it is).
In This X-Ray Version: The authors replaced "Color" with Physics.
- Instead of Red/Green/Blue, each cloud holds two numbers:
  1. Absorption ( $\beta$ ): How much it eats the X-ray (like a dark cloud).
  2. Phase-Shift ( $\delta$ ): How much it bends or delays the X-ray (like a lens).

How It Works: The "Magic Lens" Pipeline

The system tries to arrange these millions of fuzzy clouds to match the four X-ray photos it has. Here is the step-by-step process:

The Setup: The computer starts with a random cloud of dust.
The Simulation: It pretends to shine X-rays through this cloud.
The "Magic Lens" (Fresnel Propagation): This is the most critical part. When X-rays pass through soft tissue, they don't just get dimmer; they create a specific pattern of bright and dark rings (fringes) at the edges, like ripples in a pond. The computer uses a special math formula (Fresnel propagation) to predict these ripples.
The Comparison: The computer compares its predicted "ripple pattern" with the actual X-ray photo.
The Adjustment: If the patterns don't match, the computer nudges the clouds—moving them, changing their size, or adjusting their "bending" power—until the ripples look exactly right.

The Red Blood Cell Experiment

Since they didn't have real X-ray photos of this specific fast process yet, they built a virtual training ground:

They simulated red blood cells moving in fluid using physics software.
They generated fake X-ray photos with the characteristic "bright-dark edge" ripples.
They fed these fake photos into their system.

The Results:

Speed: The system reconstructed a detailed 3D model of a red blood cell in about 15 minutes on a single powerful computer chip.
Accuracy: It successfully figured out the 3D shape and the "bending" properties of the cell using only 12 of the available views (leaving 4 for testing).

The Surprising Discovery: "More is Not Always Better"

The authors found a counter-intuitive lesson about how to measure success.

The Trap: If you use a lot of clouds (244,000), the computer gets really good at matching the background brightness. It gets a high score on a standard test called "PSNR."
The Reality: But, it fails to recreate the crucial "ripple patterns" (the phase information). It's like a painter who matches the color of the sky perfectly but forgets to paint the clouds.
The Fix: By using fewer clouds (8,000) but tuning the "learning speed" carefully, the system ignored the background and focused on the ripples. This resulted in a much better, more accurate 3D model, even though the standard score looked lower.

Why This Matters

This paper proves that you can use this "cloud of physics" method to see inside fast-moving, soft objects using very few X-ray angles. It turns a nearly impossible math problem (reconstructing 3D from 4 angles) into a solvable one by using a smart, differentiable "cloud" system that understands how X-rays actually behave.

In short: They taught a computer to build a 3D model of a moving blood cell by arranging millions of invisible, physics-powered "fuzzy clouds" until they perfectly recreated the unique ripples seen in just a handful of X-ray snapshots.

Technical Summary: Gaussian Splatting for Sparse-View X-ray Phase-Contrast Reconstruction in MHz Tomoscopy

Problem Statement
MHz tomoscopy enables the imaging of fast dynamic phenomena (e.g., shock waves, cavitation, biological fluid dynamics) by capturing simultaneous X-ray projections from multiple angles within a single free-electron laser (XFEL) pulse. However, this approach faces a severe data limitation: the number of angular views is drastically restricted (typically four in current XFEL implementations) compared to the hundreds required for conventional computed tomography. This renders the 3D reconstruction problem highly underdetermined.

The challenge is further compounded by the physics of soft biological specimens. At hard X-ray energies, the refractive-index decrement ( $\delta$ ) dominates over the absorption index ( $\beta$ ) by several orders of magnitude. Standard attenuation-based reconstruction algorithms fail to produce contrast in this phase-dominant regime. Existing sparse-view neural approaches, such as ONIX, operate exclusively in the absorption regime and cannot recover $\delta$ , while traditional voxel-based iterative methods require dense angular sampling. There is a critical need for a scene representation that is compact, fast, differentiable, and physically faithful to phase-contrast physics.

Methodology
The authors adapted 3D Gaussian Splatting (3DGS), originally developed for photorealistic novel-view synthesis, to X-ray phase-contrast imaging. The core modifications include:

Scene Representation: The standard 3DGS primitives, which carry spherical-harmonic color coefficients and opacity, were replaced with physical parameters: the absorption index ( $\beta$ ) and the refractive-index decrement ( $\delta$ ) of the complex refractive index $n = 1 - \delta + i\beta$ . These values are constrained to be non-negative.
Geometry and Projection: Perspective projection was replaced with parallel-beam geometry consistent with MHz tomoscopy. The standard RGB alpha-blending compositing was replaced by a two-channel accumulation model that integrates $\beta$ and $\delta$ along each ray.
Differentiable Forward Model: A critical innovation is the integration of a differentiable Fresnel propagation model. The accumulated transmission maps are passed through this model to reproduce the near-field diffraction holograms recorded at the detector, including a Gaussian point-spread function (PSF) to simulate detector blur. This makes the entire forward model differentiable with respect to all Gaussian parameters, enabling gradient-based joint optimization of geometry and material properties from intensity-only measurements without explicit phase retrieval.
Training Pipeline: Due to the lack of large annotated experimental XFEL datasets, the authors developed a physics-based simulation pipeline. They used molecular dynamics simulations (ESPResSo/PyOIF) to generate coarse-grained red blood cell (RBC) geometries, voxelized them, and assigned X-ray optical constants at 20 keV. Synthetic phase-contrast holograms were generated using GPU-accelerated raycasting and Fresnel propagation.
Optimization Strategy: Training utilized a composite loss function combining pixel-wise L1 error, Structural Similarity (SSIM), a Fourier Ring Correlation (FRC) term, and Total Variation regularization. A multi-view gradient accumulation strategy was employed, evaluating loss from multiple simultaneous views per iteration to enforce cross-angular geometric consistency and prune inconsistent Gaussians.

Key Results
The framework was tested on simulated RBC holograms using 12 training projections and 4 withheld test views.

Performance: The best configuration (8,000 initial Gaussians, learning rate $2 \times 10^{-3}$ , 4 simultaneous views) achieved an SSIM of 0.870 and an FRC Metric (FRCM) of 0.096. Reconstruction was completed in approximately 15 minutes on a single NVIDIA A100 GPU.
Metric Analysis: The study highlighted that PSNR, a standard benchmark for 3DGS, is a misleading indicator for holographic data. A configuration with 244,000 Gaussians achieved the highest PSNR (19.7 dB) but the lowest SSIM (0.449). This configuration converged to a degenerate "absorption-only" solution that matched background intensities but failed to reproduce the Fresnel fringe structures encoding the phase signal. Conversely, SSIM and FRCM correctly identified configurations that preserved high-frequency phase information.
Optimization Dynamics: Counterintuitively, increasing the initial Gaussian count from 8k to 244k degraded structural quality under default learning rates. Targeted learning-rate tuning on a lean 8k initialization outperformed a 30-fold increase in primitive count, suggesting the bottleneck lies in optimization dynamics rather than representational capacity.
Material Recovery: The learned material fields were qualitatively correct, with phase-projection maps ( $\delta$ ) dominating and absorption maps ( $\beta$ ) remaining near zero, consistent with soft tissue physics. However, quantitative per-Gaussian $\delta/\beta$ ratios were lower than theoretical hemoglobin values, reflecting the inherent ill-posedness of phase retrieval from intensity-only measurements.

Significance and Claims
The paper establishes 3D Gaussian Splatting as an effective, compact, and differentiable representation for sparse-view X-ray phase-contrast reconstruction. The authors claim that the particle-based scene description inherently addresses the data-volume constraints of MHz tomoscopy. Furthermore, the differentiable physics-based forward model enables near-online reconstruction, with optimization converging within timescales compatible with beamtime constraints.

The work demonstrates that for holographic or interferometric data, standard metrics like PSNR can be deceptive, and structural metrics (SSIM, FRCM) are essential for evaluating the recovery of phase information. The authors identify the immediate next priorities as extending the method to multiple molecular dynamics time-steps for 4D reconstruction and validating the approach against real XFEL projections from a four-simultaneous-view experimental configuration.