🤖 AI

Learning Where and What to Lift for Bi-planar X-ray-to-CT Reconstruction

The paper proposes LiftXR, a geometry-guided framework that interleaves 3D anatomical layout recovery and CT intensity reconstruction from bi-planar X-rays to achieve state-of-the-art reconstruction quality and improved downstream segmentation performance.

Original authors: Yifei Wu, Yicheng Wu, Qiang Ma, Qi Chen, Renyang Gu, Xinyu Liu, Yongsheng Pan, Yong Xia

Published 2026-08-19

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yifei Wu, Yicheng Wu, Qiang Ma, Qi Chen, Renyang Gu, Xinyu Liu, Yongsheng Pan, Yong Xia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Medical imaging has long relied on a fundamental trade-off: to see the inside of the human body in three dimensions, doctors usually need to take hundreds of X-ray pictures from different angles. This process, known as computed tomography, or CT, creates a detailed map of internal tissues, but it also exposes the patient to a significant amount of radiation. For routine check-ups or for tracking a disease over time, this radiation dose can be a concern. Scientists have tried to solve this by using fewer X-rays, but when you try to build a 3D picture from just one or two flat images, the task becomes incredibly difficult. The flat images collapse the depth of the body, turning a complex 3D structure into a single, confusing shadow where it is impossible to tell how far back an organ lies or how thick it is.

The challenge is not just about filling in missing data; it is about understanding the rules of anatomy. In a flat X-ray, the heart, the lungs, and the spine all overlap, and their shadows merge. Without knowing where these organs are supposed to be in 3D space, a computer trying to reconstruct the image has too many guesses to make. It might place a piece of the liver where the heart should be, or blur the boundaries between tissues, because the math alone cannot decide which arrangement is correct. This is the core problem researchers have been trying to solve: how to turn two flat, ambiguous shadows into a clear, accurate 3D model without needing the hundreds of views that a standard CT scan requires.

A team of researchers has developed a new approach called LiftXR that tackles this problem by teaching the computer to first figure out the layout of the body before trying to fill in the details. Instead of guessing the final image all at once, the system works in two distinct stages that help each other. First, it looks at the two flat X-rays and predicts a rough 3D map of where the major organs are located. It asks, "Where is the heart? Where are the lungs?" This step creates a skeletal framework, a spatial guide that tells the computer the general shape and position of the body's parts. Once this layout is established, the system uses it to reconstruct the actual CT volume, filling in the specific textures and densities of the tissues.

What makes this method unique is that it does not stop there. The researchers realized that the initial 3D map, while helpful, is not perfect. So, they added a second step where the system looks at the newly created 3D image and asks, "Does this look right?" It analyzes the boundaries and the shapes it just built to refine its understanding of the anatomy. If the computer sees that the edge of a lung looks blurry or that a rib is in the wrong place, it uses this new information to correct the original layout. This refined layout is then fed back into the system to adjust the intensity of the tissues, ensuring that the final image has the correct brightness and contrast for each specific organ. It is a cycle of generation and perception, where the computer builds a guess, checks its own work, and then improves the guess based on what it learned.

The researchers tested this system on two large public datasets containing thousands of chest CT scans. They compared LiftXR against several existing methods that try to reconstruct CT images from just two X-rays. The results showed that LiftXR consistently produced clearer and more accurate images. In terms of standard measurements for image quality, it achieved higher scores than any previous method. More importantly, the images it produced were better at showing the actual shapes and boundaries of organs. When the researchers used the reconstructed images to automatically identify and segment different body parts, the system was significantly more accurate than those using older techniques. This suggests that the new method does not just make the picture look sharper; it actually understands the anatomy better, creating a model that is truer to the human body.

One of the most compelling aspects of this work is how it handles the uncertainty inherent in the task. The researchers ran a special test where they gave the computer the perfect, correct layout of the organs and asked it to build the image. The result was nearly perfect, proving that if the computer knows where the organs are, it can reconstruct the image with high fidelity. This confirmed their hypothesis: the main barrier to a good reconstruction is not the lack of data, but the lack of a clear spatial guide. By explicitly teaching the system to recover the layout first, they removed much of the guesswork that usually leads to errors.

The system also proved to be robust when the X-rays were not taken at the perfect, ideal angles. In real-world medical settings, patients might move slightly, or the X-ray machine might not be perfectly aligned, causing the two images to be slightly off. When the researchers tested LiftXR with these small deviations, it maintained its high performance better than other methods. This resilience suggests that the system's ability to reason about the 3D layout allows it to compensate for minor geometric inconsistencies, making it a more practical tool for real clinical use.

While the results are promising, the researchers acknowledge that the technology is not yet ready for every medical scenario. The process of refining the image through these multiple steps requires significant computing power, which could be a hurdle for use in mobile or low-resource settings. Furthermore, the current tests focused on healthy anatomy and standard organ shapes; the ability to detect small, unusual, or highly variable pathological structures remains to be fully explored. However, the study demonstrates a clear path forward. By shifting the focus from simply filling in pixels to understanding the spatial organization of the body, LiftXR offers a new way to see inside the human body with less radiation and greater clarity. It shows that when a computer is taught to understand the "where" before the "what," it can solve problems that were previously thought to be too ambiguous to crack.

Technical Summary: LiftXR for Bi-planar X-ray-to-CT Reconstruction

Problem Statement

Reconstructing a 3D Computed Tomography (CT) volume from only two bi-planar X-ray projections (frontal and lateral) is a severely ill-posed inverse problem. The fundamental challenge arises because X-ray imaging models the projection of a volumetric attenuation field, where each measurement records the accumulated attenuation along a ray path. This process collapses depth information, causing the 3D spatial locations of anatomical regions and their corresponding intensity distributions to become highly entangled and ambiguous.

Existing methods often rely on implicit geometric priors or optimize voxel-level appearances without explicitly modeling anatomical organization. Consequently, while these methods may achieve favorable voxel-wise metrics, they frequently produce volumes with inaccurate regional extents, blurred boundaries, and inconsistent spatial relationships between organs. The authors observe that once the spatial organization (layout) of anatomical regions is established, estimating their CT intensities becomes significantly more tractable.

Methodology: LiftXR

The paper proposes LiftXR, an end-to-end, interleaved, geometry-guided framework that explicitly incorporates anatomical layout recovery into the CT reconstruction process. The framework operates through a generation-perception feedback loop involving four main stages:

Bi-planar X-ray Initialization:
The two input projections (PA and Lateral) are up-sampled and replicated along their unobserved spatial dimensions to form an explicit 3D X-ray volume ( $V_{xray}$ ). This is achieved by averaging the lifted representations of both views, preserving ray-integrated observations in a unified volumetric coordinate system.
Layout Lifting and Intensity Rendering (Generation):
- Layout Lifter ( $\theta_{ll}$ ): A neural network predicts an initial 3D anatomical layout ( $L_{lifted}$ ) solely from the X-ray volume. This layout encodes organ occupancy, spatial extent, and relative arrangement, serving as a generative geometric prior.
- Intensity Renderer ( $\theta_{ir}$ ): Conditioned on both the X-ray volume and the predicted layout, this module reconstructs a coarse CT volume ( $V_{rendered}$ ). The layout provides explicit spatial constraints, guiding the estimation of voxel-wise intensities.
Layout Parsing and Intensity Calibration (Perception):
- Layout Parser ( $\theta_{lp}$ ): This module performs a reconstruction-conditioned anatomical perception task. It analyzes the initially reconstructed CT volume ( $V_{rendered}$ ) alongside the X-ray evidence to predict a refined anatomical layout ( $L_{parsed}$ ). This step exploits spatially resolved boundary and intensity cues to correct regional identities and boundaries.
- Intensity Calibrator ( $\theta_{ic}$ ): Using the refined layout as a guide, this module corrects region-specific intensity inconsistencies in the CT volume, producing the final calibrated output ( $V_{calibrated}$ ).
Interleaved Supervision:
The framework is trained with a joint objective that balances anatomical layout supervision and CT reconstruction supervision.
- Layout Supervision: Uses Cross-Entropy and Dice losses against ground-truth anatomical masks (extracted from 7 major thoracoabdominal regions) to constrain 3D geometry.
- CT Supervision: Employs a multi-scale feature matching loss, L1 loss, and adversarial loss to ensure voxel-wise intensity accuracy and realistic volumetric appearance.

Key Contributions

The paper outlines three primary contributions:

Explicit Layout Modeling: The authors formulate the bi-planar X-ray-to-CT task by explicitly modeling anatomical layout as a spatial constraint. This approach decouples the problem into "where" (layout) and "what" (intensity), using the layout to guide intensity estimation.
LiftXR Framework: The proposal of a unified generation-perception feedback process that couples volumetric generation with anatomical perception. This includes projection-conditioned layout generation, conditional CT rendering, reconstruction-conditioned anatomical parsing, and region-specific intensity calibration.
State-of-the-Art Performance: Extensive experiments demonstrate that LiftXR outperforms recent methods in both reconstruction quality and downstream segmentation tasks, establishing a new state of the art.

Experimental Results

The method was evaluated on two public chest CT datasets: CT-RATE and LIDC-IDRI.

Quantitative Performance: LiftXR achieved the best results across all metrics (PSNR, SSIM, Dice score, and HD95) on both datasets. Notably, the improvements in anatomical metrics (Dice, HD95) were more pronounced than in PSNR, suggesting that the method effectively resolves geometric ambiguities and boundary inaccuracies that voxel-wise metrics might overlook.
Downstream Segmentation: The reconstructed CT volumes yielded superior performance in external downstream segmentation tasks, indicating improved anatomical fidelity and structural consistency.
Qualitative Analysis: Visual comparisons showed that LiftXR produces clearer lung contours, more accurate vertebral and rib structures, and better inter-organ relationships compared to methods like X2CT, PerX2CT, and DiffuX2CT.
Ablation Studies: Removing either the anatomical layout component or the interleaved training strategy resulted in performance degradation, confirming that both the explicit geometric constraints and the reciprocal refinement process are essential.
Robustness and Generalization: LiftXR demonstrated robustness to small projection angle deviations (e.g., 88° or 92° instead of 90°) and showed the ability to generalize to real X-ray images (MIMIC dataset) after style transfer, maintaining plausible anatomical structures where other methods produced boundary artifacts.

Significance and Claims

The paper claims that LiftXR establishes a new state of the art by addressing the core limitation of previous approaches: the implicit handling of anatomical structure. By transitioning from projection-conditioned layout generation to reconstruction-conditioned anatomical perception, the framework allows the reconstructed volume to provide feedback for geometry refinement, which in turn guides region-specific intensity calibration.

The authors emphasize that this interleaved interaction reduces the geometric uncertainty inherent in bi-planar projections, constraining the solution space to anatomically plausible configurations. While the method shows promising results, the paper modestly notes limitations, including the need for further exploration of clinical effectiveness, the computational cost of the interleaved process, and the inherent difficulty of recovering highly variable or small pathological structures from limited information.