3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network for Real-Time Operator Guidance

Technical Summary: 3D Ultrasound-Derived Pseudo-CT Synthesis Using a Transformer-Augmented Residual Network

Problem Statement
Computed Tomography (CT) is a cornerstone of clinical diagnosis and image-guided interventions but exposes patients to ionizing radiation, a risk associated with potential radiation-induced cancers. While Ultrasound (US) offers a safe, non-ionizing, and cost-effective alternative, it suffers from significant limitations: it is highly operator-dependent, lacks quantitative tissue characterization (intensity values do not correspond to Hounsfield Units), and is susceptible to artifacts like acoustic shadowing and varying probe pressure. These factors often lead to diagnostic uncertainty and unnecessary follow-up CT examinations. Existing deep learning approaches for cross-modality synthesis face a trade-off: standard 3D Residual U-Nets (ResUNets) capture local features well but struggle with global anatomical relationships in large 3D volumes, while architectures like ResViT that integrate transformers throughout the network often incur high computational costs and may degrade fine-grained local details, particularly in noisy US data.

Methodology
The paper proposes a framework to generate 3D ultrasound-derived pseudo-CT (UD-pCT) volumes. The primary objective is not to replace diagnostic CT but to provide real-time, CT-like anatomical references to guide US operators, thereby improving scan consistency and reducing unnecessary CT referrals.

1. Data Preprocessing Pipeline:

   • Registration: The authors utilize the TRUSTED dataset, containing paired 3D kidney US and CT volumes. A landmark-based multimodal registration pipeline is employed to spatially align the modalities. This involves estimating a similarity transformation (rotation, scaling, translation) using anatomical landmarks via the Umeyama algorithm.
   • Cropping and Masking: To ensure voxel-wise correspondence and reduce memory usage, Region-of-Interest (ROI) cropping is performed based on US kidney segmentation masks. Furthermore, a Field-of-View (FOV) masking step is applied to the CT volumes to match the pyramidal (frustum-shaped) geometry of the US acquisition, masking out regions outside the US acoustic sector.

2. Proposed Architecture (BT-ResUNet3D):
   The core of the framework is a Bottleneck Transformer Residual U-Net3D (BT-ResUNet3D), designed as a supervised paired cross-modality image translation model.

   • Generator: The generator follows a 3D residual encoder–decoder structure (U-Net backbone). It employs deep residual blocks to capture local textures and fine-grained anatomical details. Crucially, a lightweight Transformer module is introduced exclusively at the bottleneck. This design allows the network to model long-range volumetric dependencies and global anatomical coherence without disrupting the early-stage convolutional feature extraction required for local detail preservation. The global context from the transformer is fused with local features via residual addition.
   • Discriminator: A 3D Conditional PatchGAN discriminator is used to enforce local structural realism. It evaluates the realism of local volumetric patches rather than a single global score, encouraging the synthesis of plausible textures and sharp structural boundaries.
   • Loss Function: The model is trained using a composite loss function combining a pixel-wise L1 reconstruction loss (to ensure voxel-level accuracy) and an adversarial loss (to enhance perceptual realism).

Key Contributions

• Robust Preprocessing Pipeline: The authors establish a pipeline involving landmark-based registration, ROI cropping, and FOV masking to construct anatomically consistent, spatially aligned training pairs, which is critical for reliable supervised cross-modality synthesis.
• Novel Architecture (BT-ResUNet3D): The introduction of a transformer-augmented bottleneck within a 3D residual encoder–decoder. This design explicitly captures long-range volumetric dependencies while preserving local anatomical fidelity, addressing the limitations of both pure CNN and full-transformer architectures.
• Conditional PatchGAN Integration: The use of a 3D conditional PatchGAN to enforce local structural realism and high-frequency anatomical details in the synthesized volumes.
• Clinical Utility Focus: The work demonstrates the feasibility of using UD-pCT volumes specifically for operator guidance during US acquisition to improve scan consistency and reduce unnecessary CT usage, rather than as a direct diagnostic substitute.

Experimental Results
The method was evaluated on the TRUSTED dataset (59 paired volumes, split into 47 training and 12 testing cases).

• Quantitative Performance: The proposed method achieved a PSNR of 23.26 dB and an SSIM of 0.71, outperforming established baselines including 3D ResUNet, 3D UNet, Pix2Pix, and diffusion-based models (MedDiffusion, 3D Conditional Diffusion). The ablation study confirmed that the transformer bottleneck significantly improved both metrics compared to a standard 3D ResUNet.
• Qualitative Analysis: Visual results show that despite the speckle noise and low contrast in input US images, the model successfully recovers coherent organ-level structures (kidneys) that align closely with real CT anatomy. The synthesized volumes preserve organ boundaries and spatial extent.

Significance and Limitations
The paper claims that the proposed framework offers a practical, safer alternative for improving guidance and efficiency in medical imaging. By providing real-time anatomical context, UD-pCT volumes can help operators optimize probe positioning and orientation, potentially reducing inter-operator variability and the volume of unnecessary CT scans.

The authors explicitly acknowledge limitations, noting that the relatively small size of the paired dataset (59 volumes) may limit the generalizability of the model. They state that future work will focus on acquiring larger, more diverse datasets and investigating semi-supervised or weakly supervised strategies to reduce dependence on paired CT data. The study maintains a modest scope, positioning the technology as a supportive tool for operator guidance rather than a replacement for definitive diagnostic CT imaging.