Representation Paradigms in AI-based 3D Radiological Image Reconstruction: A Systematic Review

1. Problem Statement

Radiological imaging (CT, MRI, PET, SPECT, Ultrasound) is critical for clinical diagnosis, but traditional reconstruction methods face significant challenges:

• Data Scarcity & Noise: Clinical constraints often lead to low-dose, sparse-view, or undersampled acquisitions, resulting in noisy or incomplete data.
• Limitations of Physics-Based Methods: Traditional analytical (e.g., Filtered Back Projection) and iterative methods (e.g., OSEM) are sensitive to noise, modeling mismatches, and missing data.
• AI Limitations: While deep learning (CNNs, Transformers, GANs, Diffusion models) has improved reconstruction, existing reviews often focus on specific network architectures or imaging modalities. There is a lack of a unified framework that categorizes methods based on how the 3D target is parameterized (the representation paradigm).
• Clinical Barriers: Current AI methods struggle with computational efficiency, interpretability, and ensuring anatomical fidelity in complex 3D volumes.

2. Methodology

The authors conducted a systematic literature review following PRISMA guidelines:

• Data Sources: Web of Science, Google Scholar, and Scopus.
• Selection Criteria: Papers published in the last five years containing keywords related to "AI," "Deep Learning," "3D," "Medical Imaging," and "Reconstruction."
• Filtering: Papers were selected based on research focus, citation count, and publication in top-tier venues (e.g., IEEE T-MI, MICCAI, CVPR).
• Final Corpus: 64 representative studies were analyzed.
• Taxonomy Construction: Instead of grouping by modality or network type, the authors classified methods into four representation families based on the mathematical formulation of the reconstructed target:
  1. Discrete Grid Representations
  2. Explicit Basis Expansion Representations
  3. Explicit Primitive Representations
  4. Implicit Neural Representations

3. Key Contributions

The paper makes four primary contributions:

1. Representation-Oriented Taxonomy: It proposes a novel classification system that organizes AI-based 3D reconstruction methods into four distinct families, clarifying the relationships between seemingly disparate techniques (e.g., clarifying that Radiance Fields are a subtype of Implicit Neural Representations).
2. Comprehensive Survey: It covers a wide range of modalities (CT, MRI, PET, SPECT, US) and tasks (Task I: Raw data to image; Task II: Low-dose to high-dose; Task III: Sparse-view to full-view).
3. Resource Compilation: It summarizes publicly available benchmark datasets (e.g., AAPM Low Dose CT, fastMRI) and standard evaluation metrics (PSNR, SSIM, LPIPS, etc.) specific to radiological reconstruction.
4. Critical Analysis & Future Directions: It provides a comparative analysis of the strengths/weaknesses of each paradigm and outlines future research directions focusing on physics integration, clinical evaluation, efficiency, privacy, and interpretability.

4. Technical Breakdown of Representation Paradigms

A. Discrete Grid Representations

• Concept: The image is stored on a fixed, sampled spatial grid (voxels or slices). The model predicts intensity values for each grid point.
• Sub-types:
  • Slice-wise: Processes 2D slices independently (e.g., U-Net). Fast but suffers from inter-slice inconsistencies.
  • Multi-plane/Stack: Processes adjacent slices jointly to improve continuity.
  • Volumetric: Jointly reconstructs the entire 3D volume (e.g., 3D U-Net, 3D GANs, 3D Diffusion).
• Pros: Intuitive, compatible with existing pipelines, high adaptability.
• Cons: Fixed resolution, potential "staircase" artifacts, high memory usage for high-resolution 3D volumes.

B. Explicit Basis Expansion Representations

• Concept: The image is represented as a linear combination of predefined basis functions (e.g., B-splines, Kaiser-Bessel "blobs," or kernel matrices).
• Mechanism: $\lambda(r) = \sum \alpha_m \phi_m(r)$, where $\alpha_m$ are learnable coefficients and $\phi_m$ are basis functions.
• Pros: Smoother spatial transitions than grids; compact representation; retains explicit interpretability of coefficients.
• Cons: Performance heavily depends on the choice of basis functions; computationally demanding projection/back-projection due to basis overlap.

C. Explicit Primitive Representations

• Concept: The image is approximated by a finite set of adaptive, learnable local elements (primitives) rather than a uniform grid.
• Key Types:
  • Gaussian Primitives: Inspired by 3D Gaussian Splatting (3DGS). Uses learnable Gaussians (position, covariance, density) to represent anatomy.
  • Simplex Meshes: Uses adaptive triangular/tetrahedral meshes.
• Pros: Highly efficient rendering; adaptive spatial support (high resolution where needed, low elsewhere); excellent for sparse-view and novel-view synthesis.
• Cons: Requires specialized initialization and optimization; less interpretable than grids; complex implementation.

D. Implicit Neural Representations (INR)

• Concept: The image is modeled as a continuous function $f_\theta(r)$ mapping spatial coordinates to intensity, without explicit storage of voxels.
• Sub-types:
  • Coordinate-based Intensity Fields: Maps $(x,y,z) \to$ intensity (e.g., NeRP).
  • Radiance Fields (NeRF): Models density and view-dependent color (e.g., MedNeRF, Ultra-NeRF).
• Pros: Continuous resolution (infinite zoom); excellent for handling sparse data and missing views; strong continuity priors.
• Cons: High computational cost (slow training/inference); "black box" nature (low interpretability); memory intensive for large 3D volumes.

5. Results and Comparative Analysis

The authors evaluated the four families across five criteria: Effectiveness, Efficiency, Adaptability, Simplicity, and Interpretability.

Family	Effectiveness	Efficiency	Adaptability	Simplicity	Interpretability
Discrete Grid	High (4)	High (4)	Very High (5)	Very High (5)	Low (3)
Basis Expansion	Moderate (3)	Moderate (3)	High (4)	Moderate (3)	Moderate (4)
Explicit Primitive	High (4)	Very High (5)	Moderate (3)	Low (2)	Low (3)
Implicit Neural	Very High (5)	Low (2)	Moderate (3)	Low (2)	Very Low (2)

• Key Finding: No single paradigm is universally optimal.
  • Grids are best for general-purpose, easy-to-deploy clinical workflows.
  • Primitives (Gaussian) offer the best trade-off for speed and sparse-view rendering.
  • Implicit (NeRF) offers the highest theoretical fidelity for continuous structures but suffers from efficiency and interpretability issues.
  • Traditional Iterative methods remain the most interpretable baseline.

6. Significance and Future Directions

This review shifts the discourse from "which network architecture is best?" to "how should the 3D data be represented?" This distinction is crucial for advancing medical AI.

Identified Challenges & Future Directions:

1. Physics Integration: Future representations must tightly integrate with imaging physics (forward models, attenuation laws) rather than treating reconstruction as a pure black-box image-to-image task.
2. Clinical Evaluation: Moving beyond PSNR/SSIM to include reader studies, anatomical fidelity, and downstream diagnostic utility (e.g., lesion detection accuracy).
3. Efficiency & Deployment: Developing lightweight architectures, knowledge distillation, and edge-computing strategies to make high-fidelity 3D reconstruction feasible in real-time clinical settings.
4. Privacy: Utilizing Federated Learning, differential privacy, and unsupervised methods to train models without compromising patient data privacy.
5. Interpretability: Designing models with traceable mechanisms (e.g., unfolding optimization steps) to build clinician trust and ensure safety.

Conclusion:
The field is evolving toward hybrid representations that combine the efficiency of explicit methods with the continuity of implicit methods, all grounded in physical imaging models. This systematic review provides the necessary taxonomy and roadmap to guide this evolution toward clinically practical, high-quality 3D radiological reconstruction.