⚡ electrical engineering

Model-based Dynamic 3D MRI Reconstructions using Neural Fields and Tensor Product Expansions

This paper introduces a memory-efficient, discretization-free framework for dynamic 3D MRI reconstruction that represents magnetization and coil sensitivities as continuous differentiable functions using tensor product neural fields, achieving superior performance over state-of-the-art methods in highly undersampled scenarios.

Original authors: Ray Sheombarsing, Max van Riel, David Heesterbeek, Nico van den Berg, Alessandro Sbrizzi

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ray Sheombarsing, Max van Riel, David Heesterbeek, Nico van den Berg, Alessandro Sbrizzi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Pixelated" Puzzle

Imagine you are trying to solve a massive jigsaw puzzle, but someone has taken 90% of the pieces away. In the world of MRI scans, this is what happens when doctors try to take pictures of a beating heart very quickly. They need to skip most of the data (undersampling) to catch the motion before the heart moves again.

Traditional MRI reconstruction methods try to solve this by treating the image like a grid of tiny, separate pixels.

The Flaw: When you have so few pieces, trying to guess the missing pixels based on a rigid grid is like trying to draw a smooth circle using only square Lego bricks. You end up with jagged edges, blurry motion, and "ghosting" artifacts. To fix this, computers have to use heavy "rules" (regularization) to force the picture to look smooth, but these rules often blur out the fine details or the actual movement of the heart.
The Memory Crash: Trying to calculate all these missing pixels for a 3D movie of a heart requires a computer to hold a massive amount of data in its memory at once. It's like trying to carry a library of books in one hand; eventually, the computer runs out of space and crashes.

The Solution: The "Infinite Paintbrush"

The authors of this paper propose a new way to think about the image. Instead of a grid of pixels, they treat the image as a smooth, continuous painting that can be drawn at any point, no matter how close you zoom in.

They use a tool called Neural Fields, which are essentially "smart paintbrushes" (neural networks) that learn the shape of the image.

The Analogy: Imagine you want to describe a mountain range.
- Old Way (Pixels): You take a photo and count every single square inch of grass and rock. If you miss a square, you have a hole in your map.
- New Way (Neural Fields): You describe the mountain with a mathematical formula. You can ask, "What is the height at this specific coordinate?" and the formula gives you the answer instantly, even if you never visited that exact spot. The image is smooth and continuous, not blocky.

The Secret Sauce: The "Lego Tower" Trick

There is a catch. If you try to use one giant "smart paintbrush" to describe a 3D movie (which has height, width, depth, and time), the math gets so complicated that even the most powerful supercomputers can't handle it. It's like trying to build a skyscraper out of a single, giant block of concrete; it's too heavy to lift.

The authors' breakthrough is using Tensor Product Expansions.

The Metaphor: Instead of one giant, heavy block, they build the image using four separate, lightweight Lego towers (one for time, one for width, one for depth, one for height).
How it works: They combine these four simple towers to create the complex 3D movie.
- The Benefit: This is incredibly efficient. Instead of needing to calculate billions of points at once, the computer only needs to calculate a few thousand points for each of the four towers and then snap them together.
- The Result: The paper claims this reduces the computer's memory usage by 31 times. A task that previously required a massive, expensive 48GB graphics card (and often failed) can now run on a standard 12GB card.

What They Did and Found

The researchers tested this new method on three types of MRI data:

Dynamic 2D Heart Scans: They simulated taking pictures of a heart at different speeds.
- Result: When they skipped 16 times more data than usual (aggressive undersampling), the old method (Compressed Sensing) produced blurry, flickering images. The new method produced sharp, clear images where the heart's motion was perfectly smooth.
Dynamic 3D Thigh Scans: They scanned a thigh while applying pressure.
- Result: The new method could reconstruct the video directly from raw, unorganized data without needing to group the data into "frames" first. This is something traditional methods couldn't do without running out of memory.
Dynamic 3D Heart Scans: They tested it on a real 3D heart movie.
- Result: Again, the new method preserved fine details and motion much better than the standard methods, which suffered from "streaking" and blurring.

The Bottom Line

This paper introduces a way to reconstruct MRI scans that treats the image as a smooth, continuous object rather than a grid of pixels. By breaking the problem down into smaller, manageable "towers" (tensor products) and using smart algorithms, they made it possible to create high-quality, fast-moving 3D movies of the heart on standard computer hardware.

Key Takeaways from the paper:

No more "pixelated" limits: The method works at any resolution because it's not tied to a fixed grid.
Memory Saver: It uses 31 times less memory than previous neural network methods, making 3D dynamic MRI feasible on standard equipment.
Better Motion: It handles fast-moving organs (like the heart) much better than current top-tier methods, even when the data is very incomplete.
No Training Data Needed: Unlike many AI methods, this doesn't need a library of thousands of pre-scanned patients to learn from; it figures out the image directly from the specific scan it is working on.

Technical Summary: Model-based Dynamic 3D MRI Reconstructions using Neural Fields and Tensor Product Expansions

Problem Statement
Conventional MRI reconstruction methods treat images and coil sensitivities as discrete arrays. While suitable for digital data, this approach incurs high memory demands and lacks structural awareness, necessitating strong regularization (e.g., Total Variation) to enforce consistency. These limitations become critical in highly undersampled scenarios, such as dynamic 2D and 3D cardiac magnetic resonance (CMR), where high acceleration factors are required to resolve cardiac motion. Existing deep learning approaches offer alternatives but typically rely on large supervised training sets, reducing adaptability across scanners and protocols, and remain tied to fixed resolutions. Furthermore, while Implicit Neural Fields (INFs) offer continuous representations, applying fully multivariate neural networks to high-dimensional (dynamic 3D) problems is computationally prohibitive due to the exponential growth in evaluation points required for dense grids.

Methodology
The authors propose a discretization-free, model-based framework that represents magnetization and coil sensitivities as continuous, differentiable functions using Neural Field Expansions (NFEs) based on tensor products of univariate neural fields.

Tensor Product Architecture: Instead of a single multivariate network, the method decomposes the representation into a tensor product of univariate Multi-Layer Perceptrons (MLPs) for each coordinate direction (space and time). If a grid has dimensions $P_1 \times \dots \times P_d$ , a multivariate network requires evaluating $\prod P_j$ points, whereas the tensor product approach evaluates only $\sum P_j$ points across $d$ univariate networks. This reduces computational complexity from exponential to linear with respect to dimensionality.
Continuous Representation: Both the magnetization $m(t, x)$ and coil sensitivities $S(x)$ are modeled as NFEs. The magnetization is defined over spatiotemporal domains, while coil sensitivities are defined over spatial domains. The authors utilize SIREN networks (sinusoidal activation functions) to accurately capture high-frequency signals and their derivatives.
Joint Optimization: The framework jointly reconstructs magnetization and coil sensitivities from undersampled k-space data without requiring external training datasets or separate coil-calibration scans.
Objective Function: The reconstruction minimizes a loss function comprising:
- Data Consistency: A weighted $L_2$ -norm penalizing k-space deviations, designed to handle model mismatch and heavy-tailed residuals.
- Regularization: Spatial and temporal Total Variation (TV) on magnetization, an $L_1$ -penalty on the temporal derivative, and a squared $L_2$ -penalty on coil sensitivity derivatives to promote smoothness.
Stochastic Optimization: To manage memory and enable dynamic 3D reconstruction, the objective is optimized using Stochastic Gradient Descent (SGD). The regularization terms are evaluated on a mixed set of randomly sampled continuous points and discrete Cartesian grid points corresponding to measurement locations. This stochastic formulation allows for efficient memory usage and fine-grained control over the optimization process.

Key Contributions

Tensor Product Formulation: The primary contribution is the introduction of a tensor product architecture of univariate neural fields. This structure replaces single multivariate networks, enabling memory- and flop-efficient evaluation and differentiation crucial for dynamic 3D MRI.
Joint Reconstruction: The method represents and jointly reconstructs both magnetization and coil sensitivities as neural fields, addressing the ill-posedness of joint estimation through principled regularization.
Stochastic Model-Based Optimization: The integration of stochastic sampling with continuous-domain modeling allows for reconstruction at high acceleration factors (e.g., factor 16) on standard GPU hardware, a task previously infeasible for dynamic 3D data due to memory constraints.

Results
The method was evaluated on three datasets: retrospective undersampled dynamic 2D CMR, prospective undersampled dynamic 3D thigh MRI, and prospective undersampled dynamic 3D CMR.

Memory Efficiency: The tensor product design yielded a 31× reduction in GPU memory for a dynamic 2D example (reducing usage from ~23GB to ~750MB). For dynamic 3D cases, it reduced memory requirements from >48GB (infeasible on a 48GB GPU) to 9–12GB.
Dynamic 2D Performance: At acceleration factors (AF) of 8, 12, and 16, NFE-based reconstructions maintained high visual quality and temporal consistency. In contrast, Compressed Sensing (CS) baselines exhibited pronounced flickering, streaking artifacts, and blurring, particularly at AF 12 and 16. Quantitative metrics (SSIM and PSNR) consistently favored NFEs across all slices and time frames.
Dynamic 3D Performance: The method successfully reconstructed time-resolved images directly from raw, unbinned k-space readouts (temporal resolution $\delta t = 5.4$ ms), a task computationally infeasible for standard CS pipelines. While some noise and localized boundary-intensity differences were observed (likely due to strong spatial TV regularization), the reconstructions remained anatomically consistent and temporally coherent across subjects.
Dynamic 3D CMR: In qualitative comparisons, NFE reconstructions preserved anatomical detail and remained free of the flickering artifacts seen in CS reconstructions at AF 16.

Significance and Claims
The paper claims that the proposed framework enables computationally feasible and high-quality dynamic 3D MRI reconstruction on standard hardware by leveraging the structural efficiency of tensor product expansions. The method demonstrates robustness across different acceleration factors and scanner types (1.5T and 3T) without requiring supervised training data.

The authors modestly note that while NFEs outperform state-of-the-art CS methods in preserving structure and motion under aggressive undersampling, the framework still faces challenges:

Hyperparameter Sensitivity: The method involves a significant number of hyperparameters (modes, network architecture, regularization weights). While heuristic guidelines were developed to make tuning tractable, this remains a limitation compared to fully automated approaches.
Interpretability: The analogy to generalized Fourier series is formal; the learned modes are not orthonormal or linearly independent, meaning coefficient magnitudes cannot be directly interpreted as mode importance.
Data Limitations: The evaluation on dynamic 3D CMR was limited to a single moderate-quality scan due to the lack of publicly available ground-truth datasets for unbinned dynamic 3D acquisitions.

The work establishes that continuous-domain modeling, when combined with tensor product decompositions and stochastic optimization, offers a viable path forward for high-acceleration, scan-specific MRI reconstruction.

The Problem: The "Pixelated" Puzzle

The Solution: The "Infinite Paintbrush"

The Secret Sauce: The "Lego Tower" Trick

What They Did and Found

The Bottom Line

More like this