🔢 mathematics

Nonlinear RMM-GKS for Large-Scale Dynamic and Streaming Inverse Problems with Uncertain Forward Operators

This paper proposes a nonlinear recycled majorization-minimization generalized Krylov subspace (NL-RMM-GKS) framework that enables high-quality, memory-efficient reconstruction for large-scale dynamic and streaming inverse problems with uncertain forward operators by jointly estimating images and model parameters through alternating or variable projection strategies.

Original authors: Toluwani Okunola, Mirjeta Pasha, Misha E. Kilmer, James G. Nagy, Eric de Sturler

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Toluwani Okunola, Mirjeta Pasha, Misha E. Kilmer, James G. Nagy, Eric de Sturler

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a clear photograph of a moving object, but your camera is broken in two specific ways:

The Lens is Blurry: You don't know exactly how the light is bending (the "forward operator" is uncertain).
The Object is Moving: The thing you are photographing is changing shape or position over time.

Usually, when scientists try to fix a blurry photo, they assume they know exactly how the camera lens works. But in real life—like in medical CT scans or photoacoustic imaging—the "lens" might be slightly off because the machine isn't perfectly calibrated, or the sensors are in the wrong spot. If you ignore this, your final image will have weird ghostly artifacts, like a double-exposed photo.

This paper introduces a new, smart software tool called NL-RMM-GKS to fix these messy images. Here is how it works, explained through everyday analogies.

1. The Core Problem: The "Blind Sculptor"

Think of the imaging process as a sculptor trying to carve a statue (the image) based on a set of instructions (the measurements).

The Old Way: The sculptor assumes the instructions are perfect. If the instructions are slightly wrong (because the camera angle is off), the sculptor carves a statue that looks nothing like the real thing.
The New Way (NL-RMM-GKS): The sculptor realizes, "Wait, my instructions might be wrong." So, they do two things at the same time:
1. They carve the statue (reconstruct the image).
2. They tweak the instructions (estimate the camera angles/sensor positions).
  They keep doing this back-and-forth until the statue looks right and the instructions make sense.

2. The Memory Trick: The "Recycling Bin"

Usually, solving these problems requires a computer to remember every single step it took to get the answer. If the problem is huge (like a high-definition medical scan), the computer runs out of memory, like a backpack that gets too heavy to carry.

The authors use a clever trick called Recycling.

The Analogy: Imagine you are trying to find a hidden treasure. You take many steps, but you can't carry a map of every single step you've ever taken.
The Solution: Instead of carrying the whole map, you keep a "highlight reel" of the most important directions you've tried. If you try a new direction that isn't very useful, you throw it away and replace it with a new one. If you find a direction that is really helpful, you keep it.
The Result: The computer never runs out of memory, no matter how long the process takes. It keeps the "best directions" and discards the rest, allowing it to solve massive problems that would normally crash a computer.

3. The Streaming Feature: The "Puzzle Assembly Line"

Sometimes, the data is so huge it doesn't even fit on the computer's hard drive all at once. It arrives in chunks, like a puzzle delivered one box at a time.

The Old Way: You have to wait for the whole puzzle to arrive before you can start.
The New Way (Streaming): You start working on the first box immediately. When the second box arrives, you don't throw away your work on the first box. Instead, you carry over the "shape" of the pieces you've already figured out and use that to help you solve the next box.
The Benefit: You can solve problems that are larger than your computer's memory, processing data as it comes in, like an assembly line.

4. The Dynamic Feature: The "Moving Picture"

If the object you are imaging is moving (like a beating heart), the problem gets harder. The image changes from frame to frame.

The paper offers two ways to handle this motion:

The "Rigid" Approach (Anisotropic TV): This assumes the object just shifts slightly from one frame to the next, like a person standing still and blinking. It's simple but can get blurry if the object moves fast.
The "Flow" Approach (Optical Flow): This is like a weather forecaster tracking wind. It calculates exactly how every pixel is moving from one frame to the next. The paper shows that for objects moving smoothly (like the digits in the Moving MNIST test), this "flow" method creates much sharper, clearer videos than the rigid approach.

5. Two Ways to Solve the Puzzle

The authors built two different engines for their software to solve the "image + camera angle" problem:

The Alternating Method (AltMin): "I'll fix the image, then you fix the camera angle, then I'll fix the image again." It's a steady, reliable back-and-forth. The paper notes this is very robust, even if you start with a bad guess.
The Variable Projection Method (VarPro): "I'll pretend the image is already solved and just focus on fixing the camera angle." This is faster if you start with a good guess, but it can get confused if your starting guess is terrible.

Summary of Results

The authors tested this on simulated medical scans (CT and Photoacoustic Tomography).

Memory: They proved their method uses a fixed amount of memory, unlike older methods that grow until they crash.
Quality: By fixing the camera angles while fixing the image, they removed the "ghosting" artifacts that usually ruin these scans.
Speed vs. Quality: They found a sweet spot. If you break the data into too many small chunks (streaming), it's super fast but the image gets a bit blurry. If you use fewer chunks, it's slower but sharper.
Motion: For moving objects, the "Optical Flow" method was the clear winner, keeping the edges of moving objects sharp, while the simpler method made them look like a blur.

In short, this paper gives scientists a new, memory-efficient toolkit to take clear pictures of moving objects, even when they don't know exactly how their camera is calibrated. It's like giving a sculptor a self-correcting chisel that never runs out of space in their toolbelt.

Technical Summary: Nonlinear RMM-GKS for Large-Scale Dynamic and Streaming Inverse Problems

Problem Statement
Many practical imaging systems, including computed tomography (CT) and photoacoustic tomography (PAT), suffer from uncertainties in acquisition geometry, such as imperfectly calibrated projection angles or sensor positions. These uncertainties render the forward operator $H(p)$ dependent on unknown parameters $p$ , transforming the reconstruction task into a nonlinear inverse problem. The goal is the joint estimation of the image $u$ and the geometric parameters $p$ by minimizing a regularized least-squares objective:
$\min_{u, p} \frac{1}{2}\|H(p)u - b\|_2^2 + \lambda\|\Psi u\|_1$
where $\Psi$ is a regularization operator (typically promoting edge preservation via total variation). Standard linear approaches fail here, and existing nonlinear methods often struggle with scalability, memory constraints, or sensitivity to initialization. Furthermore, dynamic imaging scenarios introduce temporal changes, requiring joint estimation of time-varying images and potentially time-invariant geometric parameters.

Methodology
The authors propose the Nonlinear Recycled Majorization–Minimization Generalized Krylov Subspace (NL-RMM-GKS) framework. This method extends the linear RMM-GKS approach to handle the nonlinearity introduced by $p$ while maintaining bounded memory usage for large-scale problems.

The framework is built on two complementary formulations for solving the joint optimization problem:

Alternating Minimization (AltMin): This approach decouples the problem, alternating between:
- Image Update: Solving for $u$ with fixed $p$ using the RMM-GKS algorithm. This involves constructing a quadratic majorant for the nonsmooth $\ell_1$ penalty and projecting onto a recycled Krylov subspace.
- Parameter Update: Solving for $p$ with fixed $u$ using damped Gauss-Newton iterations with backtracking line search.
Variable Projection (VarPro): This approach analytically eliminates the image variable $u$ by solving the majorized subproblem implicitly, reducing the joint problem to an optimization over $p$ alone. The reduced objective is then optimized using Gauss-Newton steps.

Key Technical Components:

Recycling Strategy: To address the memory bottleneck of standard Krylov methods where subspace dimension grows unboundedly, the authors employ an "enlarge-compress" cycle. The Krylov subspace is expanded up to a maximum dimension ( $k_{max}$ ) and then compressed back to a minimum dimension ( $k_{min}$ ) using truncated SVD, retaining only the most informative search directions. This ensures bounded memory usage regardless of the number of iterations.
High-Regularization Stabilization: To prevent the parameter update from being attracted to poor local minima due to insufficient regularization in the image estimate, an optional secondary solve with a large regularization parameter is performed. This provides a smoother image estimate for the parameter update step while retaining the primary solution for reconstruction.
Streaming Variants (s-NL-RMM-GKS): For datasets too large to fit in memory or arriving sequentially, the method processes data in blocks. The compressed Krylov basis, image estimate, and parameter estimate are carried over between blocks, allowing reconstruction without forming the full forward operator.
Temporal Regularization: For dynamic problems, the framework incorporates two plug-in strategies:
- Anisotropic Total Variation (ANISO-TV): Penalizes spatial gradients and frame-to-frame intensity differences.
- Optical Flow (OF): Enforces a physics-based motion model by estimating velocity fields between frames and incorporating them into the regularization operator.

Key Contributions

Unified Nonlinear Framework: The paper presents NL-RMM-GKS, a comprehensive framework that generalizes previous linear methods (MM-GKS, RMM-GKS, s-RMM-GKS, and MMGKS-OF) to the nonlinear setting with uncertain forward operators.
Dual Realizations: The development of both AltMin and VarPro realizations, offering a trade-off between robustness to initialization (AltMin) and convergence speed with good initialization (VarPro).
Bounded Memory for Nonlinear Problems: The adaptation of the recycling strategy to the nonlinear setting, ensuring that memory requirements remain bounded even as the forward operator changes with parameter updates.
Streaming and Dynamic Extensions: The introduction of streaming variants for sequential data processing and the integration of temporal regularization strategies for dynamic imaging.
Convergence Analysis: A theoretical proof demonstrating that the sequence generated by the algorithm converges to a stationary point of the joint objective under standard regularity assumptions.

Numerical Results
The authors validate the framework through extensive numerical experiments in fan-beam CT and PAT:

Static Reconstruction: In CT experiments with uncertain projection angles, NL-RMM-GKS with recycling achieved significantly lower reconstruction errors (RRE) compared to standard MM-GKS without recycling. Streaming variants (s-NL-RMM-GKS) demonstrated a controllable trade-off between memory/runtime and reconstruction quality based on the number of data blocks.
Initialization Sensitivity: In PAT experiments with sensor radius uncertainty, VarPro converged faster with good initial guesses but exhibited oscillatory behavior and instability with poor initialization. Conversely, AltMin showed greater robustness to poor initialization, maintaining stable convergence.
Dynamic Imaging: In dynamic CT (Moving MNIST) and PAT (Blocks Phantom) experiments, Optical Flow regularization consistently outperformed ANISO-TV, particularly when inter-frame motion was significant or when using streaming with many blocks. The framework successfully recovered both the image sequence and the geometric parameters, outperforming baselines that used only coarse grid-search initialization.
Memory Efficiency: The recycling strategy successfully prevented unbounded memory growth, with peak memory usage remaining constant regardless of the total number of iterations or dataset size.

Significance and Claims
The paper claims that NL-RMM-GKS provides a robust, scalable solution for large-scale nonlinear inverse problems with uncertain forward operators. Its significance lies in:

Scalability: By bounding memory usage through recycling and enabling streaming processing, the method addresses problems that are intractable for standard Krylov subspace methods.
Joint Estimation: It effectively couples image reconstruction with parameter estimation, mitigating the artifacts caused by geometric uncertainties that plague linear approaches.
Flexibility: The framework is modular, accommodating different temporal regularization strategies and offering a choice between AltMin and VarPro based on the specific problem characteristics and initialization quality.
Theoretical Guarantee: Unlike many heuristic hybrid approaches, the method is supported by a convergence guarantee to a stationary point.

The authors note that while the method is effective, global convergence to the global minimum cannot be guaranteed for nonconvex problems, and future work will focus on establishing convergence rates, extending to high-dimensional parameter spaces, and validating on real-world data.