Tensor-Based Reduced-Order Modeling for Optimization-Based Inverse Problems

Technical Summary: Tensor-Based Reduced-Order Modeling for Optimization-Based Inverse Problems

Problem Statement
Inverse problems governed by parameter-dependent dynamical systems (PDEs or ODEs) require repeated evaluations of a high-fidelity forward model to infer unknown parameters from indirect, noisy measurements. When the parameter dimension is moderate to high, or when the problem must be solved repeatedly for different data realizations (as in digital twin applications), the computational cost of full-order model (FOM) evaluations and sensitivity analysis becomes prohibitive. Classical reduced-order modeling (ROM) techniques often focus on accelerating forward solves but struggle to address the multidimensional nature of the parameter-to-observation map required for global optimization, particularly when local surrogates lack the robustness needed for multi-start optimization in nonconvex landscapes.

Methodology
The authors propose a Tensor Reduced-Order Modeling (TROM) framework that approximates the parameter-to-observation map directly in the Tensor-Train (TT) format. Unlike traditional ROMs that reduce the state space, this approach compresses the joint dependence of observations on parameters.

1. Offline Construction:

   • The parameter domain is discretized into a grid, and the resulting observations form a high-dimensional tensor $T$.
   • TT-SVD: The full tensor is assembled and compressed via truncated SVD of its matricizations. This provides a controlled approximation error but requires access to all tensor entries, which becomes infeasible for high dimensions.
   • TT-Cross: To avoid full tensor assembly, the authors employ TT-Cross interpolation combined with AMEn (Alternating Minimal Energy) enrichment. This method adaptively samples a small subset of tensor entries to construct the compressed representation, making it viable for higher parameter dimensions ($D \ge 6$).
   • Preprocessing: For large observation spaces, a preliminary dimension reduction (via SVD or nonlinear encoders) is applied to the observation vector before TT compression to further reduce offline costs.

2. Online Inversion and Integration:
   The TROM is integrated into a regularized nonlinear least-squares formulation ($\min_\alpha \frac{1}{2}\|f(\alpha) - y_{obs}\|^2 + \frac{\lambda}{2}\|\alpha - \alpha_{ref}\|^2$) in four specific ways:

   • C1 (Direct Approximation): The surrogate $\hat{f}$ replaces the FOM in the objective function.
   • C2 (Reduced-Coordinate Gauss-Newton): By orthogonalizing the first TT core, the least-squares term is reformulated in a reduced coordinate space of dimension $r_2$ (the second TT rank). This allows the assembly of Gauss-Newton (GN) quantities ($J^T J$ and $J^T \eta$) directly from the low-rank tensor structure without forming the full observation-space Jacobian.
   • C3 (Tensor-Based Optimization): The objective function is minimized directly over the discrete parameter grid using the TT representation. This can serve as a stand-alone approximate solver or, more effectively, as a data-informed initialization for a subsequent GN solve.
   • C4 (Robustness Analysis): The method accounts for both measurement noise and TROM approximation error when selecting the regularization parameter via the discrepancy principle.

Numerical Experiments
The framework is evaluated on two distinct inverse problems:

1. Inverse Heat Transfer: A parabolic PDE problem in a heterogeneous medium involving the recovery of locations and radii of low-conductivity inclusions. Parameter dimensions tested are $D=3, 6, 9$.
2. FitzHugh–Nagumo Parameter Estimation: A nonlinear ODE system modeling excitable dynamics. The objective landscape is highly nonconvex with multiple local minima, making it sensitive to initialization.

Key Results

• Accuracy and Efficiency: TROM surrogates (both TT-SVD and TT-Cross) reproduce the behavior of full-order inversion with substantially reduced online costs. For the heat-transfer problem, online inversion times are on the order of milliseconds, representing speedups of $10^3$ to $10^4$ compared to FOM-based Gauss-Newton methods.
• Scalability: TT-Cross successfully extends the approach to $D=9$, a regime where full tensor assembly for TT-SVD is computationally infeasible.
• Robustness:
  • In the heat-transfer problem, the method remains stable under measurement noise and varying spatial discretizations.
  • In the FitzHugh–Nagumo problem, the tensor-based optimization step (C3) significantly outperforms random initialization. It identifies a basin of attraction close to the global minimum, whereas standard GN methods with random starts frequently converge to false local minima.
• Deep Integration Benefits: Using the reduced-coordinate formulation (C2) and tensor-based initialization (C3) provides an additional speedup factor of 4 to 8 over "plain" TROM inversion (where the surrogate is used only for forward evaluation and finite differences are used for Jacobians).

Significance and Claims
The paper claims that the primary contribution lies in moving beyond the use of TROMs merely as fast forward surrogates. Instead, the authors demonstrate that the low-rank tensor structure can be exploited throughout the inverse solve to:

1. Reformulate the optimization problem in reduced coordinates, decoupling the online cost from the full observation dimension.
2. Assemble sensitivity information (Jacobian products) efficiently without explicit Jacobian formation.
3. Provide a robust, data-informed initialization strategy that mitigates the sensitivity of nonlinear solvers to poor starting points in nonconvex landscapes.

The authors conclude that while the offline construction of the TROM surrogate incurs a significant computational cost (especially for high dimensions), the resulting online inverse solves are inexpensive, stable, and robust. This makes the approach particularly suitable for applications requiring repeated inversions, such as digital twins, provided the offline cost can be amortized. The method effectively balances approximation error, measurement noise, and regularization to produce stable reconstructions.