Fast Quadratic Manifold Learning For Nonlinear Dimensionality Reduction in Large-scale Systems using Riemannian Optimization

Technical Summary: Fast Quadratic Manifold Learning for Nonlinear Dimensionality Reduction

Problem Statement
Large-scale physical systems, such as those governed by partial differential equations (PDEs), are computationally expensive to simulate, hindering real-time applications like design and optimization. Model reduction techniques, specifically Reduced-Order Models (ROMs), address this by approximating system dynamics in lower-dimensional subspaces. While Proper Orthogonal Decomposition (POD) is the standard for linear dimensionality reduction, it often fails to capture the intrinsic curvature of highly nonlinear solution manifolds (e.g., turbulent flows) without requiring prohibitively large subspace dimensions.

Quadratic Manifolds (QM) offer a solution by augmenting linear projections with structured quadratic correction terms. However, existing QM approaches face significant limitations:

1. Suboptimal Basis Selection: Standard methods rely on the leading principal components (POD modes) for the linear subspace. While optimal for linear reconstruction, these modes are often suboptimal for capturing the nonlinear interactions required for quadratic corrections.
2. Combinatorial Complexity: Greedy selection methods attempt to improve representational capacity by searching a larger pool of candidate modes. However, these methods are inherently discrete and combinatorial, suffering from factorial complexity as the candidate pool grows, which limits the feasible basis sizes in practice.
3. Scalability: Many optimization-based approaches for QM construction involve cost functions that scale with the full state-space dimension ($N$), making them computationally intractable for large-scale systems.

Methodology: FastQM
The paper proposes FastQM, a framework that reframes the identification of an optimal quadratic approximation as a continuous optimization problem on the Stiefel manifold.

• Continuous Subspace Identification: Instead of discrete selection of POD modes, FastQM treats the basis identification as finding an optimal orthonormal frame within a predefined span of candidate singular vectors. The linear subspace ($V_r$) and the quadratic correction subspace ($V_q$) are constructed as linear combinations of $m$ candidate modes (typically the leading $m$ left singular vectors of the snapshot data).
• Riemannian Optimization: The search for the optimal basis is formulated as minimizing a regularized least-squares objective function subject to orthogonality constraints. The optimization variables are rotation matrices $Q_r$ and $Q_q$ that map the candidate basis to the optimal linear and quadratic subspaces. This problem is solved using Riemannian optimization (specifically utilizing the Pymanopt library and JAX for automatic differentiation and hardware acceleration).
• Feature-Space Formulation: A critical innovation is the reformulation of the objective function to ensure computational efficiency. By precomputing the projection of snapshot data onto the candidate basis ($\tilde{S} = \tilde{V}^T S$) and utilizing trace identities, the evaluation of the cost function and its gradients scales with the number of candidate modes ($m$) and reduced dimensions ($r, q$), rather than the full state dimension ($N$). This ensures the method remains scalable for large-scale problems where $N \gg m$.
• Algorithm: The process involves:
  1. Computing the SVD of the centered snapshot matrix.
  2. Selecting the top $m$ singular vectors as the candidate span.
  3. Solving the constrained optimization problem on the Stiefel manifold $St(m, r+q)$ to find the optimal rotation matrices $Q_r$ and $Q_q$.
  4. Reconstructing the optimal bases and solving for the quadratic weighting matrix $\Xi$ via linear least squares.

Key Contributions

1. Novel Optimization Framework: The paper introduces a shift from discrete, combinatorial basis selection to continuous subspace identification via Riemannian optimization, allowing for the "rotation" of the basis to better align with nonlinear structures.
2. Scalable Objective Function: The derivation of an objective function whose evaluation is independent of the full state-space dimension $N$, enabling application to large-scale systems.
3. Performance Validation: A comprehensive comparison demonstrating that FastQM outperforms standard POD, POD-based QMs, and greedily constructed QMs in reconstruction accuracy.

Numerical Results
The efficacy of FastQM was demonstrated on a large-eddy simulation (LES) of a turbulent airfoil wake (NACA 0012 at $Re=23,000$), featuring a state dimension of $N \approx 6.6 \times 10^5$.

• Accuracy: Across various reduced dimensions ($r$) and numbers of nonlinear modes ($q$), FastQM consistently achieved the lowest relative reconstruction error compared to POD, POD-based QM, and Greedy QM.
• Comparison with Greedy Methods: While Greedy QM performed similarly to POD-based QM at low dimensions (due to the initial selection matching leading singular vectors), FastQM avoided this limitation by optimizing the continuous rotation of the basis, capturing critical nonlinear features that discrete selection missed.
• Mode Structure: Visual analysis of the optimized modes revealed that they resolve significantly finer scales compared to standard POD modes, facilitating better quadratic approximation.
• Convergence: The optimization converged efficiently (e.g., within 86 iterations for the LES case) with a manageable computational footprint (approx. 20 seconds for the specific test case).

Significance and Claims
The authors claim that FastQM establishes a viable choice for large-scale model reduction tasks by overcoming the suboptimality of standard POD-based quadratic manifolds. The paper asserts that:

• The optimality of POD coordinates is strictly limited to linear approximations; for quadratic manifolds, the coordinate system must be optimized.
• Continuous optimization on the Stiefel manifold provides a richer search space than greedy selection, optimizing the synergy between linear and quadratic terms without the combinatorial explosion.
• The method achieves superior reconstruction fidelity with minimal computational overhead, making it suitable for high-dimensional systems where traditional optimization approaches would be intractable.

The work concludes that while the specific application was a turbulent airfoil wake, the framework is general, and future work aims to extend this to a wider variety of engineering problems to further assess generalizability.