🔢 mathematics

A Dynamic Subspace Approach for Low-rank Approximation of Large-scale Nonlinear Systems

This paper introduces a dynamic subspace approach that learns time-continuous trajectories on the Grassmannian manifold to efficiently approximate large-scale nonlinear systems, effectively overcoming the Kolmogorov barrier and achieving higher accuracy than static methods by decoupling geometric drift from intrinsic state evolution while maintaining computational scalability.

Original authors: Jack DeChant, Rudy Geelen, Shane A. McQuarrie, Johann Guilleminot

Published 2026-05-26

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Jack DeChant, Rudy Geelen, Shane A. McQuarrie, Johann Guilleminot

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Rigid Suit" vs. The "Chameleon"

Imagine you are trying to describe a complex, moving object—like a swirling cloud of smoke or a wave of water crashing against a wing.

The Old Way (Static Approximation):
Traditional methods, like the ones used for decades (called POD), try to describe this moving object using a rigid suit. They pick a fixed set of "building blocks" (basis vectors) at the very beginning and say, "We will describe everything using these specific blocks."

The problem? If the smoke swirls or the wave moves, that rigid suit doesn't fit anymore. To make the suit fit a moving shape, you have to keep adding more and more extra layers (increasing the "rank" or dimension). Eventually, the suit becomes so heavy and bulky that it defeats the purpose of simplifying the data. This is what the authors call the Kolmogorov barrier: the point where trying to force a moving shape into a static box requires so much space that you lose the benefit of compression.

The New Way (Dynamic Subspace):
The authors propose a chameleon suit. Instead of a rigid set of blocks, they create a set of building blocks that can morph and rotate as time passes. They call this a "dynamic subspace."

The Secret Ingredient: The "Grassmannian" Highway

How do they make these blocks move smoothly? They use a mathematical concept called the Grassmannian manifold.

Think of the Grassmannian as a special highway where every point represents a different "angle" or "direction" your building blocks can face.

Static methods try to jump from one point to another on this highway, often taking a messy, inefficient path.
This new method finds the geodesic. In geometry, a geodesic is the shortest, smoothest path between two points on a curved surface (like a plane flying the "great circle" route between two cities).

By forcing the building blocks to travel along this smooth, shortest path (the geodesic), the method can track the physics of the moving system perfectly without needing to add extra layers. It separates the movement of the shape (the blocks rotating) from the size of the shape (the data inside the blocks).

The "Feature Space" Trick: Solving the Puzzle in a Small Room

There is a catch: The data they are analyzing is massive. Imagine trying to solve a puzzle with millions of pieces (the full state of a fluid flow). Doing the math to figure out how the blocks should rotate would take forever on a normal computer.

The authors use a clever trick called Feature Space Reduction:

The Summary: First, they look at all the data and create a "summary sketch" using the most important patterns (like taking a high-level photo of the puzzle).
The Small Room: They do all the complex math to figure out the moving blocks inside this small summary room, not in the giant room with millions of pieces.
The Result: Because they are doing the heavy lifting in a small room, the computer speed doesn't depend on how huge the original data was. It scales efficiently.

What They Tested It On

The authors tested this "chameleon suit" on two specific scenarios to prove it works better than the "rigid suit":

A Moving Wave (1D Transport): Imagine a single pulse of energy moving across a line.
- The Old Way: Needed many blocks to track the pulse as it moved, and the error was high.
- The New Way: The blocks rotated to follow the pulse perfectly. It achieved much higher accuracy with the same number of blocks.
Turbulent Airflow (Airfoil Wake): They looked at a massive, complex simulation of air swirling behind an airplane wing (a turbulent wake).
- The Old Way: The static blocks smoothed out the details, missing the fine, chaotic swirls of the turbulence.
- The New Way: The dynamic blocks rotated to capture the fine details of the turbulence. It reduced the error by nearly half compared to the old method, all while using the same amount of computing power.

The Bottom Line

This paper presents a new way to simplify complex, moving data. Instead of forcing a moving shape into a static box (which requires a huge box), they let the box itself rotate and adapt along the smoothest possible path.

By doing this math in a "small room" (feature space), they can handle massive datasets efficiently. The result is a method that is more accurate, more efficient, and better at tracking moving physics than the standard tools currently used in engineering and science.

Technical Summary: A Dynamic Subspace Approach for Low-rank Approximation of Large-scale Nonlinear Systems

Problem Statement
Effective low-dimensional approximation is critical for data compression, machine learning, and model reduction. However, standard static methods, such as Proper Orthogonal Decomposition (POD) and linear autoencoders, rely on fixed projection spaces. These static representations fail to adequately capture transient, transport-dominated phenomena where physical modes or coherent structures drift over time. To maintain accuracy in such scenarios, static linear methods often require a significant increase in approximation rank (dimensionality), a limitation known as the "Kolmogorov barrier." While nonlinear methods can overcome this expressivity limit, they often sacrifice interpretability or computational efficiency and rely on fixed embeddings that do not adapt to the evolving state.

Methodology
The authors propose a dynamic subspace approach that models the evolution of the approximation basis as a time-continuous trajectory on the Grassmannian manifold ( $G(r, N)$ ), the space of all $r$ -dimensional linear subspaces in $\mathbb{R}^N$ .

Geometric Formulation:
Instead of a fixed basis $V$ , the method employs a time-varying basis $V(t)$ parameterized as a geodesic path (the shortest path between subspaces). The basis is defined by:
$V(\tau) = V_1 \cos(\tau \Theta) + V_2 \sin(\tau \Theta)$
where $V_1$ and $V_2$ are orthonormal matrices, $\tau$ is normalized time, and $\Theta$ is a diagonal matrix of principal angles representing the rotation rates. This formulation decouples the geometric drift of the subspace from the intrinsic evolution of the low-dimensional coordinates, allowing the basis to adapt to the instantaneous physics without inflating the rank $r$ .
Scalable Optimization via Feature Space:
Direct optimization over the high-dimensional state space (dimension $N$ ) is computationally intractable for large-scale systems. To address this, the authors introduce a feature space reduction:
- A fixed set of $n_f$ feature modes (derived from the leading singular vectors of the snapshot data via POD) is used to construct a reduced basis $\tilde{V} \in \mathbb{R}^{N \times n_f}$ .
- The dynamic basis is constrained to lie within the span of these features: $V(\tau) = \tilde{V} T(\tau)$ , where $T(\tau)$ is a low-dimensional rotation matrix.
- This shifts the optimization variables from the high-dimensional space ( $N \times 2r$ ) to the feature space ( $n_f \times 2r$ ), making the computational cost independent of the original state dimension $N$ .
Riemannian Optimization:
The learning problem is formulated as minimizing the mean squared projection error over the training snapshots. The optimization is performed on a product manifold $\mathcal{M} = \text{St}(n_f, 2r) \times \mathbb{R}^r$ , where $\text{St}$ denotes the Stiefel manifold of orthonormal matrices. The authors utilize a Riemannian conjugate gradient solver (via the Pymanopt package with a JAX backend) to strictly enforce orthogonality constraints and navigate the manifold geometry. The implementation uses vectorized operations to compute projections for all time steps simultaneously, eliminating the overhead of iterative time-looping.

Key Contributions

Geodesic Parameterization: Reframing dimensionality reduction as a differential-geometric task where the basis evolves along geodesics on the Grassmannian, enabling adaptive tracking of non-stationary dynamics.
Decoupling of Costs: A scalable framework that decouples the optimization cost from the high-dimensional state space $N$ by performing learning in a precomputed, reduced feature space.
Rank Efficiency: The method achieves higher accuracy than static linear approximations at equivalent ranks, effectively breaking the Kolmogorov barrier for transport-dominated phenomena without requiring rank inflation.
Vectorized Implementation: A fully vectorized, loop-free optimization strategy that leverages modern automatic differentiation and parallel processing capabilities.

Results
The framework was evaluated on two distinct problems:

1D Transport Equation: A benchmark for traveling waves. The dynamic subspace approach demonstrated significantly faster convergence in relative error compared to static POD. At a reduced dimension of $r=20$ , the dynamic method achieved accuracy comparable to a static POD basis with rank $2r$ , validating the efficacy of the geodesic perspective.
Turbulent Airfoil Wake (LES): A large-scale, high-fidelity dataset ( $N \approx 6.6 \times 10^5$ $N \approx 6.6 \times 1 0^{5}$ ) involving complex, non-stationary turbulence.
- The dynamic subspace approach consistently outperformed static POD across all tested ranks.
- At $r=200$ , the dynamic method achieved a relative error of 2.4% compared to 4.0% for POD on a test snapshot.
- The dynamic basis captured fine-scale turbulent structures and maintained time continuity, whereas static POD modes produced smoother, more diffused results.
- The optimization converged in approximately 25 seconds for $r=80$ on a standard CPU, demonstrating scalability.

Significance and Claims
The paper claims that this dynamic subspace approach offers a robust and scalable alternative to traditional manifold learning for the low-rank modeling of complex, non-stationary systems. By parameterizing the basis as a geodesic, the method allows the approximation space to evolve alongside the high-dimensional state, ensuring synchronization with instantaneous physics.

The authors emphasize that this approach:

Mitigates the Kolmogorov barrier by avoiding the rank inflation typically required by static methods for transport-dominated flows.
Remains independent of governing equations, requiring no explicit parametrization of the underlying physics, which facilitates integration into existing production-level simulation codes.
Provides a mathematically rigorous framework that shifts computational focus from manipulating high-dimensional vectors to optimizing temporal coefficients and principal angles.

The work positions this method as a viable enabler for high-fidelity digital twins in the exascale computing era, offering near-optimal subspace alignment with significantly lower computational complexity than traditional approaches. Future work is noted to focus on extending the method to piecewise geodesics for longer time horizons and coupling the adaptive basis with existing data-driven model reduction frameworks.