Regularized dynamical parametric approximation

1. Problem Statement

The paper addresses the numerical approximation of evolution equations (ordinary and partial differential equations) of the form $\dot{y} = f(y)$ using nonlinear parametrizations $u(t) = \Phi(\theta(t))$, where $\theta(t)$ are time-dependent parameters to be determined.

Key Challenges:

• Irregular Parametrizations: In many high-dimensional applications (e.g., multi-Gaussian approximations in quantum dynamics, tensor networks, and deep neural networks), the derivative map $\Phi'(\theta)$ is often ill-conditioned. It may possess arbitrarily small singular values, varying rank, or a non-trivial nullspace.
• Ill-posed Subproblems: Standard approaches (like the Dirac–Frenkel time-dependent variational principle) require solving linear least-squares problems to determine $\dot{\theta}$. When $\Phi'$ is singular or near-singular, these problems become ill-posed, leading to numerical instability and error amplification.
• The Paradox: The authors note that while standard numerical analysis dictates avoiding ill-posed subproblems to ensure stability, this specific class of problems requires solving them to achieve high-fidelity approximations in complex systems.

2. Methodology

The core methodology introduces a regularized dynamical parametric approximation. Instead of solving the standard (potentially singular) least-squares problem, the time derivative $\dot{\theta}$ is determined by minimizing a regularized linear least-squares functional:

$$\min_{\dot{\theta}} \left( \| \dot{u} - f(u) \|_H^2 + \varepsilon(t)^2 \| \dot{\theta} \|_Q^2 \right)$$

where:

• $\dot{u} = \Phi'(\theta)\dot{\theta}$.
• $\varepsilon(t) > 0$ is a regularization parameter.
• $H$ is the state space (Hilbert space) and $Q$ is the parameter space.

Key Algorithmic Features:

1. Regularized Projection: The method can be interpreted as an orthogonal projection of the vector field $f(u)$ onto the tangent space of the manifold defined by $\Phi$, weighted by the regularization parameter $\varepsilon$.
2. Time Discretization: The resulting differential equation for $\theta$ is solved using standard time-stepping methods (Euler, Runge-Kutta). Crucially, the paper establishes a nontrivial interplay between the step size $h$ and the regularization parameter $\varepsilon$.
3. Adaptive Selection: The authors propose an adaptive strategy to select $\varepsilon_n$ and $h_n$ at each step to balance the defect size (residual) and the regularization term, ensuring stability without excessive computational cost.
4. Handling Conservation Laws: The paper extends the framework to enforce conserved quantities (e.g., energy, mass) by adding constraints to the least-squares problem, utilizing Lagrange multipliers.

3. Key Contributions

A. Theoretical Error Analysis

The paper provides rigorous error bounds for both time-continuous and time-discretized settings:

• Defect-Dependent Bounds: The global error is shown to be bounded by the defect size $\delta$ (the minimum value of the regularized functional) and the time step size, rather than the condition number of $\Phi'$.
• Step Size Restriction: A critical theoretical finding is the necessary condition for stability in the discrete case:
  $$h \delta \leq c \varepsilon^2$$
  This implies that as the regularization parameter $\varepsilon$ decreases (to improve accuracy), the time step $h$ must decrease quadratically relative to the defect.
• Sensitivity Analysis: The authors analyze the sensitivity to initial values. They show that while the parameters $\theta$ may be highly sensitive to perturbations (scaling as $e^{ct/\varepsilon}$), the physical solution $u = \Phi(\theta)$ remains stable up to the order of the defect size. This distinguishes the behavior of linear vs. nonlinear parametrizations.
• Quasi-Optimality: The method achieves quasi-optimality, meaning the error is comparable to the best possible approximation within the tangent space, provided the defect is small.

B. Case Studies

1. Gradient Systems (Optimization):
   • Applied to minimizing a function $V(y)$ via parametric steepest descent.
   • Result: Unlike standard gradient descent on the parameter space (which can get stuck in local minima of the composed function $V \circ \Phi$), the regularized parametric approach converges to the global minimum of strongly convex functions, with an error controlled solely by the defect size.
2. Time-Dependent Schrödinger Equation:
   • Addresses the challenge of unbounded operators (the Laplacian).
   • Modifications: The authors propose modified regularized schemes (e.g., modifying the projection to handle the Laplacian term specifically) to preserve the self-adjoint nature of the Hamiltonian.
   • Strang Splitting: They derive error bounds for a regularized parametric Strang splitting method, proving optimal-order convergence ($O(h^2 + \delta)$) for quantum dynamics.

C. Numerical Experiments

The theory is validated through two distinct applications:

1. Lotka-Volterra System (Neural Networks): Approximating the flow map of a predator-prey model using a deep neural network. Results confirm the theoretical step-size restriction and show $O(\varepsilon)$ convergence.
2. Double-Well Quantum Dynamics (Multi-Gaussian): Simulating tunneling dynamics using a sum of 36 complex Gaussians. The experiments demonstrate that the method preserves energy and norm conservation (when enforced) and accurately captures quantum tunneling, even with small $\varepsilon$.

4. Results and Significance

• Resolution of the "Ill-Posed" Paradox: The paper demonstrates that solving ill-posed subproblems via regularization is a viable and stable strategy for dynamical approximation, provided the step size is adapted to the regularization parameter.
• Robustness in High Dimensions: The approach is particularly significant for high-dimensional problems where traditional grid-based methods fail (curse of dimensionality). It provides a theoretical foundation for using neural networks and tensor networks in time-dependent simulations.
• Unified Framework: It unifies various disparate methods (Dirac-Frenkel, neural Galerkin schemes, Gaussian wavepackets) under a single regularized framework, offering a systematic way to handle the numerical instabilities inherent in these methods.
• Practical Guidance: The derived condition $h \delta \leq c \varepsilon^2$ provides concrete, actionable guidelines for practitioners implementing these methods, preventing the common failure mode of choosing $\varepsilon$ too small relative to the time step.

In summary, this paper establishes a rigorous mathematical foundation for regularized dynamical parametric approximation, proving that despite the inherent ill-conditioning of the parameter space, stable and accurate numerical solutions can be achieved through careful regularization and adaptive time-stepping. This bridges the gap between theoretical approximation capabilities and practical numerical stability in quantum dynamics and machine learning-based ODE/PDE solvers.