Technical Summary: Diffeomorphic Vector Field Alignment for Learned Models (DFORM)

Problem Statement

The paper addresses two primary challenges in the analysis of high-dimensional dynamical systems, particularly recurrent neural networks (RNNs) used as surrogates for brain networks in theoretical neuroscience:

1. Coordinate System Heterogeneity: Dynamical models often implement qualitatively similar mechanisms (e.g., limit cycles, attractors) but exist in arbitrary coordinate systems. There is no a priori equivalence between the state spaces of different models, making direct comparison of their vector fields impossible. Traditional methods like dimensionality reduction or Representation Similarity Analysis (RSA) often rely on trajectory alignment or assume state independence, failing to capture the intrinsic geometric structure of the generative vector fields.
2. Intractability of Low-Dimensional Motifs: Identifying important dynamical features, such as saddle points or saddle limit cycles, in high-dimensional nonlinear models is analytically intractable. While stable attractors can be found via forward simulation, "improbable" limit sets (e.g., saddles) have zero measure basins of attraction and cannot be discovered through standard trajectory simulation.

Methodology: DFORM

The authors propose Diffeomorphic vector field alignment FOR learned Models (DFORM), a framework that learns a smooth, invertible coordinate transformation (a diffeomorphism, $\phi$) to align the vector fields of two dynamical systems.

Mathematical Formulation

Given two systems $\dot{x} = f(x)$ and $\dot{y} = g(y)$, DFORM seeks a diffeomorphism $\phi: \mathbb{R}^n \to \mathbb{R}^n$ such that the pushforward of $f$ by $\phi$ ($\phi_* f$) aligns with $g$. The systems are smoothly orbitally equivalent if $\phi_* f(y) = c(y)g(y)$ for some positive scalar function $c(y)$, meaning the vector fields share the same direction but may differ in magnitude (speed).

The core of the method is the Orbital Similarity Loss, which minimizes the directional mismatch between the transformed vector field and the target:
$$J_{f,g,\phi}(y) = \frac{1}{n} \left\| \frac{\phi_* f(y)}{\|\phi_* f(y)\|_2} - \frac{g(y)}{\|g(y)\|_2} \right\|_2^2$$
This loss is minimized when the angle between the pushforward vector field and the target vector field is zero. To ensure robust alignment, the authors minimize the sum of forward and backward alignment losses.

Implementation

• Architecture: The diffeomorphism $\phi$ is modeled as the composition of a linear affine transformation ($H$) and a nonlinear flow generated by a Neural Ordinary Differential Equation (Neural ODE). The Neural ODE parameterizes a time-invariant vector field $v(x)$ using a feedforward neural network.
• Optimization: The model is trained using a two-stage scheme:
  1. Pre-training: The nonlinear component is fixed to identity, and the linear layer $H$ is optimized to provide a coarse alignment.
  2. Full Training: All weights (linear and nonlinear) are optimized using the orbital similarity loss.
• Cross-Dimensionality: The framework is extended to align high-dimensional systems with low-dimensional "templates" (e.g., canonical bifurcation models). This allows for the identification of invariant submanifolds and triangular decompositions by projecting the high-dimensional vector field onto a lower-dimensional target space.

Key Contributions

1. Nonlinear Vector Field Alignment: DFORM provides a computational method to directly optimize diffeomorphisms between high-dimensional vector fields without relying on linearization (e.g., Jacobian approximation) or kernel tricks.
2. Quantification of Topological Equivalence: The method produces a continuous index (orbital similarity) that quantifies how well two systems are topologically equivalent, even if they are not strictly diffeomorphic.
3. Discovery of Improbable Limit Sets: By aligning high-dimensional systems to low-dimensional templates, DFORM can locate dynamical motifs such as saddle limit cycles and invariant manifolds that are inaccessible to standard forward simulation.
4. Handling Coordinate Heterogeneity: The framework successfully recovers ground-truth transformations (both linear and nonlinear) between systems that implement the same dynamics but in different coordinate systems.

Experimental Results

The authors validated DFORM across several domains:

• Linear Systems: DFORM successfully aligned topologically equivalent linear systems. The authors demonstrated that while linear transformations could achieve high alignment scores, multiple optimal solutions exist (non-identifiability), and linear methods fail to capture the full geometry of nonlinearly related systems.
• Nonlinear Systems:
  • Van der Pol Oscillators: DFORM aligned systems with different bifurcation parameters, with the nonlinear variant capturing the warping of limit cycles significantly better than linear alignment.
  • Flow-Transformed Systems: The method recovered nonlinear transformations generated by flows, outperforming linear approximations in trajectory matching.
  • Bifurcating Networks: Orbital similarity scores effectively distinguished between topologically distinct systems (monostable, multistable, line attractors) and reflected their functional similarity near bifurcation points.
• High-Dimensional Motif Identification:
  • Saddle Fixed Points: DFORM aligned a high-dimensional system to a 2D SNIC (Saddle Node on Invariant Cycle) template, accurately reconstructing the positions of stable and saddle fixed points.
  • Saddle Limit Cycles: The method identified a saddle limit cycle in an 18-dimensional system by matching it to a Hopf bifurcation template, a feat impossible via standard trajectory simulation.
• Empirical Neural Data: Applied to 30 MINDy models trained on human resting-state fMRI data, DFORM successfully identified models containing stable limit cycles, showing high agreement with prior numerical simulations and providing a quantitative measure of dynamical motifs.

Significance and Claims

The paper claims that DFORM represents a significant advance in the analysis of high-dimensional nonlinear models by operationalizing the theoretical concept of smooth orbital equivalence.

• Beyond Linearization: The authors argue that linearization-based methods (like DSA) are insufficient for nonlinear systems with nontrivial limit sets because they fail to account for the multiplicity of solutions and the nonlinear geometry of the state space. DFORM addresses this by optimizing directly over the group of diffeomorphisms.
• Mechanistic Insight: By enabling the comparison of models with heterogeneous coordinate systems, DFORM allows researchers to reason about shared and unique mechanisms across different models.
• Access to Improbable Dynamics: The ability to locate saddle limit sets and invariant manifolds provides new insights into the mechanistic underpinnings of models (e.g., decision-making transitions) that go beyond asymptotic behavior.
• Future Directions: The authors suggest that DFORM could facilitate the exact linearization of nonlinear systems within their basins of attraction, allowing the application of linear systems theory to complex models. They also note potential extensions to learning transformations directly from time-series data and handling time-varying inputs.

The paper maintains a modest tone regarding its claims, acknowledging that while DFORM provides a powerful tool for alignment and motif discovery, the selection of sample distributions and dynamical templates remains a critical practical consideration.