KANDy: Kolmogorov-Arnold Networks and Dynamical System Discovery

1. Problem Statement

The discovery of governing equations for nonlinear dynamical systems, particularly those exhibiting chaos, faces significant hurdles:

• Predictability Limits: Chaotic systems (e.g., Lorenz, Kuramoto-Sivashinsky) suffer from exponential divergence of trajectories (Lyapunov instability). Standard data-driven models often fail to maintain long-term coherence, diverging significantly after just one or two Lyapunov times.
• Limitations of Sparse Regression: Methods like SINDy (Sparse Identification of Nonlinear Dynamics) rely on the assumption that governing equations are sparse (composed of a few terms from a pre-defined library). This fails for systems with non-sparse structures (e.g., Ikeda map, Holling-type functional responses) or complex topological invariances where the "true" library terms are unknown or infinite.
• Limitations of Deep Learning: While deep neural networks (MLPs, Neural ODEs) can approximate vector fields, they often lack interpretability (acting as black boxes) and struggle to recover explicit symbolic equations. Furthermore, standard deep KANs (Kolmogorov-Arnold Networks) with multiple hidden layers can obscure the global equation structure due to compositional complexity.
• Topological Obstructions: Certain systems possess symmetries or topological structures (e.g., Hopf fibrations, quotient spaces) that standard architectures fail to capture without specific inductive biases.

2. Methodology: The KANDy Framework

The authors propose KANDy (Kolmogorov-Arnold Network for Dynamics), a novel architecture that synthesizes Kolmogorov-Arnold Networks (KANs) with sparse regression principles but overcomes their limitations through a specific design:

A. Zero-Depth, Wide Architecture

Unlike traditional deep KANs, KANDy employs a zero-depth (single-layer) wide architecture.

• Structure: It maps system states directly to time derivatives ($\dot{x}$) without hidden layers.
• Mechanism: It functions as a generalized regression where the "weights" are learnable univariate spline functions.
• Benefit: This preserves the interpretability of the model, allowing the learned mapping to be directly translated into symbolic governing equations.

B. Lifted Feature Space (Koopman-Inspired)

To handle non-sparse and complex dynamics, KANDy does not rely solely on raw state variables ($x, y, z$). Instead, it uses a lifting mechanism:

• Input Augmentation: The input vector $\Theta$ is constructed by applying a lifting function $\Phi$ to the observables $x$. This includes physically informed terms (e.g., products like $xy$, $xz$, or higher-order polynomials) and potentially infinite-dimensional features.
• Theoretical Basis: This approach aligns with Koopman operator theory, transforming the nonlinear dynamics in the original space into approximately linear dynamics in a lifted, higher-dimensional space.
• Flexibility: Unlike SINDy, KANDy does not require the user to pre-specify a sparse library; the KAN learns the optimal nonlinear transformations of the lifted features.

C. Training Objective

KANDy utilizes a composite loss function to ensure both local accuracy and global stability:

1. Derivative Loss ($L_{deriv}$): Minimizes the error between the predicted vector field and the true time derivatives ($\dot{x}$).
2. Rollout Loss ($L_{roll}$): Minimizes the error between the model's autoregressive trajectory and the true trajectory over a time horizon. This is crucial for preventing error accumulation and ensuring the model respects the system's attractor geometry.

• Total Loss: $L(\theta) = L_{deriv}(\theta) + \lambda_{roll} L_{roll}(\theta)$.

D. Symbolic Extraction

A key innovation is the symbolic formula extraction procedure:

• After training, the learned spline activations on each edge are fitted to a library of symbolic functions (polynomials, trigonometric, etc.).
• A complexity-regularized selection process prunes edges with poor fits or high complexity, effectively "zeroing out" irrelevant terms.
• This yields a sparse, human-readable symbolic equation directly from the trained network.

3. Key Contributions

1. Synthesis of KANs and Sparse Regression: KANDy bridges the gap between the interpretability of sparse regression and the expressive power of neural networks, relaxing the strict sparsity constraints of SINDy.
2. Theoretical Analysis of KAN Depth: The authors prove (via Theorems 1–7) that deep KANs with narrow widths are insufficient for representing certain bilinear terms (e.g., $xy$ in the Lorenz system) due to algebraic obstructions. They demonstrate that a wide, zero-depth architecture with lifted features is necessary to recover these dynamics.
3. Topological Discovery: KANDy successfully recovers invariant structures in systems with non-trivial topology, specifically demonstrating the ability to learn the Hopf fibration (a quotient map collapsing $S^3$ to $S^2$), a task where standard deep KANs failed.
4. Robustness to Non-Sparse Systems: The method successfully discovers equations for systems that violate sparsity assumptions, such as the Ikeda optical cavity map and Holling-type ecological models, where traditional sparse regression fails.

4. Results and Benchmarks

The model was evaluated on discrete systems, continuous ODEs, and chaotic PDEs:

• Hopf Fibration (Topological Test):

  • Standard KAN: Failed to preserve fiber structure, collapsing orbits incorrectly.
  • KANDy: Successfully learned the quotient map, preserving the topological invariance with near-zero error ($R^2 \approx 1$) and recovering the correct symbolic form involving quadratic terms.

• Discrete Systems (Hénon & Ikeda Maps):

  • Hénon: Recovered the governing equations with 100% fidelity to true parameters.
  • Ikeda: Successfully modeled the non-sparse, trigonometric dynamics of the optical cavity, a known failure case for SINDy.

• Continuous ODEs (Lorenz System):

  • Equation Discovery: Recovered the Lorenz equations with high accuracy. While minor deviations existed in linear damping coefficients (likely due to data centering), the critical nonlinear coupling terms ($xy$, $xz$) and parameters ($\rho, \beta$) were identified within acceptable error margins.
  • Forecasting: Maintained phase and amplitude coherence with the attractor for 5 Lyapunov times, significantly outperforming standard autoregressive rollouts that typically diverge after 1–2 Lyapunov times.

• Partial Differential Equations (KS & Inviscid Burgers):

  • Kuramoto-Sivashinsky (KS): Recovered the correct library terms (nonlinear advection $u u_x$, second derivative $u_{xx}$, fourth derivative $u_{xxxx}$). The model captured the spatiotemporal chaos and pattern formation.
  • Inviscid Burgers: Successfully handled shock formation and discontinuities. In head-to-head comparisons, KANDy achieved a significantly lower Normalized RMSE (0.049) compared to SINDy/OLS (0.150) and PDE-Find (1.562), correctly identifying the single dominant flux term while suppressing spurious terms.

5. Significance and Conclusion

KANDy represents a paradigm shift in scientific machine learning for dynamical systems:

• Beyond Depth: It challenges the conventional deep learning dogma that "depth equals complexity," showing that for dynamical systems, width and appropriate feature lifting are more critical than depth for equation discovery.
• Interpretability: It provides a direct path from data to symbolic governing equations, even for complex, non-sparse, and topologically rich systems.
• Stability: By integrating rollout losses, it produces models that are not just locally accurate but globally stable, preserving the geometric structure of attractors over long horizons.
• Generalizability: The framework is applicable to a wide range of problems, from low-dimensional chaos to high-dimensional PDEs and topological quotient maps, offering a unified approach to data-driven modeling where traditional sparse regression fails.

In summary, KANDy effectively combines the theoretical foundations of the Kolmogorov-Arnold representation theorem with practical Koopman lifting and sparse regression techniques to create a robust, interpretable, and highly accurate tool for discovering the laws of nature from data.