🤖 machine learning

Discovery of Nonlinear Dynamics with Automated Basis Function Generation

This paper introduces AutoSINDy, a hybrid framework that combines PySR-based symbolic regression with SINDy's sparse identification to automatically generate and curate basis functions, enabling robust discovery of accurate and parsimonious governing equations for nonlinear systems even under high noise.

Original authors: Mohammad Amin Basiri, Charles Nicholson

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammad Amin Basiri, Charles Nicholson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the secret rules of a mysterious machine just by watching it move. You have a video of the machine's gears spinning, a pendulum swinging, or a fluid swirling. Your goal is to write down the exact mathematical formula that explains why it moves that way.

This is the challenge of discovering governing equations. For a long time, scientists have had two main ways to solve this puzzle, but both have a major flaw:

The "Guess-and-Check" Method (SINDy): Imagine you have a giant box of Lego bricks. You know the machine is made of Legos, so you try to build the machine using every brick in the box. If you leave out the one specific brick the machine actually uses, your model fails. If you use too many bricks, your model becomes a messy, unstable tower that falls over. The problem? You have to guess which bricks to put in the box before you start building.
The "Wild Imagination" Method (Symbolic Regression): Imagine you let a monkey type on a keyboard, randomly hitting keys to create sentences. Eventually, it might accidentally type a perfect sentence. This method is very flexible and doesn't need a pre-made box of bricks. But it's messy, slow, and often produces gibberish that looks like a sentence but makes no sense when you try to read it. It's also very sensitive to static on the line (noise).

Enter AutoSINDy: The Smart Architect

The paper introduces AutoSINDy, a new method that acts like a smart architect who combines the best of both worlds. Instead of guessing the bricks or letting a monkey type, AutoSINDy uses a three-step process to find the perfect, simple formula automatically.

Here is how it works, step-by-step:

Step 1: The "Sniper" Search (Discovery)

Instead of looking at the whole machine at once, AutoSINDy takes tiny, random snapshots of the machine's movement (like taking quick photos of a spinning fan). It uses a powerful tool called PySR (a digital "monkey" that is actually a very smart evolutionary algorithm) to look at these tiny snapshots and shout out, "Hey, I see a pattern here! Maybe it's a sine wave? Maybe it's a square?"

The Analogy: Think of this as a scout running through a forest, picking up interesting leaves, rocks, and twigs. The scout doesn't try to build the house yet; they just gather a huge, messy pile of potential building materials.

Step 2: The "Clean-Up" Crew (Curation)

The scout's pile is messy. It has duplicates, broken pieces, and things that look similar but aren't quite right. AutoSINDy now acts as a strict editor.

It breaks complex sentences down into their smallest words (atomic parts).
It checks for "clones." If it finds two pieces that do the exact same job, it throws one away.
It keeps the simplest version of a pattern. If a simple "x" explains the movement, it won't keep a complicated "x + x² - x" version.
The Analogy: This is like a chef tasting a giant pot of soup, removing the duplicate ingredients, and throwing out the burnt spices until only the perfect, essential flavors remain. The result is a clean, organized spice rack.

Step 3: The "Final Build" (Identification)

Now, with a clean, organized spice rack (the library of terms), AutoSINDy uses the SINDy method to mix the ingredients. Because the ingredients are already clean and non-redundant, the mixing is fast, stable, and accurate. It figures out exactly how much of each "spice" (coefficient) to use to recreate the machine's motion perfectly.

The Analogy: This is the final assembly. Because the chef didn't have to sort through a messy pile of garbage, the final dish is delicious, simple, and exactly what the recipe called for.

Why is this a big deal?

The paper tested this on six different "machines" (mathematical systems), ranging from simple swings to chaotic, unpredictable weather-like systems. They added "static" (noise) to the data to make it harder to read, just like a bad phone connection.

Here is what happened:

The Old Ways: The "Guess-and-Check" method often built towers that were too tall and fell over (unstable). The "Wild Imagination" method often wrote gibberish that looked okay for a second but fell apart immediately.
AutoSINDy: It found the exact right formula 92.8% of the time, even with noisy data.
Simplicity: The formulas it found were short and sweet (like a haiku), whereas the other methods produced long, bloated paragraphs that were hard to understand and slow to run.

The Bottom Line

AutoSINDy is a tool that automates the hardest part of scientific discovery: figuring out what to look for. It doesn't need a human to say, "I think there's a sine wave in there." It hunts for the patterns, cleans them up, and builds the simplest, most accurate rulebook for how the system works.

It's like giving a scientist a magic magnifying glass that not only finds the clues but also organizes them into a clear, easy-to-read story, even when the clues are scattered and dirty.

Technical Summary: Discovery of Nonlinear Dynamics with Automated Basis Function Generation

Problem Statement
The discovery of governing mathematical equations from observational data is a fundamental challenge in scientific modeling, particularly when the underlying structure is unknown. Existing methods face a dichotomy:

Sparse Identification of Nonlinear Dynamics (SINDy): While effective at producing parsimonious, interpretable models, SINDy relies on a user-specified library of candidate basis functions. If critical terms are omitted due to unconventional dynamics or domain ignorance, the method fails. Conversely, enriching the library to avoid omissions introduces severe multicollinearity, destabilizing sparse regression and inflating coefficient uncertainty.
Symbolic Regression (SR): Approaches like PySR offer unlimited flexibility by searching the space of mathematical expressions without a pre-built library. However, they are computationally intensive, highly sensitive to measurement noise, and prone to producing overfitted, numerically unstable equations that fail during long-horizon integration. Furthermore, standard SR optimizes state variables independently, struggling to enforce the coupled structure of multi-dimensional dynamical systems.

The core problem addressed is the lack of a method that can automatically construct a minimal, physics-appropriate function library from raw data without prior knowledge, balancing the expressivity of SR with the robustness of sparse regression.

Methodology: The AutoSINDy Framework
The authors propose AutoSINDy, a hybrid "Discovery-then-Solve" framework that decomposes the equation discovery problem into three sequential stages:

Stage 1: Symbolic Mining (Discover)
- Instead of applying symbolic regression to the full trajectory, the method operates on $K$ short, randomly sampled data chunks (bootstrapped windows).
- PySR (a multi-population evolutionary symbolic regression tool) is applied to each chunk to mine candidate functional forms.
- Crucially, the method harvests the entire Pareto-optimal front of expressions, not just the single best model. This ensures that structurally correct atomic sub-terms are captured even if the full expression is a poor fit for a specific local window.
- Nested trigonometric compositions are explicitly forbidden to prevent degenerate identities.
Stage 2: Library Curation (Curate)
- The raw pool of symbolic expressions is processed to construct a compact, non-redundant library.
- Decomposition: Expressions are parsed and decomposed into additive atomic sub-terms (e.g., $3x^2 + 2\sin(y) \to \{x^2, \sin(y)\}$ ).
- Expansion: Configurable strategies (Gentle, Severe, or Hybrid) expand grouped polynomial or trigonometric terms into monomials to ensure coverage.
- Collinearity Pruning: A greedy forward-selection procedure sorts candidates by complexity (operator count) and filters them. A candidate is accepted only if its correlation with already-accepted terms is below a threshold ( $\rho_{max}$ ) or if its Variance Inflation Factor (VIF) is acceptable. This enforces Occam's razor at the library level, ensuring simpler terms are preferred over complex, redundant ones.
Stage 3: Sparse Identification (Identify)
- The curated library is fed into a SINDy optimizer (STLSQ or SR3).
- An Ensemble approach (inspired by E-SINDy) is used: $B$ bootstrap replicates are drawn, and sparse models are fitted to each.
- A hard-cutoff inclusion filter retains only terms that appear in a user-specified fraction ( $\kappa$ ) of the bootstrap models. This provides strong noise rejection, discarding spurious terms that appear only due to noise artifacts.
- The framework supports both separate libraries (per state variable) and unified libraries (shared across variables), with separate libraries yielding better parsimony in experiments.

Key Contributions

Automated, Assumption-Free Library Generation: A PySR-based engine that mines candidate functional forms from bootstrapped data without requiring prior specification of the mathematical structure, directly addressing the library design bottleneck of SINDy.
Principled Symbolic Curation Pipeline: A three-stage mechanism (decomposition, expansion, collinearity pruning) that transforms unstructured genetic algorithm output into a clean, non-redundant linear basis suitable for robust sparse regression.
Ensemble-Based Sparse Identification: Integration of the curated library with SINDy optimizers and bootstrap ensemble filtering to achieve high noise robustness and interpretability.
Systematic Benchmarking: Comprehensive evaluation across six canonical nonlinear systems (oscillatory, limit-cycle, and chaotic) under varying noise levels, comparing against standard SINDy with enriched libraries and standalone PySR.

Results
Extensive experiments on six benchmark systems (Harmonic Oscillator, Damped Pendulum, Modulated Oscillator, Van der Pol, Duffing Oscillator, and Complex Lorenz) demonstrate:

Ground-Truth Recovery: AutoSINDy achieves a 92.8% ground-truth recovery rate across all trials (defined as $R^2 > 0.99$ on a clean, unseen validation trajectory). This significantly outperforms Standard SINDy (58.9%) and Standard PySR (28.3%).
Simulation Stability: AutoSINDy exhibits superior long-horizon simulation stability, with 0% hard integration crashes (NaN/Inf or divergence) across 180 trials. In contrast, Standard SINDy failed in 51.7% of trials and Standard PySR in 75.0% of trials.
Parsimony: AutoSINDy recovers equations with complexity (operator count) within 1–2 operators of the ground truth. Standard SINDy produced equations with complexity up to 29.2 $\times$ the ground truth (e.g., 175 operators for Van der Pol vs. true 6), while Standard PySR often under-compressed, missing critical terms.
Noise Robustness: AutoSINDy maintains high accuracy even at high noise levels ( $\sigma = 0.05$ ), whereas baselines degrade significantly or fail entirely.
Computational Efficiency: While discovery time is comparable to standalone PySR (due to the shared engine), the resulting compact equations make AutoSINDy the fastest method for simulation, as the reduced complexity avoids the numerical stiffness associated with bloated equations.

Significance and Claims
The paper claims that AutoSINDy successfully bridges the gap between the flexibility of symbolic regression and the robustness of sparse identification. By automating the feature-engineering bottleneck, it enables the discovery of correct governing terms without prior domain knowledge. The authors assert that the method produces equations that are not only structurally accurate but also dynamically stable and interpretable, offering a scalable pathway for discovering complex dynamics in biological and physical systems where the mathematical form is unknown. The work emphasizes that predictive accuracy on training data is insufficient; true scientific value requires equations that remain stable under forward integration and possess low symbolic complexity.