🔭 astrophysics

NestyNet. III. Symbolic Regression from Analytic Neural Surrogates

NestyNet-SR is a novel symbolic regression framework that leverages analytic neural surrogates to recursively decompose complex data into separable components, which are then distilled into exact closed-form expressions, achieving state-of-the-art performance on benchmarks and successfully rediscovering fundamental physical laws from real astronomical data.

Original authors: Rodrigo Ibata, Wassim Tenachi, Foivos Diakogiannis, Neil Ibata, Anirudh Shankar

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rodrigo Ibata, Wassim Tenachi, Foivos Diakogiannis, Neil Ibata, Anirudh Shankar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Nature often hides its simplest rules behind a veil of complexity. A physical law might look like a tangled knot of variables when written in raw data, yet it could be a single, elegant sentence if only we knew how to look at it. For centuries, scientists have relied on human intuition to find that right way of looking, to untangle the knot and reveal the clean structure underneath. But as the universe presents us with more data and more intricate systems, our unaided minds can struggle to see the pattern. The question has become whether machines can help us survey a much wider range of possibilities, uncovering relationships that fit the data perfectly and might even predict how things behave in new situations we have never seen before. This is the goal of a field called symbolic regression: the search for the actual mathematical sentences that describe the world, rather than just fitting a curve to a graph.

The challenge is that the number of possible sentences is so vast that searching through them one by one is impossible. It is like trying to find a specific book in a library where every shelf contains every possible combination of words. To solve this, researchers Rodrigo Ibata and his team have developed a new method called NestyNet-SR. Instead of guessing random sentences, their system first builds a highly accurate, flexible model of the data using a type of artificial neural network. This model acts as a surrogate, a stand-in that knows the data intimately. The crucial innovation is that this surrogate is designed to be mathematically transparent; it allows the researchers to calculate exactly how the output changes when they tweak the inputs, down to the finest details. By studying these changes, the system can detect if the data can be broken down into simpler, independent parts. It asks questions like: does this part of the system depend only on temperature, while that part depends only on pressure? Or is there a hidden combination of variables, like a specific ratio or a sum, that makes the whole thing behave simply?

Once the system identifies these simpler building blocks, it moves to the second stage: translating them back into human-readable language. This is where the method shines. It does not just look for a single perfect formula; it separates the search for the structure from the search for the numbers. Imagine trying to find the shape of a key without knowing the exact size of its teeth. The system first finds the shape—the skeleton of the equation—and then fits the precise numbers to it. This approach allows it to handle complex layers of nesting that usually stump other methods. It can discover that a relationship involves a sine wave, or a square root, or a power law, and it can do so even when the data is noisy or comes from multiple different experiments at once. The system is also strict about physical reality; it checks that the units of measurement make sense, ensuring that a formula does not accidentally add meters to seconds.

The team tested their method on a famous set of 120 physics equations, known as the AI Feynman benchmark, which are designed to be difficult for computers to solve. In a world without noise, the system successfully recovered every single one of the 120 equations exactly, a first for this type of benchmark. It did this by finding the hidden internal coordinates that made the complex equations look simple. When they added realistic noise to the data, simulating the imperfections of real-world measurements, the system still managed to find the correct structure for nearly half of the problems, and it could tell the researchers which parts of the structure were still reliable. This ability to distinguish between a true law and a lucky guess is a significant step forward.

To prove the method works on real astronomical data, the researchers applied it to a collection of spiral galaxies from the SPARC survey. They gave the computer only the separate contributions of gas and stars to the galaxy's rotation, without telling it how to combine them. The system was asked to find the rule that governs how these galaxies spin. Without any prior instructions, the algorithm discovered that the rotation speed depends on a single, combined measure of acceleration from both the gas and the stars. It found the correct way to weigh the stars against the gas, matching the known mass-to-light ratio of stellar populations. Furthermore, it confirmed that the relationship between the observed motion and this combined acceleration follows a specific, non-unique form that has been debated in astronomy for years. The system also provided a statistical measure of how confident it was in the slope of this relationship, showing that the deep, slow-moving parts of the galaxy follow a different rule than the fast-moving outer edges.

This work demonstrates that by combining accurate mathematical surrogates with a smart, tiered search strategy, we can move from raw data to interpretable laws of nature. The system does not just predict numbers; it reveals the structure of the universe, showing us that even in the complex dance of galaxies, there are simple, discoverable rules waiting to be found if we know how to look. The researchers have made their code and data available, allowing others to test these methods on their own problems, from the motion of planets to the behavior of fluids. The path from a neural network to a closed-form equation is no longer a mystery; it is a practical route to understanding the fundamental laws that govern our world.

Technical Summary: NestyNet. III. Symbolic Regression from Analytic Neural Surrogates

Problem Statement
Symbolic regression (SR) seeks closed-form mathematical expressions that fit data without assuming a fixed model class. The fundamental challenge is the combinatorial vastness of the expression space, which renders the problem NP-hard. Practical SR methods must rely on strong inductive biases to reduce the search space. A primary bias is separability: the observation that many physical laws decompose into additive, multiplicative, or compound-coordinate structures. However, detecting these structures reliably requires accurate function values and, crucially, accurate mixed second derivatives. Standard neural surrogates often lack the necessary precision for these derivative-based tests, leading to false positives and negatives that corrupt the downstream search. Furthermore, existing methods often entangle structural search with coefficient fitting, meaning a correct inner structure receives no credit until its constants are also found.

Methodology: The NestyNet-SR Pipeline
The authors present NestyNet-SR, a two-stage framework that leverages the NestyNet surrogate (introduced in Paper I) to provide highly accurate analytic derivatives. The pipeline separates structure discovery from symbolic rewriting.

Stage A: Structure Discovery (Decomposition)
- A NestyNet surrogate is trained to fit the data.
- Using analytic derivatives (gradients and Hessians), the system tests for additive separability ( $f = g(u) + h(v)$ ), multiplicative separability ( $f = g(u)h(v)$ ), and compound-coordinate dependence (where $f = g(z(x))$ ).
- The system searches for internal coordinates $z(x)$ , including monomials, affine combinations, radial groups, and centered coordinates.
- This stage recursively decomposes the multivariate problem into a tree of low-arity "neural atoms" (univariate or simple multivariate functions).
- Generalized Symmetry (GS) Layer: A novel component that solves a determining equation for Lie symmetries to discover coordinate families (monomial, affine, radial, etc.) from the gradient geometry of the target. This layer can recover coordinates with non-integer or oblique coefficients that standard detectors miss, seeding them directly into the search.
Stage B: Analytic Closure (Rewriting)
- The neural atoms from Stage A are rewritten into closed-form expressions.
- Tiered Search: The system first applies a library of specialized, high-confidence rewrite rules (e.g., homogeneity peeling, ratio invariance). If these fail, it escalates to the Factorized Symbolic Search (FSS).
- Factorized Symbolic Search (FSS): This core innovation decouples the search for the internal coordinate (skeleton) from the calibration of the outer map.
  - Instead of searching full expressions, FSS proposes constant-free "carriers" (skeletons).
  - For each carrier, it fits a family of outer maps (polynomials, power laws, sinusoids, etc.) and additive residuals.
  - Candidates are scored by the loss of the best-fitting outer map, not the full expression. This allows the system to reward a correct internal coordinate even if the outer constants are not yet discrete tokens.
  - The search uses residual basins (fingerprinting the failure modes of candidates) and an Upper Confidence Bound (UCB) strategy to guide mutations and repairs, balancing exploitation of good basins with exploration of under-visited ones.
- Dimensional Analysis: Enforced at two levels: local checks on individual operations and a global "Buckingham–Sudoku" solver that propagates dimensional constraints across the entire expression tree to prune infeasible candidates before fitting.
- Multi-Dataset Refinement: The system can fit multiple datasets simultaneously, distinguishing between class-shared parameters (e.g., physical constants) and experiment-specific parameters.

Key Contributions

Derivative-Accurate Decomposition: By utilizing the NestyNet surrogate, the authors achieve significantly higher accuracy in derivative-based separability tests compared to standard MLPs, enabling reliable detection of complex structural motifs.
Factorized Symbolic Search (FSS): A novel search strategy that separates the discovery of internal coordinates from the calibration of outer functions. This avoids the entanglement of structure and constants, allowing the system to score entire families of expressions based on a single skeleton.
Generalized Symmetry Layer: A method to discover internal coordinates via Lie symmetry analysis on the surrogate's gradient field, extending the search beyond standard integer-coefficient motifs.
Statistical Audit and Calibration: The framework includes rigorous statistical selection (using sealed audit sets) and calibrated abstention mechanisms, ensuring that claims about recovered laws are scoped to the domain where certificates hold.

Results

AI Feynman Benchmark (Noiseless): On the SRBench-standardized AI Feynman dataset, NestyNet-SR achieved exact symbolic recovery of all 120 noiseless equations. The authors note this is the first reported result of exact recovery for the full set on this benchmark. The median wall time was 6 minutes per equation on a single CPU core.
AI Feynman Benchmark (Noisy): Under additive Gaussian noise, the system successfully recovered a significant fraction of equations (e.g., ~47% at 10% noise level), with statistical selection methods certifying which structures survive.
Oracle Mode (FSS Isolation): When tested in isolation (without Stage A decomposition or surrogate training) on 115 equations with $\le 6$ variables, the Factorized Symbolic Search recovered 88 equations (76.5%) using exact gradients and GS carrier seeds. This demonstrates the efficacy of the search method itself.
Astrophysical Vignettes:
- Galactic Tidal Radius: The system recovered the exact law for the Jacobi (tidal) radius, $r_J = [\mu/(4\Omega^2 - \kappa^2)]^{1/3}$ , and discovered the anisotropic tidal invariant $4\Omega^2 - \kappa^2$ from mock data.
- SPARC Galaxy Rotation Curves: Given only separate gas and stellar disk acceleration components, the algorithm blindly discovered the baryonic acceleration coordinate ( $z = g_{gas} + \Upsilon_d g_{disk}$ ). It reproduced the established mass-to-light ratio ( $\Upsilon_d \approx 0.56$ ), the radial acceleration relation (RAR), and provided a posterior for the local slope of the law, including the deep-regime slope of $1/2$ .

Significance and Claims
The paper claims that analytic derivatives provide a practical route from neural surrogates to interpretable closed-form empirical laws. The authors emphasize that the Factorized Symbolic Search is a genuine algorithmic advance, particularly in scenarios where constants are deeply nested or where the correct internal coordinate is hidden. They argue that the most effective symbolic regression engine is tiered: exploiting cheap, high-confidence scientific motifs first, and escalating to broader factorized search only when necessary. The work demonstrates that symbolic regression can move beyond recovering static formulas to exposing the meta-structure linking families of empirical laws across different physical regimes. The results on the SPARC data highlight the method's ability to perform "hands-off" theory discovery, reproducing established results while adding new layers of statistical rigor (e.g., held-out generalization and calibrated abstention).

More like this