📊 statistics

SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Exploration

This paper introduces SPINEX_SymbolicRegression, a novel similarity-based symbolic regression algorithm that leverages explainable neighbor exploration to identify high-merit expressions, demonstrating competitive accuracy and structural similarity against leading methods across over 180 benchmark functions.

Original authors: MZ Naser, Ahmed Z Naser

Published 2026-08-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: MZ Naser, Ahmed Z Naser

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of modern science, data is everywhere. Sensors on bridges, telescopes in space, and instruments in laboratories generate endless streams of numbers that describe how the world works. For decades, the standard way to make sense of this data has been to use machine learning models that act like powerful black boxes. These systems are excellent at predicting what will happen next; they can forecast a storm or identify a disease with high accuracy. However, they often fail to explain why they made that prediction. They offer a result without a reason, leaving scientists with a correct answer but no understanding of the underlying mechanism.

This is where a different approach, known as symbolic regression, comes into play. Unlike traditional methods that force data into a pre-set shape, symbolic regression acts more like a curious explorer. It searches for the actual mathematical sentence that describes the relationship between variables. Instead of just guessing a number, it tries to write the equation itself, discovering the formula that governs the data. The goal is not just to predict, but to reveal the hidden laws of nature in a way that a human can read, understand, and verify. This ability to produce transparent, explainable models is crucial for fields like physics and engineering, where knowing the "why" is just as important as knowing the "what."

Building on this need for clarity, a team of researchers has developed a new algorithm called SPINEX_SymbolicRegression. This tool is designed to find these mathematical sentences more effectively by using a concept called similarity. Imagine the algorithm is searching for a specific type of tree in a vast forest. Instead of looking at every single tree in isolation, it compares them to one another. It asks, "Does this new tree look like the ones that are already doing well?" By focusing on expressions that are structurally similar to promising candidates, the algorithm can navigate the search space more intelligently. It doesn't just look for the answer that fits the numbers best; it also looks for answers that share the same building blocks—like the same variables and operations—as the most successful models it has found so far.

The researchers tested this new method against a massive collection of 182 different mathematical problems. These problems ranged from simple, randomly generated equations to complex formulas derived from real-world physical phenomena, such as the laws of motion and thermodynamics. They pitted their new algorithm against PySR, a leading tool in the field, running thousands of simulations to see which one could find the correct mathematical expression more often. The results showed a distinct advantage for the new approach in specific areas. While the established tool sometimes found answers that were slightly more accurate in terms of raw prediction, the new algorithm was far superior at finding the right ingredients. It was much more likely to identify the correct set of variables and the correct mathematical operations needed to describe the system. In many cases, when the researchers required the algorithm to use every available variable, the new method achieved a perfect match, finding the exact structure of the true equation far more often than its competitor.

A key feature of this work is its focus on explainability. The researchers did not just want a black box that worked; they wanted to see how the algorithm thought. They included tools that allow users to see exactly how the algorithm evolved its solutions, which variables it considered important, and how similar its best guesses were to the true underlying laws. In one demonstration, the algorithm was asked to find the formula for a simple curve. It successfully evolved an expression that closely matched the target, and the system provided a detailed breakdown of why certain parts of the formula worked better than others. This level of transparency allows scientists to trust the model, not just because it predicts well, but because they can see the logic behind it.

The study concludes that this similarity-based approach offers a powerful new way to uncover mathematical relationships. By balancing the need for accuracy with the need for structural similarity, the algorithm consistently produced models that were both precise and understandable. The researchers noted that while the method is highly effective, it does require significant computing power, and there is still room to improve how quickly it can calculate these similarities. However, the findings suggest that by guiding the search with the principle of similarity, scientists can generate models that are not only accurate but also aligned with the fundamental principles of the physical world. This brings the field of machine learning one step closer to a future where computers can not only predict the future but also explain the rules that govern it.

Technical Summary: SPINEX_SymbolicRegression

Problem Statement
Symbolic regression (SR) aims to autonomously discover the mathematical form and parameters of expressions that best describe data, offering a transparent alternative to "black-box" machine learning models like deep neural networks. While traditional SR methods, such as Genetic Programming (GP), utilize evolutionary algorithms to navigate the search space of potential expressions, they often struggle with balancing accuracy, structural complexity, and interpretability. Furthermore, existing approaches may not fully leverage the concept of similarity to guide the search toward meaningful, generalizable models or to ensure that discovered expressions align with known physical principles or domain-specific constraints. This study addresses the need for an SR algorithm that effectively integrates similarity-based navigation with explainable neighbor exploration to enhance robustness and model fidelity.

Methodology
The paper introduces SPINEX_SymbolicRegression, a novel algorithm extending the SPINEX (Similarity-based Predictions with Explainable Neighbors Exploration) framework. The methodology combines genetic programming with a similarity-driven approach to identify high-merit expressions. Key technical components include:

Similarity and Diversity Management: The algorithm utilizes tree edit distance metrics to quantify structural similarity between expressions. It employs a similarity_threshold to maintain population diversity and prevent premature convergence. A dynamic_similarity_threshold mechanism adjusts this threshold during evolution based on population diversity levels.
Explainable Neighbors Exploration: The algorithm tracks "neighbors" of candidate expressions to analyze local optima. It provides detailed breakdowns of evolved terms, including fitness scores, Mean Squared Error (MSE) contributions, correlation metrics, and similarity measures (Cosine and Euclidean) relative to the target function.
Variable and Operator Selection: The system assesses variable relevance using statistical measures (correlation and mutual information) to guide inclusion strategies (guided or probabilistic). It supports a force_all_variables option to ensure all input features are represented, which is critical for domains where specific variables are theoretically essential.
User Guidance and Constraints: Users can inject user_expression (domain knowledge) directly into the initial population or as constraints, guiding the evolutionary process toward established theoretical frameworks.
Adaptive Evolutionary Mechanisms: The algorithm features dynamic elite selection (dynamic_elite), adaptive complexity penalties to prevent overfitting, and robust handling of mathematical operations (e.g., safe division, safe logarithms) to ensure numerical stability.
Benchmarking Protocol: The algorithm was evaluated against 182 mathematical benchmarking functions across six problem sets: Feynman (Easy, Medium, Hard), Keijzer, Korns, Livermore, Nguyen, and R rationals. Performance was compared against PySR, a leading SR algorithm, using metrics including the Coefficient of Determination ( $R^2$ ), Exact Variable Set Match (VM), and Exact Operation Set Match (OP).

Key Contributions

Algorithmic Innovation: The proposal of SPINEX_SymbolicRegression, which uniquely integrates similarity-based search navigation with explainable neighbor analysis to enhance the discovery of structurally accurate models.
Comprehensive Benchmarking: Extensive testing across 182 diverse functions, ranging from randomly generated expressions to equations derived from real physical phenomena (e.g., Feynman Lectures).
Explainability Framework: The development of a mechanism that not only outputs the final expression but also provides granular insights into the evolutionary process, variable importance, and the performance of neighboring expressions in the search space.
Performance Validation: Demonstration that the algorithm can match or exceed the performance of leading algorithms (specifically PySR) in specific configurations, particularly regarding structural fidelity (variable and operator matching).

Results
The benchmarking results indicate that SPINEX_SymbolicRegression performs competitively across the tested problem sets:

Accuracy ( $R^2$ ): In terms of raw predictive accuracy ( $R^2$ ), PySR generally led in most problem instances, though SPINEX outperformed it in two specific instances.
Structural Fidelity: SPINEX demonstrated a significant advantage in Exact Variable Set Match (VM) and Exact Operation Set Match (OP). When configured to force_all_variables=True, SPINEX achieved up to 100% exact variable matching in several configurations, significantly outperforming PySR. This suggests SPINEX is particularly effective at recovering the correct structural components of the underlying equations.
Configuration Impact: Performance generally improved with larger population sizes (100 vs. 50) and increased generations (1000 vs. 100). The inclusion of all variables (force_all_variables=True) notably enhanced the structural accuracy of the derived expressions.
Case Study: In a test on a quadratic function ( $y=2X^2+3X+5$ ), the algorithm successfully identified a local optimum ( $X^2 + 10X$ ) with a fitness of 0.9197, outperforming its neighbors, though the paper notes further refinement may be needed to capture the exact coefficients.

Significance and Claims
The authors claim that SPINEX_SymbolicRegression offers a robust alternative for symbolic regression by capitalizing on the concept of similarity as a driving force. The paper posits that leveraging similarity measures allows the algorithm to more effectively navigate the search space, identify relevant patterns, and ensure the generation of meaningful models that adhere to structural constraints.

The significance of this work lies in its ability to produce explicit, transparent, and structurally accurate mathematical descriptions of data. By prioritizing structural similarity (matching variables and operators) alongside predictive accuracy, the algorithm addresses the rising demand for interpretability in science and engineering. The authors highlight that this approach is particularly valuable for rediscovering fundamental laws, validating theoretical models, and ensuring that generated models align with established scientific principles. While the paper acknowledges limitations regarding computational intensity and the need for further optimization of similarity metric calculations, it asserts that the proposed method consistently delivers competitive performance and, in specific structural contexts, outperforms leading existing algorithms.

Technical Summary: SPINEX_SymbolicRegression

More like this