🤖 machine learning

KAN-SR: A Kolmogorov-Arnold Network Guided Symbolic Regression Framework

This paper introduces KAN-SR, a novel symbolic regression framework leveraging Kolmogorov-Arnold Networks and simplification strategies to accurately recover ground-truth equations from the Feynman dataset and model the dynamics of in-silico bioprocess systems.

Original authors: Marco Andrea Bühler, Gonzalo Guillén-Gosálbez

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marco Andrea Bühler, Gonzalo Guillén-Gosálbez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. You have a pile of clues (data points) showing how different things change over time, but you don't know the rulebook (the mathematical equation) that explains why they change. Your goal isn't just to guess the next clue; it's to find the actual, simple sentence that describes the whole story.

This is the job of Symbolic Regression. Most computer programs today are like "black boxes": they can predict the next clue very well, but they can't tell you the rule. They are like a magician who makes a rabbit appear but won't show you the trick. The authors of this paper want to find the trick.

They introduce a new tool called KAN-SR. Here is how it works, broken down into simple concepts:

1. The New Detective Tool: KANs

Traditional AI models are like a team of workers where everyone does the same job in a rigid way. The authors use a newer type of AI called a Kolmogorov-Arnold Network (KAN).

The Analogy: Imagine a traditional AI is a factory assembly line where every worker does the exact same task. A KAN is more like a team of specialized artisans. Each worker can learn their own unique, flexible skill (a "learnable function") to handle specific parts of the problem. This makes the team much better at understanding complex, curved, or weird patterns in the data.

2. The Strategy: "Divide and Conquer"

The authors realized that trying to solve a giant, complex math puzzle all at once is too hard. So, they built a workflow that breaks the problem down, similar to how you might solve a huge jigsaw puzzle by sorting the pieces by color first.

Step 1: The Quick Scan. They first try to find very simple answers (like basic multiplication or addition). If that works, they stop.
Step 2: The "Magic" Simplification. Before building a complex model, they check if the problem can be split up. For example, does the answer depend on two separate things happening at the same time? If so, they split the problem into two smaller, easier puzzles.
Step 3: The Deep Dive. If the puzzle is still too hard, they use the powerful KAN "artisans" to learn the shape of the curve.
Step 4: Translating Back to Human Language. Once the KAN learns the shape, it translates that shape back into a clean, readable math equation (like $y = x^2 + 5$ ) instead of leaving it as a messy computer code.

3. The Test: The Feynman Challenge

To see if their tool works, they tested it on a famous set of 240 math problems inspired by the physics lectures of Richard Feynman. These problems are designed to be tricky, often containing "dummy variables" (red herrings that look important but aren't) and noisy data (like static on a radio).

The Result: KAN-SR was the champion. It solved 93% of the easy problems and 60% of the medium ones, beating all other methods. Even when the data was noisy or full of fake clues, KAN-SR was the only one that could reliably find the true underlying rule.

4. The Real-World Test: A Virtual Bioreactor

The authors also wanted to see if this could work on moving, changing systems (dynamic systems), not just static pictures. They simulated a bioprocess (a virtual factory where bacteria grow to make a product).

The Challenge: Real-world data is messy. If you try to calculate how fast the bacteria are growing by looking at noisy measurements, the math usually breaks.
The Solution: They used a special technique (Neural Controlled Differential Equations) to smooth out the noise first, creating a clean "movie" of the growth. Then, they fed this clean movie into KAN-SR.
The Result: KAN-SR successfully reconstructed the equations that govern the bacteria's growth. While it missed one tiny detail about how the bacteria stop growing when they get too crowded, it captured the main story perfectly. The model could predict the future growth almost exactly as well as the original complex simulation.

The Bottom Line

The paper claims that KAN-SR is a powerful new way to discover the "laws of nature" hidden in data.

It is smarter at ignoring fake clues.
It is better at finding simple, human-readable equations than previous methods.
It can handle noisy, real-world data by smoothing it out first.

A Note on Limits: The authors are honest about the tool's boundaries. It works best when the answer can be found within a specific set of mathematical building blocks they provided. If the true answer is something completely outside their "library" of shapes, or if the data is missing key information entirely, the tool might struggle. It is a very strong detective, but it still needs the clues to be there.

Technical Summary: KAN-SR

Problem Statement
Symbolic Regression (SR) aims to discover closed-form mathematical expressions that accurately model a given dataset, inferring both the functional form and its parameters. Unlike traditional regression with fixed structures or black-box machine learning models, SR seeks interpretable, concise relationships essential for scientific discovery. However, classical methods like Genetic Programming (GP) often suffer from sample inefficiency, high computational costs, and scalability issues. Recent neural-symbolic approaches struggle with end-to-end training of discrete representations, and a lack of universal benchmarks hinders consistent comparison. Furthermore, applying SR to dynamic engineering systems is challenging due to the sensitivity of derivative estimation to noise in time-series data.

Methodology: The KAN-SR Framework
The authors propose KAN-SR, a framework built upon Kolmogorov-Arnold Networks (KANs) [22, 23] combined with a divide-and-conquer strategy and symbolic simplification.

Network Architecture:
- KANs: Unlike standard neural networks with fixed activation functions, KANs employ learnable univariate functions. The authors implement Fast-KAN using Radial Basis Functions (specifically a Reflectional Switch Activation Function, RSWAF) instead of B-splines to improve training speed and efficiency.
- Hybrid Composition: To overcome the limitation of purely additive KANs in representing multiplicative interactions, the framework explicitly allows both additive and multiplicative combinations of learnable activations within single-layer units.
- Regularization: To enforce sparsity and interpretability, the model utilizes a regularization scheme penalizing unnecessary input usage via magnitude ( $L_1$ ) and entropy losses on activation matrices, alongside an $L_1$ penalty on base linear weights.
Workflow and Pipeline:
The framework operates through a flexible, multi-stage pipeline (see Figure 1 in the paper):
- Preprocessing: Normalization of input variables.
- Brute-Force Search: An initial exhaustive search for simple multivariate multiplicative expressions.
- Simplification and Decomposition: Inspired by AI Feynman [15], the system recursively detects separability (additive or multiplicative) and symmetries (translation, scaling, etc.) to decompose complex problems into lower-dimensional subproblems.
- Single-Layer KAN Fitting: The decomposed subproblems are fitted using minimal KAN architectures (single or multiple units).
- Symbolic Extraction: Trained univariate activations are approximated by functions from a predefined, extendable library (polynomials, trigonometric, exponential, logarithmic, Michaelis-Menten, etc.). The best fit is selected based on Mean Squared Error (MSE) or a score balancing MSE and complexity.
- Output Transformation: If fitting fails, the target variable is transformed (e.g., logarithmic) to simplify the functional form before re-attempting the search.
- Deep KAN Loop: As a fallback, deeper KAN architectures are explored if simpler models fail, though the primary experiments focus on the efficacy of the simplification pipeline without this step.
Dynamic System Extension (Neural CDEs):
For dynamic systems, the framework integrates Neural Controlled Differential Equations (Neural CDEs) [36, 28]. A Neural CDE is trained on noisy time-series data to estimate smooth derivatives. These robust derivative estimates serve as the target for the symbolic regression step, allowing the discovery of governing differential equations (kinetic rate laws) despite noisy observations.

Key Contributions

Framework Proposal: Introduction of KAN-SR, unifying compositional deep learning (KANs) with symbolic extraction and simplification.
Symbolic Pipeline: Development of a symbolic extraction and simplification pipeline inspired by AI Feynman, utilizing a comprehensive univariate function library.
Benchmark Evaluation: Comprehensive evaluation on the SRSD-Feynman dataset [16], demonstrating improvements in symbolic recovery rates.
Dynamic Modeling: Extension of the method to dynamic systems via a bioprocess case study, leveraging Neural CDEs for noise-robust derivative estimation.

Experimental Results

SRSD-Feynman Benchmark: KAN-SR achieved the highest solution rates across all difficulty levels (Easy, Medium, Hard) compared to state-of-the-art baselines (gplearn, AFP, AIF, DSR, PySR, etc.).
- Easy: 93.3% solution rate (vs. 60.0% for the next best).
- Medium: 60.0% (vs. 30.0% for PySR).
- Hard: 12.0% (vs. 4.0% for PySR).
- Robustness: The framework maintained high performance (86.7% on Easy) even when dummy (irrelevant) variables were introduced, significantly outperforming other methods. It also demonstrated robustness to Gaussian noise, solving over 50% of Easy problems with 0.1 noise levels.
Bioprocess Case Study: In a simulated batch fermentation model, the two-step approach (Neural CDE for derivatives + KAN-SR for equations) successfully recovered kinetic rate equations structurally similar to the ground truth. While the biomass-dependent self-inhibition term was not fully recovered, the resulting symbolic model reproduced system dynamics with high fidelity in parity plots.

Significance and Claims
The paper claims that KAN-SR offers a significant advancement in symbolic regression by effectively combining the expressivity of deep learning with the interpretability of symbolic forms. The "divide-and-conquer" approach, utilizing separability and symmetry detection, allows the framework to solve complex problems that often defeat other methods, particularly in the presence of irrelevant variables and noise.

The authors position KAN-SR as a tool capable of recovering ground-truth equations in scientific domains, specifically highlighting its potential for modeling dynamic engineering systems like bioprocesses. They note that while the method excels when ground truth is recoverable from training variables, it faces limitations if the optimal solution lies outside the constrained search space defined by the single-layer KAN architecture and the specific univariate function library. The framework is presented as a step toward more robust, interpretable dynamic modeling, though the authors acknowledge that broader validation on real-world time-series systems is required to fully assess generalizability.