🤖 machine learning

Symbolic--KAN: Kolmogorov-Arnold Networks with Discrete Symbolic Structure for Interpretable Learning

This paper introduces Symbolic-KAN, a novel neural architecture that integrates discrete symbolic structures into trainable deep networks to bridge the gap between scalable learning and interpretability, enabling the direct discovery of compact, closed-form governing equations for regression and physics-informed problems without requiring post-hoc symbolic fitting.

Original authors: Salah A Faroughi, Farinaz Mostajeran, Amirhossein Arzani, Shirko Faroughi

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Salah A Faroughi, Farinaz Mostajeran, Amirhossein Arzani, Shirko Faroughi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand the laws of physics, like how a pendulum swings or how heat spreads through a metal rod.

The Problem: The "Black Box" vs. The "Search Engine"
Currently, scientists have two main ways to do this, and both have flaws:

The "Black Box" (Neural Networks): These are like super-smart but mute geniuses. They can predict the future with incredible accuracy, but if you ask them how they did it, they just say, "I just know." They give you a massive, unreadable formula that no human can understand.
The "Search Engine" (Symbolic Regression): These are like librarians who try to find the perfect sentence by rearranging every word in the dictionary. They produce beautiful, simple, readable formulas (like $E=mc^2$ ), but they are incredibly slow and often get stuck trying to find the right combination in a massive library.

The Solution: Symbolic-KAN
The authors of this paper introduce Symbolic-KAN, a new type of AI that is the best of both worlds. Think of it as a smart, self-editing chef.

The Analogy: The Self-Editing Chef

Imagine you are training a chef to recreate a complex dish (the physical law) just by tasting the ingredients (the data).

1. The Kitchen Setup (The Architecture)
Traditional AI chefs (standard Neural Networks) just throw everything into a blender. They mix thousands of ingredients in a way that tastes good, but you can't tell what's in the soup.
Symbolic-KAN is different. It has a special pantry (a library of basic ingredients like salt, sugar, sine waves, and polynomials). Instead of blending everything, it builds the dish layer by layer, choosing specific ingredients to mix.

2. The "Gating" Mechanism (The Tasting Spoon)
Here is the magic trick. At first, the chef is unsure. They might think, "Maybe I need a little bit of sugar, a little bit of salt, and a tiny bit of chili." They mix them all together in a soft, blurry way. This is the training phase.

But as the chef tastes the dish and compares it to the original, a special "Gating System" kicks in. It's like a strict editor who says:

"Stop guessing. You don't need the chili. You don't need the salt. You only need sugar."
"And for this layer, you don't need the 'maybe' mixture. You need exactly the sugar."

This editor forces the chef to make hard choices. It turns the blurry "maybe" into a sharp "yes." This is called discretization.

3. The Result: A Simple Recipe
Once the training is done, the chef doesn't give you a 50-page unreadable manual. Instead, they hand you a simple, 3-line recipe:

"Take the input, apply a sine wave, then multiply by 2."

Because the AI was forced to pick from a library of simple, known mathematical shapes (like sine, cosine, or $x^2$ ) and choose only the ones that really mattered, the final result is a compact, human-readable equation.

Why is this a big deal?

It's Fast: Unlike the "Search Engine" method that tries billions of combinations, Symbolic-KAN learns efficiently like a modern AI, but it forces itself to stay simple.
It's Honest: It doesn't just predict the answer; it tells you the rule. If you ask, "Why did the pendulum swing that way?" it can show you the exact mathematical formula.
It's a "Pre-Selector": Imagine you are a detective looking for a suspect. Symbolic-KAN is like a smart filter that narrows down a list of 10,000 suspects to just 3 likely ones. Scientists can then use those 3 clues to solve the rest of the mystery much faster.

Real-World Examples in the Paper

The authors tested this "Self-Editing Chef" on three types of problems:

Simple Math: It looked at scattered dots on a graph and instantly realized, "Ah, this is just a parabola ( $x^2$ )." It ignored all the other complex ingredients it didn't need.
Oscillating Systems: It watched a bouncing ball (the Van der Pol oscillator) and figured out the exact equation governing its motion, even when the math got weird with non-integer powers.
Physics Laws: It learned how heat diffuses or how water flows by looking at the data and the rules of physics, producing simple formulas that matched the complex reality perfectly.

The Bottom Line

Symbolic-KAN is a bridge. It connects the raw power of deep learning (which is great at finding patterns) with the clarity of human logic (which is great at understanding them). It takes the "black box" and turns it into a transparent glass box, showing us not just what will happen, but why it happens, in a language we can actually read.

1. Problem Statement

Scientific Machine Learning (SciML) faces a fundamental trade-off between scalability and interpretability.

Neural Networks (e.g., MLPs, standard KANs): Scale efficiently with data and dimensionality but produce "opaque" representations that are difficult to interpret or extract governing laws from.
Classical Symbolic Regression (e.g., Genetic Algorithms, SINDy): Yield explicit analytic expressions but often suffer from high computational costs, combinatorial search limitations, and an inability to discover new functional combinations not explicitly present in a pre-defined library.
The Gap: Existing methods struggle to simultaneously learn complex, high-dimensional mappings from data while automatically discovering the underlying compact, human-readable symbolic equations without requiring post-hoc fitting or large candidate libraries.

2. Methodology: Symbolic-KAN

The authors propose Symbolic-KAN, a neural architecture that bridges this gap by embedding discrete symbolic structure directly within a trainable deep network. The framework is built upon the Kolmogorov-Arnold Representation Theorem (KART) but introduces a novel, learnable decomposition.

Core Architectural Innovations

Learned Scalar Projections: Unlike standard KANs that use fixed coordinate-wise univariate maps, Symbolic-KAN learns scalar projections ( $s_k = w^T h + b$ ) of upstream features. This allows the network to discover meaningful linear combinations of inputs before applying non-linearities.
Dynamic Primitive Library: Instead of fixed basis functions (like B-splines), each edge in the network selects from a diverse library of analytic primitives ( $P = \{0, 1, x, x^2, \sin, \cos, \tanh, e^x, \dots\}$ ).
Hierarchical Gating Mechanism: The network employs three levels of gating to transform a continuous mixture model into a discrete symbolic structure:
1. Primitive-Selection Gates: Use Gumbel-Softmax with a decaying temperature parameter ( $\tau$ ) to select a single analytic primitive per edge. Entropy regularization encourages "one-hot" selection.
2. Edge-Selection Masks: Selects a single scalar projection (edge) per unit, ensuring sparsity in the input connections.
3. Unit-Level Gating: Prunes inactive hidden units to further compress the architecture.
Training Pipeline:
- Stage I (Relaxed Training): Optimizes continuous gate parameters and soft selections. The temperature $\tau$ is annealed from high to low to sharpen the distributions.
- Stage II (Hardening): Converts soft gates to hard "one-hot" selections (discrete structure). Continuous parameters are refined using second-order optimizers (e.g., L-BFGS).
Loss Function: Combines data fidelity (or PDE residuals) with symbolic regularization terms:
- Entropy Loss: Penalizes diffuse primitive mixtures.
- Non-Maximum Suppression (NMS): Prevents different edges of the same unit from selecting identical primitives.
- Unit Sparsity: Encourages a target density of active units.

3. Key Contributions

Unified Framework: Introduces a single architecture that performs both scalable function approximation and automatic symbolic discovery, eliminating the need for separate post-hoc symbolic regression steps.
Scalable Primitive Discovery: The method acts as a "pre-library selector," identifying the most relevant analytic components from a broad library. These discovered primitives can subsequently inform sparse equation-learning methods (like SINDy), addressing the limitation of SINDy requiring prior knowledge of the functional form.
Discrete-to-Continuous Transition: Successfully bridges the gap between differentiable optimization and discrete symbolic logic through a differentiable relaxation (Gumbel-Softmax) that converges to a hard symbolic structure.
Physics-Informed Learning: Extends the framework to solve forward and inverse problems for Partial Differential Equations (PDEs), producing solutions that are not only accurate but also possess symbolic representations reflecting the true analytical structure of the governing equations.

4. Experimental Results

The authors evaluated Symbolic-KAN across three paradigms:

A. Data-Driven Regression

Task: Recovering multivariate functions from scattered data.
Results: Successfully recovered correct primitives (e.g., $x^2$ , $\sin$ , $\cos$ ) and compact closed-form expressions for 1D functions. The model demonstrated strong extrapolation capabilities, maintaining accuracy beyond the training domain.

B. Inverse Dynamical Systems (Van der Pol Oscillator)

Task: Identifying governing equations and unknown parameters (including non-integer exponents like $x^{2.15}$ ) from time-series data.
Results:
- Accurately recovered the dynamical structure and parameters ( $a, \mu, c$ ) with errors $< 1\%$ .
- Demonstrated robustness over long time horizons ( $T=50$ ) and varying data densities.
- Selected primitives ( $\sin, \cos, x$ ) aligned perfectly with the oscillatory nature of the system.

C. Physics-Informed Learning (PDEs)

Tasks: Reaction-Diffusion equation (inverse problem) and Laplace equation (forward problem).
Results:
- Accuracy: Outperformed baseline models (cPIKAN and standard PINN) significantly. For the Reaction-Diffusion problem on an extended domain $[-4, 4]$ , Symbolic-KAN reduced validation error by ~95% compared to cPIKAN and ~99% compared to PINN.
- Parameter Recovery: Estimated coefficients (e.g., reaction rate $\kappa$ ) with much higher precision than baselines.
- Symbolic Discovery: The hardened networks selected primitives ( $\sin, \cos, \sinh, \cosh$ ) that matched the functional families of the true analytical solutions, even without being explicitly told the solution form.

5. Significance and Impact

Mechanistic Insight: Symbolic-KAN moves beyond "black-box" prediction to provide mechanistic insight, revealing the actual mathematical operators governing a system.
Scalability: It offers a path to interpretable learning that scales with data and dimensionality, overcoming the combinatorial explosion of traditional symbolic regression.
Foundation for Future SciML: By acting as a scalable mechanism to discover informative primitives, it lays the groundwork for a new class of "neural solvers" that are both robust and amenable to symbolic inspection. This is crucial for applications in engineering, physics, and biology where understanding why a model works is as important as its accuracy.

In summary, Symbolic-KAN represents a significant step forward in Scientific Machine Learning by unifying the expressivity of deep learning with the transparency of symbolic mathematics, enabling the direct discovery of governing laws from data.