🤖 machine learning

Neural Discovery of Memory and Nonlocal Kernels in Integro-Differential Equations with Constrained Kolmogorov--Arnold Networks

This paper proposes a differentiable-solver framework using constrained Kolmogorov--Arnold Networks (specifically hard-constrained MC-KAN and soft-penalty Cheb-KAN) to robustly discover interpretable memory and nonlocal kernels in integro-differential equations from sparse and noisy observations, demonstrating that enforcing physical shape constraints by construction yields superior accuracy over soft penalties, particularly for multidimensional problems.

Original authors: Aruzhan Tleubek, Salah A Faroughi

Published 2026-07-14

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Aruzhan Tleubek, Salah A Faroughi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out the "ghost rules" that govern a mysterious machine. You can't see the machine's internal gears, but you can watch how it moves when you push it. In the world of physics, many systems (like rubber bands that slowly snap back, or heat spreading through a material with a memory) don't just react to what's happening right now. They remember the past. They have a "fading memory."

Mathematicians describe this memory using a special ingredient called a kernel. Think of the kernel as a recipe card that tells the system exactly how much weight to give to every moment in its past. If you know the recipe, you can predict the future perfectly. But usually, the recipe card is missing! The scientists have the machine's movements (the data), but they don't know the recipe. This is a tricky puzzle because many different recipes could produce the same movement, especially if your observations are blurry or incomplete.

The Old Way vs. The New Way

For a long time, scientists tried to solve this by doing complex math derivations specific to each problem, or by guessing a recipe and hoping it fit. The authors of this paper say, "Let's try something different." They built a neural network (a type of AI brain) that acts as a detective. Instead of guessing, this AI brain learns the recipe directly from the messy, noisy data of how the system moves.

But here's the catch: if you just let a standard AI brain guess, it might come up with a recipe that makes no physical sense. For example, it might suggest that the memory gets stronger the older the event is, or that the influence of the past suddenly becomes negative. In the real world, memory usually fades away smoothly and never flips signs.

The Two Detective Strategies

To stop the AI from making nonsense guesses, the researchers tried two different ways to force the AI to follow the rules of physics.

Strategy 1: The "Soft" Penalty (The Cheb-KAN)
Imagine you are training a dog. You tell it, "Don't jump on the couch," and if it does, you give it a gentle "no" (a soft penalty). The dog might listen, but it might also jump anyway if it really wants to. This is what the Cheb-KAN does. It tries to learn the recipe, but if the recipe breaks the rules (like becoming negative or curving the wrong way), the computer adds a small "punishment" score to the AI's homework. The AI tries to minimize the punishment, but it's not strictly forbidden from breaking the rules.

Strategy 2: The "Hard" Constraint (The MC-KAN)
Now, imagine you build a cage around the dog. The dog physically cannot jump on the couch because the cage is there. This is the MC-KAN (Monotone–Convex Kolmogorov–Arnold Network). The researchers didn't just ask the AI to be good; they built the AI's brain out of special mathematical blocks (Bernstein polynomials) that make it impossible for the AI to generate a recipe that breaks the rules. The AI is forced by its very architecture to only produce recipes that are positive, always fading away, and curving the right way. It's not a suggestion; it's a law of the machine.

The Big Test: Can the AI Find the Recipe?

The team put both strategies to the test in three different scenarios, ranging from simple to very complex.

1. The Simple Memory (1D Volterra Equation)
They started with a simple system where the memory fades like a standard exponential curve (like a cup of coffee cooling down). They gave the AI data with different levels of "noise" (static or fuzziness).

The Result: Both strategies found the correct recipe. Even when the data was very noisy, both got it right. However, the Hard Constraint (MC-KAN) was slightly more precise, finding the recipe with about 13% to 54% less error than the Soft Penalty version, depending on how much noise was in the data. It was like the caged dog never made a single mistake, while the dog with the gentle "no" made a few tiny slips.

2. The Stretchy Memory (1D Viscoelastic Wave)
Next, they looked at a rubbery rod vibrating. The memory here wasn't a simple curve; it was a "stretched exponential," which is a more complex shape often found in glassy polymers. They made the data very sparse, meaning they only gave the AI a few snapshots of the rod's movement (as few as 11 snapshots out of a possible 1001).

The Result: Amazingly, both methods could still find the recipe even with so little data. But again, the Hard Constraint (MC-KAN) was more consistent. It found the recipe with about 40% less error than the Soft Penalty version. The Soft Penalty version sometimes wobbled a bit, while the Hard Constraint version stayed steady.

3. The 2D Chaos (Nonlocal Reaction-Diffusion)
Finally, they tackled the hardest challenge: a 2D system where the memory depends on space in two directions (like a pattern forming on a surface). The data was sparse (only 7 time snapshots on a 32x32 grid) and noisy (up to 0.15 standard deviation of noise).

The Result: This is where the difference became huge. The Soft Penalty strategy started to hallucinate. It produced recipes that had weird ripples and didn't fade correctly. The Hard Constraint (MC-KAN) stayed rock solid.
- At the highest noise level (σ = 0.15), the Hard Constraint method had a kernel error of 12.36%, while the Soft Penalty method had an error of 21.95%. The Hard Constraint was nearly twice as accurate.
- The Soft Penalty method even produced "non-admissible" forms (recipes that looked like power laws instead of the correct exponential shapes) when the noise was high. The Hard Constraint method never broke the rules.

The Final Verdict

The paper concludes that while both methods work well when the data is clean and simple, enforcing the rules by building them into the AI's structure (Hard Constraints) is much more robust when the data is messy, sparse, or high-dimensional.

The authors found that for the most difficult 2D problem with noise, the Hard Constraint method reduced the error by roughly 44% compared to the Soft Penalty method. They also successfully used a tool called PySR to translate the AI's learned "black box" recipe back into a clear, readable math formula (like $e^{-0.99\tau}$ ), proving that the AI didn't just guess numbers; it actually learned the underlying physical law.

What the paper says is NOT true:

The paper does not claim this works for every type of memory. It explicitly rules out kernels that wiggle (oscillate), change signs, or don't fade away. The Hard Constraint method is built specifically for "fading memory" that is positive and smooth.
The paper does not say this is a magic bullet for all inverse problems. It admits that for the 2D problem, they had to break the recipe into pieces to find the full formula, and the coupling between the two directions wasn't perfectly recovered in one go.
The results are based on simulations (computer-generated data), not real-world experiments yet. The authors suggest this is a next step, but the current proof is in the numbers they generated.

In short: If you want to find the secret recipe of a fading memory from messy data, building a cage around your AI (Hard Constraints) works much better than just asking it nicely to behave (Soft Penalties).

Technical Summary: Neural Discovery of Memory and Nonlocal Kernels in Integro-Differential Equations with Constrained Kolmogorov–Arnold Networks

1. Problem Statement

The paper addresses the ill-posed inverse problem of identifying unknown memory or nonlocal kernels governing integro-differential equations (IDEs) from sparse and noisy spatiotemporal observations. IDEs model physical systems with memory effects (e.g., viscoelastic relaxation, anomalous diffusion) or spatial nonlocality (e.g., peridynamics, nonlocal reaction-diffusion), where the state depends on historical evolution or distant spatial interactions via a convolution kernel $K$ .

Existing identification methods face significant limitations:

Classical approaches rely on problem-specific analytical derivations, specialized observation requirements (e.g., boundary flux measurements), or restrictive assumptions to guarantee well-posedness.
Current Scientific Machine Learning (SciML) methods often represent kernels implicitly within neural network parameters, lacking closed-form interpretability, or require case-specific adjoint derivations for gradient computation. Furthermore, many methods struggle to enforce physical constraints (positivity, monotonicity, convexity) necessary for stable kernel recovery in ill-posed settings.

The goal is to develop a flexible, data-driven framework that directly recovers the kernel's functional form from observations while strictly adhering to physical shape constraints, without requiring problem-specific analytical inversion formulas.

2. Methodology

The authors propose a differentiable-solver-based framework that embeds a trainable neural kernel representation within a numerical solver for the IDE. The framework follows a "discretize-then-optimize" strategy, computing gradients via automatic differentiation through the solver, thereby avoiding the need for case-specific adjoint equations.

2.1. Neural Kernel Parameterization

The unknown kernel $K(\tau)$ (where $\tau$ represents time and/or spatial lags) is parameterized using Kolmogorov–Arnold Networks (KANs). Two specific architectures are compared:

Soft-Constrained Chebyshev KAN (Cheb-KAN):
- Uses Chebyshev polynomial bases for edge functions.
- Physical properties (positivity, monotonic decrease, convexity) are encouraged via soft penalty terms added to the loss function.
- The loss function combines data mismatch with penalties for negative values, positive first derivatives, and negative second derivatives.
Hard-Constrained Monotone–Convex KAN (MC-KAN):
- Uses Bernstein polynomial bases for edge functions.
- Physical properties are enforced by construction through constraints on the polynomial coefficients.
- Mechanism: By constraining the Bernstein coefficients to satisfy specific ordering and convexity conditions (e.g., $c_0 \ge c_1 \ge \dots \ge c_N$ for monotonic decrease), the resulting function is guaranteed to be positive, monotonically decreasing, and convex within the domain $[0, 1]$ .
- The architecture employs a specific layer ordering: the first layer uses decreasing convex edge functions, while subsequent layers use increasing convex edge functions. This composition ensures the final kernel satisfies the required physical constraints regardless of the learned coefficients.

2.2. Training and Recovery

Training: The neural kernel is embedded in a forward solver (using quadrature or FFT for integrals). Parameters are optimized to minimize the mismatch between predicted and observed spatiotemporal data.
Symbolic Regression: After training, the learned neural kernel is sampled, and symbolic regression (via PySR) is applied to extract an interpretable, closed-form mathematical expression (e.g., exponential or power-law forms).

3. Key Contributions

Differentiable Solver Framework: A general framework for kernel discovery that integrates neural representations directly into numerical solvers, eliminating the need for case-specific adjoint derivations.
Hard-Constrained KAN Architecture: The introduction of MC-KAN, which uses Bernstein polynomials to guarantee physical constraints (positivity, monotonicity, convexity) by construction, rather than relying on soft penalties.
Comparative Analysis: A systematic evaluation of hard-constrained (MC-KAN) vs. soft-constrained (Cheb-KAN) approaches across varying levels of data sparsity, noise, and dimensionality.
Interpretability: The successful integration of symbolic regression to convert learned neural kernels into closed-form constitutive laws.

4. Experimental Results

The framework was validated on three benchmark problems:

Experiment I: 1D Volterra IDE (Temporal Memory)

Task: Recover an exponential memory kernel from noisy time-series data.
Results: Both methods successfully identified the exponential form. However, MC-KAN consistently achieved lower kernel reconstruction errors than Cheb-KAN across all noise levels and network depths.
Observation: While both methods recovered the correct analytical form ( $K(\tau) = e^{-r\tau}$ ) after symbolic regression, the hard-constrained MC-KAN provided more accurate numerical approximations, particularly at higher noise levels.

Experiment II: 1D Viscoelastic Wave PIDE (Temporal Memory)

Task: Recover a stretched-exponential (KWW) kernel from sparse temporal snapshots of a viscoelastic rod's vibration.
Results: Both methods were robust to data sparsity (down to 1% of the full grid). MC-KAN achieved ~40% lower kernel reconstruction errors than Cheb-KAN across all sparsity levels.
Observation: MC-KAN reconstructions remained strictly positive, monotone, and convex, whereas Cheb-KAN exhibited slight oscillatory artifacts. Both methods successfully recovered the true KWW parameters ( $\tau_0, \beta$ ) via symbolic regression.

Experiment III: 2D Nonlocal Reaction-Diffusion Equation (Spatial Nonlocality)

Task: Recover a complex, anisotropic, 2D spatial interaction kernel from sparse and noisy data.
Results:
- In the noise-free case, Cheb-KAN performed marginally better.
- Under noisy conditions ( $\sigma \ge 0.02$ ), MC-KAN significantly outperformed Cheb-KAN. At $\sigma = 0.15$ , MC-KAN reduced kernel error by nearly a factor of two compared to Cheb-KAN.
- Robustness: The hard constraints in MC-KAN prevented nonphysical oscillations that appeared in Cheb-KAN reconstructions when data was sparse and noisy.
- Normalization: A power-law input normalization (concentrating resolution near the origin) further improved MC-KAN performance for kernels with sharp gradients.
Symbolic Recovery: While 2D coupled forms were difficult to recover directly, symbolic regression successfully identified the separable 1D slices and the coupling structure (product form) in the log-space.

5. Significance and Claims

The paper claims that enforcing physically motivated shape constraints by construction (via MC-KAN) provides greater robustness than soft penalties for multidimensional kernel discovery, particularly when observations are sparse and noisy.

Robustness: The primary benefit of hard constraints is not necessarily superior performance in well-observed, simple inverse problems, but rather the ability to maintain physically consistent and accurate representations as data quality degrades.
Generalizability: The framework is compatible with standard time integrators and spatial discretizations, making it applicable to a broad class of IDEs without requiring custom adjoint derivations.
Limitations: The authors acknowledge that the current MC-KAN is restricted to positive, monotone, and convex kernels (excluding oscillatory or sign-changing kernels). Additionally, symbolic regression for fully coupled multidimensional kernels remains challenging, often requiring staged recovery of separable components.

In summary, the work demonstrates that combining differentiable solvers with structurally constrained neural networks (specifically Bernstein-based KANs) offers a powerful, interpretable, and robust approach to solving inverse problems in nonlocal and memory-dependent systems.