🤖 machine learning

Hierarchical RBF-KAN and RBF-SKAN Architectures for Multidimensional Function Approximation and Random Field Learning

This paper proposes and theoretically validates hierarchical Kolmogorov-Arnold network architectures utilizing radial basis functions (RBF-KAN and RBF-SKAN) to effectively approximate high-dimensional deterministic functions and random fields, demonstrating their ability to mitigate the curse of dimensionality and achieve convergence under the Wasserstein-2 metric.

Original authors: Mingtao Xia, Qijing Shen

Published 2026-06-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mingtao Xia, Qijing Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand the world. Sometimes, the world is a predictable machine (like a clock), and sometimes it's a chaotic storm (like the weather). This paper introduces two new "brain architectures" designed to learn these patterns better than previous methods.

Here is the breakdown of their invention, using simple analogies:

1. The Problem: The "Dimensionality Trap"

Imagine you are trying to describe a shape.

If it's a line (1D), it's easy.
If it's a cube (3D), it's harder.
If it's a hyper-cube with 100 dimensions, it becomes a nightmare.

In math and AI, this is called the "Curse of Dimensionality." As you add more variables (dimensions), the amount of data needed to understand the shape explodes, and standard AI models get confused and fail. They try to memorize the whole giant puzzle at once and get overwhelmed.

2. The Solution: The "Hierarchical RBF-KAN" (For Predictable Things)

The authors built a new type of neural network called a Hierarchical RBF-KAN.

The Analogy: Think of a standard AI as a general contractor who tries to build a whole house by looking at every brick, window, and roof tile simultaneously. It's chaotic and slow.
The New Approach: The Hierarchical RBF-KAN is like a team of specialized subcontractors working in a strict hierarchy.
- Step 1: One team handles just the x-axis (left/right).
- Step 2: Another team handles just the y-axis (up/down).
- Step 3: A "manager" layer combines these simple, one-dimensional stories into a complex 3D picture.

Why this works:
Instead of trying to solve a 100-dimensional puzzle all at once, the network breaks it down into 100 tiny, easy 1-dimensional puzzles. It then stitches the answers together.

The Paper's Claim: This method is mathematically proven to be much better at handling high-dimensional data than older methods. It essentially "cheats" the curse of dimensionality by simplifying the problem.
The Secret Sauce: They also added "ResNet" techniques (which act like elevators in a skyscraper, allowing information to skip floors if needed) to make the learning process faster and more stable.

3. The Second Solution: The "Hierarchical RBF-SKAN" (For Chaotic Things)

Sometimes, the world isn't just a shape; it's a random field. Imagine trying to predict the wind. It's not just one number; it's a cloud of possibilities. You need to predict not just where the wind is, but the probability of it being there.

The Analogy: Previous methods (like CVAE or CNF) are like trying to guess the weather by looking at a single snapshot. They often get the "average" right but fail to predict the variability (the storms).
The New Approach: The RBF-SKAN is a probabilistic simulator. It doesn't just learn one path; it learns the entire "cloud" of possibilities.
- It uses the same "specialized subcontractor" hierarchy as the first model.
- But it adds a layer of controlled randomness (like rolling dice) inside the network. This allows it to generate many different possible outcomes that match the real-world chaos.

Why this works:
The paper claims this new model is better at capturing the "spread" of data (uncertainty) than current industry standards. It learns the "shape" of the randomness, not just the average.

4. The Results: What Did They Find?

The authors tested these new brains on three types of challenges:

High-Dimensional Functions: They tried to teach the AI a complex, wiggly math function with up to 6 dimensions.
- Result: The new Hierarchical RBF-KAN was the clear winner. Older models (like standard RBF networks or other KANs) failed to learn the pattern as the dimensions increased. The new model stayed accurate.
Chaotic Systems (The Lorenz System): They tried to predict the path of a chaotic weather system (the famous "butterfly effect").
- Result: The new model predicted the path and the underlying rules of motion more accurately than any other network they tested.
Random Fields (Uncertainty): They tried to reconstruct a noisy, random data field.
- Result: The new RBF-SKAN was the only model that could accurately predict both the average value and the variability (standard deviation) of the data. The competitors (CVAE and CNF) were good at the average but terrible at predicting the chaos.

Summary

The paper presents a new way to build AI brains that:

Breaks big, complex problems into small, simple 1D pieces (solving the "Curse of Dimensionality").
Uses a hierarchy of specialized layers rather than a giant, messy web.
Can handle both predictable math and chaotic randomness better than current top-tier models.

The authors conclude that by organizing the network this way, they can learn complex, high-dimensional things with much higher accuracy and less confusion than before.

Technical Summary: Hierarchical RBF-KAN and RBF-SKAN Architectures

Problem Statement
The paper addresses the challenges of approximating high-dimensional deterministic functions and learning random field models. While Radial Basis Function Neural Networks (RBFNNs) have established theoretical foundations for universal approximation and offer favorable optimization properties, existing multilayer RBF architectures often struggle with the "curse of dimensionality" in complex multivariate settings. Furthermore, recent Kolmogorov–Arnold Network (KAN) architectures, which aim to replicate the Kolmogorov–Arnold representation theorem, have primarily focused on spline-based activations or uniform layer structures. The authors identify a gap in rigorously analyzing how hierarchical, non-uniform RBF-based architectures can effectively learn high-dimensional functions and stochastic fields, particularly regarding their ability to mitigate dimensionality issues and approximate random fields under Wasserstein metrics.

Methodology
The authors propose two novel architectures:

Hierarchical RBF-KAN: Designed for multidimensional deterministic function approximation. Unlike standard RBFNNs or previous RBF-KAN models that utilize fully connected layers with uniform neuron counts, this architecture employs a "hierarchical block" structure. Each block consists of two distinct activation layers where the number of neurons in the first post-activation layer is $(2d+1)$ times that of the second, explicitly mimicking the structure of the Kolmogorov–Arnold representation theorem. The activation function is a Gaussian kernel $B(x) = \exp(-x^2)$ . The framework supports both standard feedforward propagation and ResNet-style skip connections.
Hierarchical RBF-SKAN: A stochastic extension of the RBF-KAN designed for random field learning. This architecture integrates two RBF-KAN modules. The first module processes inputs, while the second module introduces randomness by sampling weights ( $w_{i,j}$ ) and RBF scale parameters from independent distributions. This allows the network to output a distribution rather than a single deterministic value, enabling the learning of random field models.

Theoretical Contributions
The paper establishes rigorous universal approximation theorems for both architectures:

Deterministic Approximation: The authors prove that the hierarchical RBF-KAN can approximate any continuous multivariate function to arbitrary precision. Crucially, they derive quantitative error bounds showing that under specific smoothness conditions, the convergence rate does not explicitly deteriorate as the input dimension $d$ increases. This suggests the architecture can partially alleviate the curse of dimensionality by reducing the effective dimensionality of the approximation problem.
Stochastic Approximation: For the hierarchical RBF-SKAN, the authors prove an asymptotic universal approximation result under the squared Wasserstein-2 ( $W_2$ ) metric. They demonstrate that the framework can approximate random field models provided the underlying probability measures satisfy uniform Lipschitz continuity conditions with respect to the input space.

Experimental Results
The authors validate their methods through three primary numerical experiments:

Multidimensional Function Approximation: The hierarchical RBF-KAN was tested on a highly oscillatory function (Eq. 23) with input dimensions ranging from 1 to 6. Results indicate that while standard RBF-MLPs and naive RBF-KANs fail to maintain accuracy as dimensionality increases, the proposed hierarchical RBF-KAN maintains high accuracy. The incorporation of ResNet techniques was found essential for stable training and accuracy in multi-block configurations.
Chaotic Dynamical System Reconstruction: The architecture was applied to reconstruct the Lorenz system. The hierarchical RBF-KAN (with two blocks and ResNet) achieved the lowest relative errors in both predicted trajectories and learned dynamics compared to various MLPs and KAN variants.
Random Field Learning: The hierarchical RBF-SKAN was compared against Conditional Normalizing Flows (CNF) and Conditional Variational Autoencoders (CVAE) for reconstructing a multivariate random field. The proposed method demonstrated superior performance in predicting the standard deviation of the random field, particularly in higher dimensions, whereas CVAE and CNF approaches showed significant errors in capturing stochasticity.

Significance and Claims
The paper claims that the proposed hierarchical design is essential for faithfully reproducing the Kolmogorov–Arnold representation structure, which is critical for learning multivariate functions. Key claims include:

The hierarchical RBF-KAN offers a theoretical pathway to partially alleviate the curse of dimensionality in function approximation, a property not explicitly guaranteed in existing multilayer RBF or standard KAN architectures.
The hierarchical RBF-SKAN provides a robust framework for uncertainty quantification in random field learning, outperforming prevailing machine learning approaches like CNF and CVAE in capturing distributional properties (specifically standard deviation).
The integration of ResNet techniques into multi-block hierarchical RBF-KANs significantly enhances approximation accuracy and training stability.

The authors conclude that while the proposed methods incur higher computational costs than standard MLPs (due to the hierarchical structure and $W_2$ loss evaluation), they offer superior accuracy and expressivity for high-dimensional and stochastic learning tasks. Future work is suggested to explore deeper block structures, integration into transformer architectures, and extension to classification tasks.