Partition-of-Unity Gaussian Kolmogorov-Arnold Networks

Technical Summary: Partition-of-Unity Gaussian Kolmogorov–Arnold Networks (PU-GKAN)

1. Problem Statement

Kolmogorov–Arnold Networks (KANs) have emerged as a powerful alternative to Multilayer Perceptrons (MLPs) by placing learnable univariate functions on edges rather than fixed activations on nodes. While Gaussian Radial Basis Functions (RBFs) are an attractive basis for KANs due to their smoothness and locality, they suffer from a significant classical weakness: extreme sensitivity to the scale parameter ($\epsilon$).

• Small $\epsilon$: Leads to overly localized bases that fail to capture global trends.
• Large $\epsilon$: Leads to nearly flat, highly correlated features that cause numerical ill-conditioning and loss of rank.
  The paper seeks to address this instability and improve the approximation accuracy of Gaussian-based KANs through a mathematically grounded normalization technique.

2. Methodology

The author introduces the Partition-of-Unity Gaussian KAN (PU-GKAN). The core innovation is the application of Shepard-type normalization directly to the Gaussian basis functions within each KAN edge.

• Mathematical Construction: Instead of using raw Gaussian functions $\phi_g(t)$, the model uses normalized basis functions $e\phi_g(t)$:
  $$e\phi_g(t) = \frac{\exp\left(-\frac{(t-c_g)^2}{\epsilon^2}\right)}{\sum_{h=1}^{G} \exp\left(-\frac{(t-c_h)^2}{\epsilon^2}\right)}$$
  where $c_g$ are fixed centers and $G$ is the number of centers. This ensures that $\sum e\phi_g(t) = 1$, creating a partition of unity.
• Finite-Feature & Kernel Viewpoint: The paper provides a rigorous derivation showing that both standard GKAN and PU-GKAN can be viewed as finite-feature layers. This allows the author to explicitly define the induced additive kernels and empirical feature matrices, which are used to analyze the model's conditioning.
• Scale Selection Rule: To mitigate the $\epsilon$ sensitivity, the author proposes a practical reference interval for the scale parameter:
  $$\epsilon \in \left[ \frac{1}{G-1}, \frac{2}{G-1} \right]$$
  The lower bound ensures sufficient overlap between adjacent centers, while the upper bound is derived from a conditioning threshold to prevent numerical instability.

3. Key Contributions

• Architectural Innovation: Introduces PU-GKAN, which achieves Shepard-type normalization without the complexity of domain-decomposition or additional sub-networks. It preserves the original KAN structure while adding the property of exact constant reproduction at the edge level.
• Theoretical Framework: Formulates the KAN as a composition of finite-feature layers, providing a clear link between neural architecture and classical kernel/RBF approximation theory.
• Robust Scale Selection: Establishes a conditioning-based heuristic for choosing $\epsilon$, reducing the need for exhaustive hyperparameter sweeps.
• Generalizability: Demonstrates that the normalization principle is not exclusive to Gaussians and can be applied to other RBFs, such as Matérn functions.

4. Results

The paper evaluates PU-GKAN across a diverse battery of tests:

• Function Approximation: For smooth and moderately non-smooth targets ($F_1$–$F_6$), PU-GKAN consistently outperformed standard GKAN, often reducing RMSE by 30% to 78%.
• Robustness to Hyperparameters: PU-GKAN showed significantly reduced sensitivity to the scale parameter $\epsilon$ and performed better across varying sample sizes ($N$) and numbers of centers ($G$).
• High-Dimensionality: The benefits persisted in higher-dimensional settings ($d=1$ to $d=4$), maintaining lower minimum errors and broader "low-error" regions in the $\epsilon$ landscape.
• Physics-Informed Learning (PINNs): In solving the Helmholtz equation and the 1D Wave equation, PU-GKAN demonstrated superior convergence and lower final validation errors compared to standard GKAN, proving its utility in scientific machine learning.
• Exception: For highly discontinuous targets ($F_7$), the improvement was less uniform, as smooth RBFs inherently struggle with discontinuities.

5. Significance

This work provides a simple yet mathematically profound stabilization mechanism for RBF-based KANs. By transforming the Gaussian basis into a partition of unity, the author effectively regularizes the internal coordinate distribution and improves the conditioning of the feature matrices. This makes Gaussian KANs more reliable, easier to train, and more accurate for complex scientific computing tasks, including solving partial differential equations (PDEs).