Geometry-Aware R-Structured Kolmogorov-Arnold Networks

Technical Summary: Geometry-Aware R-Structured Kolmogorov–Arnold Networks (GRS-KAN)

Problem Statement
While Kolmogorov–Arnold Networks (KANs) have demonstrated significant promise in scientific computing by replacing weight matrices with learnable univariate functions, they remain "black box" in their handling of explicit geometric constraints. Standard KANs must implicitly infer complex geometric structures—such as feasible regions, Boolean conditions, and discontinuities—from training data using smooth spline bases. This implicit inference often leads to suboptimal accuracy near boundaries and requires large parameter counts to approximate sharp transitions. Conversely, R-functions (developed by V. L. Rvachev) offer a rigorous analytical framework for representing logical operations and complex geometric shapes as single differentiable equations, but they have not been integrated into modern neural architectures for learning smooth functional relationships.

Methodology
The paper proposes the Geometry-aware R-Structured Kolmogorov–Arnold Network (GRS-KAN), a hybrid architecture that decouples the learning of smooth nonlinear structures from the encoding of known geometric constraints.

1. Core Integration: The framework retains the standard KAN branch, which learns smooth functional behavior from data using learnable edge functions (composed of a smooth SiLU base and a B-spline correction). Simultaneously, it introduces an R-function branch that analytically encodes known geometric regions (e.g., rectangles, circles, or complex Boolean combinations) using differentiable R-conjunctions ($\land$) and R-disjunctions ($\lor$).
2. Differentiable Geometry: The R-functions map logical conditions (e.g., "inside a rectangle") to real-valued functions where the sign determines the region. These are smoothed using a logistic sigmoid ($\sigma(\kappa R(x))$) to create differentiable gates suitable for gradient-based backpropagation.
3. Architectural Variants: The authors introduce three specific configurations:
   • Targeted Additive GRS-KAN: Models functions with a smooth background plus a localized jump within a known region ($f_{smooth} + c \cdot \text{gate}$).
   • Targeted Multiplicative GRS-KAN: Models functions that are non-zero only within a known geometric support ($f_{smooth} \cdot \text{gate}$).
   • Agnostic GRS-KAN: A unified architecture that combines a baseline KAN, an additive branch, and a multiplicative branch. It employs learnable structure-selection parameters ($w_{KAN}, w_{add}, w_{mul}$) to automatically determine the dominant interaction mechanism (purely smooth, additive, or multiplicative) based on the data.

Key Contributions

• Theoretical Framework: A method for integrating analytical R-function representations of geometric and logical constraints directly into the differentiable KAN framework.
• Analytical Constructions: Explicit closed-form expressions for geometric indicators (e.g., rectangles) using R-functions, complete with analytical gradients for backpropagation.
• Architectural Innovation: The proposal of additive, multiplicative, and agnostic variants that allow for explicit geometric encoding without sacrificing the differentiability required for optimization.
• Automatic Structure Discovery: Demonstration that learnable branch weights can identify whether a specific geometric prior is beneficial for a given task, effectively performing structure selection.

Experimental Results
Numerical experiments were conducted on smooth benchmarks and discontinuous problems involving rectangular supports.

• Smooth Benchmarks: On standard KAN benchmarks (e.g., $f(x,y) = \exp(\sin(\pi x) + y^2)$ and $f(x,y)=xy$), the authors' implementation successfully replicated the ability of KANs to prune to sparse structures and recover exact symbolic expressions, validating the baseline.
• Discontinuous/Geometric Benchmarks:
  • Rectangular Additive Discontinuity: The targeted additive GRS-KAN reduced test RMSE by 67% and boundary-band RMSE by 61% compared to standard KANs. The explicit geometric encoding allowed the model to localize the discontinuity sharply, whereas standard KANs produced smooth, inaccurate transitions.
  • Masked Product: For functions defined only within a rectangle, the multiplicative GRS-KAN reduced test RMSE by 58% and boundary RMSE by 50%.
  • Agnostic Performance: The agnostic variant successfully identified the correct structure for specific tasks (e.g., suppressing geometry branches for unconstrained $xy$ and prioritizing multiplicative weights for masked $xy$). While it did not always outperform the specialized targeted variants, it achieved competitive accuracy without requiring prior knowledge of the functional form.

Significance and Claims
The paper claims that GRS-KAN represents a distinct direction from existing KAN variants, which primarily focus on modifying approximation mechanisms (e.g., basis functions or attention). Instead, GRS-KAN incorporates explicit analytical geometry directly into the network architecture.

The primary significance lies in the ability to:

1. Improve Accuracy: Explicitly encoding known geometric constraints significantly enhances predictive accuracy and boundary localization in problems with discontinuities or restricted domains.
2. Enhance Interpretability: The model provides an explicit analytical representation of the learned geometric structure, moving beyond black-box approximations to reveal how geometry interacts with the smooth background.
3. Automate Prior Selection: The agnostic variant demonstrates that neural networks can automatically determine whether geometric priors are necessary, offering a robust solution for scientific problems where the functional role of geometric constraints is unknown.

The authors note that while the experiments utilize simple rectangles and circles, the framework is theoretically applicable to arbitrary implicit regions, including non-convex domains and Boolean combinations of constraints, paving the way for applications in engineering design spaces and process control.