📊 statistics

On the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimators

This paper establishes that Kolmogorov-Arnold Networks (KANs) with B-spline components achieve minimax-optimal regression rates independent of ambient dimension, while also providing adaptive knot-selection rules and clarifying the non-identifiability of their univariate components.

Original authors: Wei Liu, Eleni Chatzi, Zhilu Lai

Published 2026-07-31

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Wei Liu, Eleni Chatzi, Zhilu Lai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to predict the weather. The world is messy, with temperature, humidity, wind speed, and pressure all interacting in complex ways. In the world of machine learning, this is called "nonparametric regression." It's the art of finding a hidden pattern in a sea of data without forcing the data into a simple, pre-made box like a straight line. For decades, scientists have used two main tools for this. The first is the "neural network," a digital brain made of layers that can learn almost anything but is often a "black box"—we know it works, but we can't easily see how it works or why it gets the answer right. The second is the "spline," a mathematical tool that fits smooth curves through points, like a flexible ruler. Splines are transparent and easy to understand, but they struggle when the data gets too complicated or has too many dimensions (like adding more weather variables).

Recently, a new type of neural network called a Kolmogorov–Arnold Network, or KAN, has appeared. Think of a KAN as a clever hybrid: it has the layered structure of a neural network, but instead of using mysterious, tangled connections, it builds its predictions by stacking simple, one-dimensional curves (splines) on top of each other. It's like building a skyscraper out of clear glass panels rather than opaque concrete. The big question for scientists was: "Does this new glass-skyscraper actually work as well as the old concrete ones, and can we prove it mathematically?" This paper dives into that question, not just by running experiments, but by building a rigorous mathematical proof to show exactly how fast and how accurately these networks learn.

The Glass Skyscraper vs. The Concrete Wall

The authors of this paper set out to prove that KANs are not just a cool idea, but a mathematically optimal way to learn from data. They focused on a specific type of KAN where the building blocks are "B-splines" (those flexible rulers mentioned earlier). Their main discovery is a "speed limit" for learning. They proved that if the data they are trying to predict has a certain level of smoothness (let's call it "smoothness $r$ "), the KAN learns at a speed of roughly $O((\log n/n)^{2r/(2r+1)})$ .

To put this in everyday terms, imagine you are trying to guess the shape of a hidden object by feeling it with your fingers. If the object is very smooth (like a polished marble), you need fewer touches to figure it out. If it's bumpy and jagged, you need many more. The paper shows that the KAN gets better at guessing the shape at a rate that depends only on how smooth the object is, not on how many dimensions the object has. This is a huge deal. Usually, when you add more dimensions (more variables to track), learning gets exponentially harder—a problem known as the "curse of dimensionality." It's like trying to find a needle in a haystack that keeps growing bigger every time you add a new room to the barn. The authors found that because the KAN is built from simple, one-dimensional pieces, it completely sidesteps this curse. It learns just as fast whether the data has 5 dimensions or 20, provided the data actually follows the KAN's specific structure.

The Logarithmic Hiccup and the "Log" Factor

However, the story isn't perfectly smooth. The authors found that the KAN's learning speed is slightly slower than the absolute theoretical best by a tiny factor involving a logarithm (specifically, a factor of $(\log n)^{2r/(2r+1)}$ ). They traced this "hiccup" not to the KAN's architecture itself, but to the fact that the network is non-linear.

Think of it this way: if you are trying to find a specific book in a library where the books are arranged in a perfect, straight line (a linear system), you can find it instantly. But if the books are arranged in a complex, twisting maze (a non-linear system), you have to do a bit more searching, which adds that small "logarithmic" delay. The authors showed that if you simplify the KAN so it acts like a straight line, that extra delay disappears. This suggests the KAN isn't inherently "hard" to learn; it's just that the math of navigating its non-linear twists adds a tiny bit of overhead.

The Mystery of the Hidden Components

One of the most fascinating parts of the paper is what happens when you try to look inside the KAN to see the individual pieces it used to build the answer. The authors proved that you cannot uniquely identify these individual pieces just by looking at the final answer. It's like trying to figure out the exact ingredients of a cake just by tasting the finished product. If you have a cake that tastes like "vanilla," you can't be sure if the baker used 1 cup of vanilla extract and 2 cups of flour, or 2 cups of vanilla and 1 cup of flour, because the recipe allows for a "scale group"—a way to swap amounts between ingredients without changing the final taste.

They showed that simply "centering" the data (making sure the average is zero) isn't enough to solve this mystery. The network can still shuffle the weights around in a way that changes the internal components but leaves the final prediction exactly the same. This means that while the KAN is great at predicting the result, you can't always trust the internal parts to tell you the "true" underlying cause unless you add extra rules to lock them down.

The Knots and the Adaptability

To make these networks work, you need to decide how many "knots" (the points where the flexible ruler bends) to use. Too few, and the ruler is too stiff to follow the curve; too many, and it starts to wiggle wildly and memorize the noise instead of the pattern. The authors derived a perfect rule for this: the number of knots should grow roughly like $n^{1/(2r+1)}$ , where $n$ is the amount of data you have.

Even cooler, they showed that you don't need to know the "smoothness" ( $r$ ) of the data ahead of time to get this right. They created a method where the network can automatically pick the right number of knots by testing different options, much like a hiker trying different paths to find the one that gets them to the summit fastest. In their simulations, this "adaptive" method worked just as well as if they had known the answer from the start.

Testing the Theory in the Lab

Finally, the authors didn't just stop at the math; they put it to the test. They created fake data with known smoothness levels and watched the KAN learn. The results were spot-on.

The Speed: The KAN learned at the predicted speed, getting more accurate as they added more data.
The Dimensions: When they increased the number of variables from 5 to 20, the KAN kept its speed, while other standard methods (like k-nearest neighbors) slowed down drastically, confirming that the KAN truly escapes the "curse of dimensionality."
The Knots: The optimal number of knots they found in the experiments matched their mathematical prediction perfectly.

The paper concludes that KANs are a powerful, mathematically sound tool. They offer the best of both worlds: the learning power of deep neural networks and the transparency of splines. While there are still some open questions about how to perfectly identify the internal parts of the network, the proof that they learn at the optimal rate (up to that tiny logarithmic factor) is a major step forward. It tells us that when the data has a specific structure, KANs are not just a clever trick, but the most efficient way to learn it.

Technical Summary: On the Rate of Convergence of Kolmogorov–Arnold Network Regression Estimators

Problem Statement
The paper addresses the statistical convergence rates of Kolmogorov–Arnold Networks (KANs) in nonparametric regression. While KANs have gained empirical traction as a structured alternative to deep neural networks—approximating multivariate functions via compositions of univariate transformations—formal statistical guarantees regarding their estimation rates have been scarce. The authors investigate the performance of the least-squares estimator over a KAN sieve where the univariate components are B-splines. Specifically, they ask: what rate does the empirical risk minimizer achieve over a class of KAN functions with Sobolev-smooth univariate components, and is this rate minimax optimal?

Methodology
The authors analyze the least-squares estimator $\hat{f}_n$ defined over a sieve space $\mathcal{F}_n$ constructed from KAN architectures. The target function $f_0$ is modeled as a finite sum of $Q$ units:
$f(x) = \sum_{q=1}^Q g_q(T_q(x))$
where $T_q(x)$ represents either an additive aggregation ( $\sum \psi_{qj}(x_j)$ ) or a multiplicative aggregation ( $\prod \psi_{qj}(x_j)$ ). Both the inner functions $\psi_{qj}$ and outer functions $g_q$ are approximated using B-splines of order $m$ with $k_n$ knots.

Key methodological steps include:

Assumptions: The design distribution has a bounded density, errors are i.i.d. and sub-Gaussian, and the regression function belongs to a class $\mathcal{F}^{KAN}_r$ where univariate components have Sobolev smoothness $r$ . Crucially, the outer functions $g_q$ are constrained in the $L_\infty$ -Sobolev scale ( $W^{r,\infty}$ ) rather than $L_2$ , and the sieve imposes explicit sup-norm bounds on coefficients to ensure stability.
Theoretical Tools: The analysis combines three classical ingredients:
- An approximation bound for KAN compositions (Lemma C.1), utilizing a telescoping argument for multiplicative nodes.
- A metric-entropy bound for the nonlinear sieve (Lemma C.2), accounting for the nonlinearity of the composition.
- An oracle inequality for least squares over uniformly bounded classes (Lemma C.3).
Identifiability Analysis: The authors examine whether the learned components converge to the true components, proving that centering alone is insufficient for identifiability due to scale invariance and mixing ambiguities.

Key Contributions and Results

Convergence Rate: The least-squares estimator over the KAN spline sieve attains a uniform convergence rate of:
$O\left(\left(\frac{\log n}{n}\right)^{\frac{2r}{2r+1}}\right)$
This rate holds uniformly over a ball of KAN functions with univariate components of Sobolev smoothness $r$ .
Dimension-Free Exponent: The exponent $\frac{2r}{2r+1}$ is independent of the ambient dimension $d$ . The authors clarify that this does not violate the minimax lower bound $n^{-2r/(2r+d)}$ for general Sobolev classes on $[0,1]^d$ . Instead, the dimension-free rate arises from the structural assumption that the target admits a KAN representation with a fixed number of units $Q$ .
Minimax Optimality: A matching lower bound of order $n^{-2r/(2r+1)}$ is established, showing the estimator is minimax optimal up to a logarithmic factor. The authors trace this logarithmic factor to the nonlinearity of the sieve (requiring chaining over a covering) rather than the KAN architecture itself. They show that for linear sub-models (e.g., fixed outer functions), the logarithmic factor disappears.
Identifiability: The paper proves that centering constraints do not identify a KAN representation; a one-parameter scale group remains, and further ambiguities exist for $d=1$ or $Q \ge 2$ . Consequently, the consistency of the fitted function $\hat{f}_n$ does not imply the consistency of the individual univariate components $\hat{\psi}_{qj}$ without additional normalization and non-degeneracy conditions.
Adaptivity: A knot-selection rule $k_n \asymp (n/\log n)^{1/(2r+1)}$ is derived. The authors show that penalized selection over a dyadic knot grid attains the optimal rate adaptively, without prior knowledge of the smoothness $r$ (up to the spline order).

Experimental Verification
The theoretical predictions are verified on synthetic targets with exactly known smoothness $r$ .

Approximation Error: The univariate spline approximation error decays as $k^{-r}$ , matching theoretical bounds.
Convergence Rates: The empirical $L_2$ risk decays at rates consistent with $n^{-2r/(2r+1)}$ (up to logarithmic factors) across additive, univariate, and multiplicative targets.
Dimension Independence: In dimension sweeps ( $d=5$ to $d=20$ ), the KAN estimator maintains a steep convergence slope, whereas structure-free baselines (k-NN, Kernel Ridge) degrade toward the curse-of-dimensionality rate $n^{-2r/(2r+d)}$ .
Knot Selection: The risk exhibits a U-shape with respect to the number of knots, and cross-validation successfully locates the optimal knot count without knowing $r$ .

Significance and Claims
The paper claims to provide the first rigorous convergence analysis for KANs, establishing that they achieve minimax-optimal rates for structured nonparametric regression. The primary significance lies in demonstrating that the KAN architecture, by leveraging a sum of univariate Sobolev components, escapes the curse of dimensionality inherent in general nonparametric estimation, provided the target function possesses the specific KAN structure.

The authors are modest regarding the logarithmic factor, noting it is an artifact of the nonlinear sieve's entropy rather than an intrinsic limitation of KANs, and identifying the removal of this factor as an open technical challenge requiring local-entropy analysis. Furthermore, they emphasize that while the regression function is consistently estimated, the recovery of the underlying univariate components requires specific identifiability conditions not guaranteed by standard centering. The work bridges the gap between the empirical success of KANs and classical nonparametric theory, situating them alongside spline-based methods in terms of optimality while retaining the layered composition structure of neural networks.