🤖 machine learning

LTBs-KAN: Linear-Time B-splines Kolmogorov-Arnold Networks

LTBs-KAN introduces a novel Kolmogorov-Arnold Network architecture that achieves linear-time complexity and reduced parameter counts by replacing intensive B-spline computations with a more efficient base-spline approach and utilizing product-of-sums matrix factorization.

Original authors: Eduardo Said Merin-Martinez, Andres Mendez-Vazquez, Eduardo Rodriguez-Tello

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Eduardo Said Merin-Martinez, Andres Mendez-Vazquez, Eduardo Rodriguez-Tello

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Slow and Clunky" Brain

Imagine you are trying to teach a robot how to recognize a cat.

Currently, most robots use a "brain" called an MLP (Multilayer Perceptron). Think of an MLP like a massive wall of millions of tiny, simple light switches. To recognize a cat, the robot flips these switches on and off. It’s incredibly fast, but it’s a bit "dumb"—it takes a massive amount of switches to understand complex shapes.

Recently, scientists invented a new kind of brain called a KAN (Kolmogorov-Arnold Network). Instead of simple light switches, KAN uses "flexible rubber bands" (called B-splines). These rubber bands can stretch, bend, and curve to perfectly trace the outline of a cat’s ear or a whisker. This makes the KAN much "smarter" and more precise than the MLP, but there’s a huge catch: it’s incredibly slow.

Because calculating exactly how a rubber band bends at every single point is mathematically exhausting, the KAN brain gets "stuck" doing math, making it much slower than the old-fashioned light-switch brain.

The Solution: The LTBs-KAN (The "Smart Shortcut")

The authors of this paper created a new version called LTBs-KAN. They didn't change the "rubber band" idea, but they changed how the robot calculates the bending.

Here is how they fixed it using three clever tricks:

1. The "Pre-Bent" Shortcut (The LTBs Algorithm)

Imagine if, instead of calculating how a rubber band bends every single time you moved it, you had a machine that pre-calculated all the possible bends and stored them in a neat, organized folder.
The authors developed a mathematical "fast-track" (the LTBs Algorithm). Instead of doing heavy, repetitive math during the actual learning process, they do a quick "pre-calculation" step. This turns a mountain of math into a small hill, allowing the robot to process information in "linear time"—which is a fancy way of saying "it doesn't get exponentially slower as things get more complex."

2. The "Lego-Brick" Weights (Matrix Factorization)

In a normal brain, every connection between neurons is a unique, heavy weight that the robot has to remember. This takes up a lot of "memory space."
The authors used a trick called Matrix Factorization. Instead of remembering a million unique weights, they use a few "master shapes" (like Lego bricks) and just learn how to mix them together in different amounts. This significantly shrinks the size of the brain without making it any less smart.

3. The "Adaptive Map" (Grid Updates)

Imagine you are drawing a map of a city. If you are looking at a tiny alleyway, you don't need a map of the whole country; you need a zoomed-in, detailed map of that alley.
The LTBs-KAN does this automatically. It looks at the data coming in and "zooms in" its mathematical grid to where the action is. This ensures the "rubber bands" are always placed exactly where they are needed most.

Does it actually work? (The Results)

The researchers tested their new "Smart Brain" on three famous tests: MNIST (handwritten numbers), Fashion-MNIST (clothing items), and CIFAR-10 (colorful objects like cars and birds).

On Numbers & Clothes: The LTBs-KAN was a superstar among the "new" KAN brains. It was faster and more efficient than almost all its competitors. While the old-fashioned "light-switch" (MLP) brain was still slightly faster, the LTBs-KAN was much smaller and more "expressive."
On Colorful Objects (CIFAR-10): This is where they really shined. They turned the LTBs-KAN into a "Vision Brain" (ConvNet). It beat the traditional models, proving that this "flexible rubber band" approach is actually better at seeing the complex patterns in real-world images.

The Bottom Line

The researchers took a brilliant but "clunky" new way of thinking (KAN) and gave it a high-performance engine (LTBs). They created a brain that is smart like a curve, but fast like a line.

Technical Summary: LTBS-KAN (Linear-Time B-Splines Kolmogorov-Arnold Networks)

1. Problem Statement

Kolmogorov-Arnold Networks (KANs) represent a paradigm shift from traditional Multilayer Perceptrons (MLPs) by replacing fixed activation functions at nodes with learnable univariate functions (splines) on the edges. While KANs offer superior expressibility and interpretability, they suffer from a critical bottleneck: computational inefficiency.

The standard implementation relies on the Boor-Mansfield-Cox algorithm to compute B-spline basis functions, which is computationally expensive ( $O(nm^2)$ complexity). This makes KANs significantly slower (often up to 10x) than MLPs, limiting their practical application in deep learning tasks requiring high-speed inference and training.

2. Methodology

The authors propose LTBS-KAN, a novel architecture designed to achieve linear-time complexity for B-spline computations and reduce parameter overhead. The methodology is built on three pillars:

LTBs Algorithm (Linear-Time B-splines): Instead of the traditional recursive Cox-De Boor approach, the authors implement a parallel algorithm based on a differential-recurrence relation. This algorithm computes the Bernstein–Bézier coefficients of B-spline basis functions. By shifting the heavy computation to a preprocessing stage, the forward pass evaluation is reduced to evaluating a polynomial of degree $m$ , achieving an effective complexity of $O(n)$ relative to the grid size.
Adaptive Grid/Knot Updates: To ensure the splines remain relevant to the data distribution, the model implements a gridUpdate procedure. This dynamically adjusts the knot vector based on the empirical range (min/max) of the current input batch, ensuring numerical stability and optimal coverage of the input domain.
Parameter Reduction via Matrix Factorization: To address the high parameter count inherent in KANs, the authors replace the standard dense weight matrix $W$ with a product-of-sums matrix factorization. The linear transformation is represented as a weighted sum of a small set of fixed, low-rank matrices ( $M_{jk}$ ), significantly reducing the number of trainable parameters without sacrificing performance.
KAN-ConvNet Extension: The authors extend the KAN concept to convolutional layers (KAN-Conv2D). Instead of a simple dot product, the layer applies a shared LTBS-KAN transformation to local spatial patches extracted via an im2col (unfold) operation, maintaining translational equivariance while increasing local non-linear expressiveness.

3. Key Contributions

A novel parallel linear algorithm (LTBs) for B-spline coefficient computation that optimizes the forward pass.
An efficient KAN implementation using "Blocks" (Boundary initialization and Backward recursion) to leverage GPU acceleration via PyTorch.
A structured parameterization method using sum-product decomposition to mitigate the "parameter explosion" typical of KAN architectures.
A new Convolutional KAN architecture (KAN-ConvNet) that integrates spline-based operators into the standard CNN pipeline.

4. Experimental Results

The model was evaluated on three benchmark datasets:

MNIST: LTBS-KAN achieved the best performance among all tested KAN variants (surpassing EfficientKAN, FastKAN, and BSRBF-KAN) in terms of accuracy, precision, and recall, while maintaining a competitive training speed and low parameter count.
Fashion-MNIST: The model showed strong convergence, though it was slightly outperformed by models using simpler linear approximations (like FastKAN), suggesting a "bias-variance dilemma" where higher-degree splines require more data or specific regularization to match simpler models.
CIFAR-10: In the convolutional setting, the LTBS-KAN ConvNet outperformed both a standard AlexNet and an EfficientKAN-ConvNet across all metrics (Accuracy, F1-score, etc.). Notably, it achieved higher accuracy than the EfficientKAN-ConvNet while being significantly faster in execution time.

5. Significance

This work is significant because it bridges the gap between the theoretical expressibility of Kolmogorov-Arnold Networks and the practical requirements of modern Deep Learning. By optimizing the mathematical foundation of the spline computation and introducing structured parameterization, the authors demonstrate that KANs can be made efficient enough for real-world applications, such as computer vision, where they can outperform traditional CNNs in feature extraction and non-linear mapping.