🤖 AI

TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions

This paper introduces TruKAN, a novel Kolmogorov-Arnold Network architecture that replaces B-spline bases with truncated power functions to achieve a superior balance of accuracy, computational efficiency, and interpretability, demonstrating significant performance gains over existing KAN variants when integrated into EfficientNet-V2 frameworks for computer vision tasks.

Original authors: Ali Bayeh, Samira Sadaoui, Malek Mouhoub

Published 2026-02-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Bayeh, Samira Sadaoui, Malek Mouhoub

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize pictures of cats, dogs, and cars. To do this, the robot needs a "brain" made of mathematical layers that can spot patterns.

For a long time, the standard brain for these tasks was called an MLP (Multi-Layer Perceptron). Think of an MLP as a factory assembly line where every worker (neuron) uses the exact same, pre-set tool (a fixed activation function like ReLU) to do their job. It's fast and reliable, but it's a bit rigid.

Then, a new type of brain called KAN (Kolmogorov-Arnold Network) arrived. KANs are like a team of master craftsmen. Instead of using a pre-set tool, every single worker learns their own unique, custom tool (a "spline") to solve the specific problem they face. This makes KANs incredibly smart and easy to understand (interpretable) because you can look at their tools and see exactly how they work.

However, there's a catch: Learning these custom tools is slow and expensive. It's like asking every worker to hand-carve their own hammer from scratch every time they start a new job. The paper calls this the "computational bottleneck."

Enter TruKAN: The Smart Shortcut

The authors of this paper, Ali Bayeh, Samira Sadaoui, and Malek Mouhoub, introduced a new architecture called TruKAN. Their goal was to keep the "master craftsman" benefits of KANs but make the training process as fast and efficient as the standard factory line.

Here is how they did it, using a simple analogy:

1. Swapping the "Carved Wood" for "Lego Bricks"

Standard KANs use something called B-splines to build their custom tools. Imagine B-splines as complex, curved pieces of wood that require a very specific, recursive carving process (the de Boor-Cox algorithm) to shape. It's precise but slow.

TruKAN swaps these for Truncated Power Functions.

The Analogy: Think of B-splines as hand-carved wooden joints. Think of Truncated Power Functions as Lego bricks.
Instead of carving a curve from scratch, TruKAN builds its curves by snapping together simple, pre-defined Lego pieces (polynomials) and adding "knots" (connection points) where the shape needs to bend.
This is mathematically equivalent to the old method (they can build the same shapes), but it's much faster to assemble because you don't need the complex recursive carving algorithm.

2. The "Shared vs. Individual" Blueprint

The paper explores two ways to arrange these Lego bricks:

Shared Knots: Imagine a construction crew where everyone uses the same set of pre-measured connection points. This is efficient and keeps the team coordinated.
Individual Knots: Imagine every worker gets their own custom set of connection points. This is more flexible but takes up more space and time to organize.

The researchers found that Shared Knots often worked best, offering a sweet spot between speed and accuracy.

3. The "Stabilizer" (Normalization)

Because these Lego-like functions can sometimes get a bit wobbly (numerically unstable) when stacked too high, the researchers added a "stabilizer" called Layer Normalization.

The Analogy: It's like adding a shock absorber to a car. It smooths out the bumps in the road (training process) so the car doesn't crash (explode in error) when going fast. They found that adding this stabilizer made TruKAN significantly more accurate.

The Results: Speed and Smarts

The team tested TruKAN on four famous picture-recognition datasets (CIFAR-10, CIFAR-100, Oxford-Pets, and STL-10). They compared it against:

MLP: The standard factory line.
Standard KAN: The slow, hand-carving master craftsmen.
SineKAN: A variation using sine waves (like a different type of musical instrument).

The Findings:

Accuracy: TruKAN was often the winner or a very close second. It matched or beat the standard MLPs and crushed the other KAN variants.
Speed: TruKAN trained much faster than standard KANs. In some tests, it was 3 to 4 times faster per step.
Memory: TruKAN used significantly less computer memory (RAM) than standard KANs. One version used less than 120 MB, while standard KANs used over 500 MB.
Interpretability: Just like the original KAN, TruKAN remains "transparent." You can still look at the model and see exactly how it is bending the data to make a decision, unlike the "black box" nature of standard MLPs.

The Bottom Line

The paper argues that TruKAN is the best of both worlds. It keeps the "explainable" and "smart" nature of KANs but replaces the slow, hand-carved math with a faster, Lego-like construction method.

By using Truncated Power Functions (the Legos) and Shared Knots (the efficient blueprint), TruKAN allows computers to learn complex visual tasks (like identifying pets or cars) much faster and with less memory, without losing the ability to understand how the computer is thinking.

What the paper does NOT claim:

It does not claim this works for medical diagnosis or self-driving cars yet (though it mentions these as potential future areas).
It does not claim TruKAN is perfect for every single task; it performed best on the specific image datasets they tested.
It does not claim to have solved all problems; they note that some variations (like individual knots without stabilization) can still struggle with generalization.

Technical Summary of "TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions"

Problem Statement

Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to Multi-Layer Perceptrons (MLPs), offering superior interpretability and the ability to learn univariate edge-wise functions. However, standard KANs rely on B-spline basis functions for their learnable activations. Evaluating these B-splines requires the recursive de Boor–Cox algorithm, which is computationally expensive, often causing KANs to train up to 10 times slower than equivalent MLPs. Furthermore, the reliance on complex spline algorithms and fixed grid structures can lead to scalability issues in high-dimensional problems and deep architectures, characterized by noisy gradients and unstable optimization. While recent variants like SineKAN and FourierKAN attempt to address these issues, they often introduce new trade-offs regarding interpretability or remain limited to synthetic benchmarks and simple datasets like MNIST.

Methodology

The authors propose TruKAN, a new architecture that retains the canonical KAN topology and learnable activations but replaces the B-spline basis with a family of truncated power functions derived from $k$ -order spline theory.

Core Architectural Components:

Truncated Power Basis: Instead of B-splines, TruKAN utilizes truncated power functions of the form $(x - t_j)^k_+$ . Theoretically, the space spanned by these functions is identical to that of B-splines, but their computational evaluation is distinct. While B-splines have compact support (requiring $O(k^2)$ complexity per interval via recursion), truncated power functions have global support. However, the authors argue that in practice, high-resolution grids are not always necessary, allowing TruKAN to operate efficiently with a small number of knots.
Hybrid Layer Structure: Each TruKAN layer combines two components:
- A truncated power component to capture local, piecewise non-linearities.
- A polynomial component (global trend) to replace the auxiliary SiLU activation used in standard KANs. This polynomial term provides a smooth, differentiable global mapping, compensating for the lack of an explicit global non-linearity like SiLU.
Knot Configurations: The paper introduces two variants regarding knot placement:
- Shared Knots: A single set of knot locations is shared across all output channels.
- Individual Knots: Each output channel has its own set of learnable knot positions.
- Both fixed (pre-defined intervals) and learnable (optimized during training with constraints for ordering and positivity) knot strategies are explored.
Training Framework:
- Optimization: To address the instability of training deep networks with truncated power functions (which can suffer from gradient explosion due to exponential growth/decay), the authors employ a hybrid optimization strategy using AdamW wrapped by LookAhead. This replaces the standard L-BFGS used in prior KAN studies to better suit stochastic mini-batch training.
- Normalization: The study investigates the impact of Layer Normalization (and Batch Normalization) within the classifier heads, as traditional regularization techniques like dropout have shown mixed results in KANs.
- Backbone: The models are integrated into an EfficientNet-V2 framework, replacing the standard linear classifier head with MLP, KAN, SineKAN, or TruKAN layers.

Key Contributions

Architectural Innovation: The substitution of B-splines with truncated power functions, coupled with a polynomial global term, creates a model that maintains KAN's expressiveness while simplifying the basis function evaluation.
Enhanced Interpretability: By using truncated power functions, the contribution of each knot becomes explicit. A non-zero coefficient at a knot $t_j$ directly indicates an intentional $k$ -th order curvature at that specific point, offering greater transparency than the overlapping support of B-splines.
Scalability and Efficiency: The paper demonstrates that TruKAN can be effectively scaled to deep architectures and complex vision tasks, moving beyond the synthetic data limitations of previous KAN studies.
Comprehensive Benchmarking: The authors provide a rigorous evaluation across four computer vision datasets (CIFAR-10, CIFAR-100, Oxford-IIIT Pets, STL-10) using both small and deep network configurations, ensuring fair comparisons by matching parameter counts.

Experimental Results

The evaluation was conducted on four datasets using EfficientNet-V2 backbones with varying classifier sizes (small and large architectures).

Accuracy and F1-Score:
- Small Architectures: TruKAN variants, particularly those with Layer Normalization and Shared Knots (TruKAN-SN), consistently outperformed or matched baseline KAN and SineKAN models. On CIFAR-10, TruKAN-IN achieved the highest accuracy (0.8927), surpassing the best MLP baseline. On STL-10, TruKAN-SN achieved 0.9341 accuracy, trailing only the normalized MLP by a negligible margin.
- Large Architectures: TruKAN models demonstrated superior scalability. On CIFAR-10, TruKAN-IN achieved 0.8932 accuracy, outperforming the MLP-N baseline despite using fewer parameters. TruKAN variants consistently dominated KAN and SineKAN baselines across all datasets, often achieving results comparable to or better than the strongest MLPs.
- Normalization Impact: Layer normalization proved critical for TruKAN performance, significantly improving generalization, particularly for individual-knot configurations which otherwise suffered from overfitting.
Computational Efficiency:
- Training Time: Shared-knot TruKAN models (TruKAN-FS) achieved training step times of ~0.0024s, representing a 3-4x improvement over standard KAN models (0.0087s - 0.0099s).
- Memory Usage: TruKAN-FS consumed only 118.04 MB of peak GPU memory, significantly less than KAN-PBT (570.84 MB) and KAN-PBF (691.79 MB). However, individual-knot variants (TruKAN-FI) incurred high memory costs (~2.6 GB) due to expanded knot storage.
- Parameters: TruKAN models generally achieved comparable or better performance with fewer trainable parameters than their KAN and SineKAN counterparts.

Significance and Claims

The paper claims that TruKAN successfully addresses the trade-off between computational efficiency and adherence to KAN principles. By replacing the recursive B-spline algorithm with truncated power functions and a polynomial global term, TruKAN achieves:

Superior Efficiency: Significant reductions in training time and memory usage compared to standard KANs, making them viable for complex vision tasks.
Maintained Expressiveness: The model preserves the theoretical approximation capabilities of KANs while improving accuracy on real-world datasets.
Improved Interpretability: The explicit nature of truncated power functions allows for clearer inspection of learned edge functions and knot contributions compared to B-splines or global spectral methods like SineKAN.

The authors conclude that TruKAN offers a practical path forward for KANs, demonstrating advantages beyond the constrained settings of prior studies. They acknowledge that while the method improves stability, challenges remain regarding the integration of other regularization techniques (like dropout) and the potential for mixed-precision training to further enhance efficiency. Future work is proposed to explore these areas and apply TruKAN to reinforcement learning and time-series forecasting.