🤖 machine learning

KAN-Robust-Bench: A Benchmark for Evaluating the Robustness of Kolmogorov-Arnold Networks

This paper introduces KAN-Robust-Bench, a benchmark that evaluates the certified and empirical robustness of Kolmogorov-Arnold Networks against strong evasion attacks to identify optimal defense strategies and architectures.

Original authors: Mohammad Meymani, Roozbeh Razavi-Far

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammad Meymani, Roozbeh Razavi-Far

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world, artificial intelligence has become a quiet partner in daily life, sorting our emails, guiding our cars, and diagnosing illnesses. These systems learn by finding patterns in vast amounts of data, much like a child learns to recognize a cat by seeing many pictures of cats. However, this learning process has a hidden weakness. Just as a human can be tricked by a clever illusion, these computer models can be fooled by tiny, almost invisible changes to the images they see. A researcher might add a few specks of digital noise to a picture of a stop sign—changes so small a human eye would never notice it—and the computer might suddenly see it as a speed limit sign. This vulnerability is known as an "evasion attack," and it poses a serious security risk for any system that relies on artificial intelligence to make safe decisions.

To understand how to protect these systems, scientists are constantly testing new types of computer brains. For decades, the standard design has been a specific kind of network that processes information in layers. Recently, a different design called a Kolmogorov-Arnold Network, or KAN, has emerged as a promising alternative. While the old design stacks layers of fixed functions, the KAN uses flexible, learnable curves that can adapt more naturally to complex shapes in data. The big question for researchers was whether this new, more flexible design was also more fragile. If a model is better at learning, is it also easier to trick? A team of researchers from the University of New Brunswick set out to answer this by putting several new KAN designs through a rigorous series of stress tests, pitting them against the strongest known methods for fooling artificial intelligence.

The researchers focused on three specific versions of the KAN architecture, each built to handle visual tasks like recognizing objects in photos. They tested these models on two standard sets of images: one containing common objects like cars and animals, and another containing numbers found on street signs. To see how well these models held up, the team subjected them to three different types of digital attacks. The first type was a quick, one-step trick that nudged the image in the direction where the model was most likely to make a mistake. The second type was a more patient, multi-step attack that refined the trick over and over until it found the perfect way to confuse the model. The third type was a highly sophisticated optimization attack that searched for the smallest possible change needed to cause a failure.

To defend against these tricks, the team tried three different strategies. The first was to train the models by showing them both normal pictures and the tricky, altered versions, forcing the computer to learn how to ignore the noise. The second strategy involved adding a layer of random static, or noise, to the images before the model looked at them, effectively blurring the sharp edges that attackers rely on. The third method involved mathematically proving that the model's decision would not change even if the image was altered within a certain limit, creating a safety net that guaranteed stability.

The results painted a clear picture of how these new networks behave under pressure. When the models were left without any special protection, they were surprisingly fragile. Even the most advanced KAN designs could be fooled by the simplest attacks, with their accuracy dropping dramatically as the noise increased. However, when the researchers applied the strategy of training the models with the tricky images, the results changed completely. This method, known as adversarial training, proved to be the most powerful shield. The models that underwent this training maintained high accuracy even when faced with the strongest, multi-step attacks. For example, on the dataset of street signs, a model trained this way kept its accuracy above 67 percent even when the attack was at its maximum strength, whereas the untrained version collapsed to near zero.

The other two defense strategies offered different kinds of protection. The method that added random noise to the images did not stop the models from being fooled as effectively as the training method did, but it provided a different kind of safety. It offered a mathematical guarantee that the model's answer would not change if the image was altered by a specific, small amount. This is valuable because it provides a known boundary of safety, even if the model's overall performance against the strongest attacks was not as high as the trained models. The third strategy, which used mathematical bounds to check for stability, also improved the models' resilience, but it generally fell short of the protection offered by training the models with the attacks directly.

One of the most interesting findings was that the choice of the model's internal structure mattered just as much as the defense strategy. Among the three different KAN designs tested, one specific architecture consistently outperformed the others. This particular design, which combined different ways of mixing image features, proved to be the most robust across all the tests. It held up better against the quick tricks, the patient multi-step attacks, and the sophisticated optimization attacks. This suggests that simply switching to a new type of network is not enough; the specific way that network is built determines how well it can withstand an attack.

The study also revealed that the difficulty of the task changed the outcome. The models performed significantly better on the dataset of street signs than on the dataset of common objects. On the street signs, even the untrained models were surprisingly good at resisting attacks, and the protected models reached accuracy levels above 90 percent. This indicates that some types of visual data are naturally easier for these systems to handle securely than others.

Ultimately, the research shows that while these new Kolmogorov-Arnold networks are powerful tools, they are not immune to deception. The most reliable way to protect them is to teach them what an attack looks like during their learning phase. While other methods can provide mathematical guarantees of safety, they do not always translate to the same level of real-world performance against the strongest tricks. The work provides a clear roadmap for developers: if they want these new, flexible networks to be secure, they must build their defenses directly into the training process, ensuring the models learn to see through the noise rather than just ignoring it.

Technical Summary: KAN-Robust-Bench

Problem Statement

While Kolmogorov-Arnold Networks (KANs) have emerged as a promising alternative to Multi-Layer Perceptrons (MLPs) for various tasks—including time-series analysis, graph learning, and computer vision—their vulnerability to adversarial threats remains under-explored. Existing literature primarily compares KANs against traditional architectures like MLPs and Convolutional Neural Networks (CNNs), often concluding that KANs may be more susceptible to noise and attacks. However, there is a lack of systematic evaluation regarding the robustness of different KAN architectures against each other under identical adversarial conditions. Furthermore, the trade-offs between empirical robustness and certified robustness within the KAN family, specifically when employing defense strategies like Adversarial Training (AT), Randomized Smoothing (RS), and Interval Bound Propagation (IBP), have not been comprehensively analyzed.

Methodology

The authors propose KAN-Robust-Bench, a benchmark designed to evaluate the certified and empirical robustness of state-of-the-art KAN architectures. The study focuses on white-box evasion attacks, assuming the adversary has full knowledge of the model's architecture, parameters, and gradients.

Architectures Evaluated

Three specific KAN-based architectures were selected for their state-of-the-art performance in computer vision:

KAN-Mixers: Extends the MLP-Mixer architecture by replacing standard MLP layers with KAN layers, utilizing token- and channel-mixing.
KANICE: A hybrid architecture combining KAN and CNN, featuring Interactive Convolution Blocks (ICB) for spatial feature extraction and KAN linear layers for function approximation.
PoolKANNeXt: A hybrid KAN-CNN architecture inspired by ConvNeXt, utilizing pooling-based feature mixing and dual-activation KAN blocks.

Datasets

Experiments were conducted on CIFAR-10 and SVHN datasets, with data normalized using standard mean and variance vectors for each dataset.

Defense Strategies

The study evaluates three primary defense mechanisms:

Adversarial Training (AT): Models are trained on a mixture of benign and adversarial samples generated via Projected Gradient Descent (PGD).
Randomized Smoothing (RS): A smoothed classifier is constructed by injecting Gaussian noise into inputs and aggregating predictions via Monte Carlo sampling. This provides $\ell_2$ -certified robustness.
Interval Bound Propagation (IBP): A certified defense where input perturbation bounds are propagated through the network. The training loss is a linear combination of clean loss and IBP loss to balance accuracy and robustness.

Attack Configurations

The models were tested against strong white-box evasion attacks:

FGSM: Single-step attack with perturbation sizes ( $\epsilon$ ) ranging from 0.01 to 0.09.
PGD: Multi-step iterative attacks, including standard, multi-step (with restarts), and adaptive (with Expectation-over-Transformation) variants.
Carlini & Wagner (C&W): An optimization-based attack using the $\ell_2$ norm.

Metrics

Performance was measured using Standard Accuracy (SA) on clean data and Robust Accuracy (RA) on adversarial data. For RS and IBP, Certified Accuracy (CA) and Certified Radius were reported to provide mathematical guarantees of robustness within specific perturbation bounds.

Key Contributions

Systematic Evaluation: The paper provides the first systematic comparison of robustness within the KAN family (KAN-Mixers, KANICE, PoolKANNeXt) across CIFAR-10 and SVHN, rather than just comparing KANs to MLPs/CNNs.
Defense Strategy Analysis: It rigorously evaluates the impact of AT, RS, and IBP on KAN architectures, identifying optimal defense strategies for different attack types.
Certified Robustness Reporting: The study reports $\ell_2$ -certified robustness under randomized smoothing and $\ell_\infty$ -certified robustness under IBP, offering formal guarantees alongside empirical results.
Benchmarking: It establishes a baseline for future research on KAN robustness, highlighting the specific architectural and defense choices that influence model resilience.

Experimental Results

Certified Robustness (Randomized Smoothing)

Under $\ell_2$ -norm certification:

PoolKANNeXt demonstrated the strongest certified robustness on CIFAR-10, achieving a Certified Accuracy (CA) of ~80.33% at a radius of 2/255 and a mean radius of 1.18.
On SVHN, both PoolKANNeXt and KANICE showed competitive certified robustness, with CA values exceeding 92% at smaller radii.
KAN-Mixers showed lower certified robustness compared to the other two architectures on CIFAR-10 but performed competitively on SVHN.

Empirical Robustness

Against FGSM Attacks:

Adversarial Training (AT) consistently provided the strongest protection against gradient-based attacks. For instance, on CIFAR-10, PoolKANNeXt-AT maintained 45.56% robust accuracy at $\epsilon=0.09$ , whereas its undefended counterpart dropped to 7.54%.
RS and IBP offered modest improvements over undefended models but were significantly outperformed by AT in terms of empirical robustness against FGSM.
SVHN models generally exhibited higher robustness than CIFAR-10 models across all strategies.

Against PGD Attacks:

AT remained the dominant defense. PoolKANNeXt-AT achieved the highest robustness (~54.6%) on CIFAR-10, while undefended models dropped below 8%.
RS showed mixed results; while it improved robustness over plain models, it remained inferior to AT.
IBP provided moderate improvements but struggled to match the empirical performance of AT against iterative attacks.

Against C&W ( $\ell_2$ ) Attacks:

A distinct trend emerged where Randomized Smoothing became highly competitive, often outperforming AT.
On SVHN, KAN-Mixers-RS achieved the highest robustness (72.58%), slightly exceeding both plain and adversarially trained variants. This suggests that the smoothing mechanism is particularly effective against $\ell_2$ -norm perturbations.

Architectural Insights

The results indicate that robustness is highly architecture-dependent. For example, PoolKANNeXt generally excelled in certified robustness and PGD resistance, while KAN-Mixers showed superior performance against C&W attacks when combined with RS. The study highlights that architectural components (e.g., interactive convolutional blocks vs. mixer-style token processing) influence robustness differently, even within the same KAN framework.

Significance and Claims

The paper claims that while Adversarial Training is the most effective strategy for defending KANs against gradient-based attacks (FGSM and PGD), Randomized Smoothing offers a competitive, and sometimes superior, defense against $\ell_2$ -norm attacks like C&W. The study emphasizes that architectural choices significantly impact both certified and empirical robustness, suggesting that optimizing hyperparameters and selecting specific KAN variants are critical for robustness.

The authors conclude that their work provides a focused evaluation of how defense strategies and architectural choices influence robustness among KAN-based models. They modestly suggest that future research should explore more diverse certification techniques (such as verification and convex relaxation) and investigate a wider range of KAN architectures to further understand the limitations and strengths of these models.