Is the Statistical Advantage Worth the Cost? An Empirical Comparison of KANs and MLPs for Structured Data Classification

Technical Summary: An Empirical Comparison of KANs and MLPs for Structured Data Classification

Problem Statement
Structured tabular data remains a dominant input format in production machine learning systems across healthcare, finance, and other domains. While traditional ensemble methods (e.g., Random Forests, Gradient Boosted Trees) are often preferred for these tasks, there is growing interest in deep learning architectures. The Multi-Layer Perceptron (MLP) serves as the standard baseline, yet its reliance on fixed activation functions and scalar weights can limit its ability to capture localized nonlinear effects without increasing network depth or width. Recently, Kolmogorov-Arnold Networks (KANs) have emerged as a theoretically motivated alternative, replacing fixed node activations with learnable univariate functions (splines) on network edges. However, it remains an open empirical question whether this architectural shift translates to a meaningful, computationally justified performance advantage on real-world tabular tasks, particularly given the lack of standardized comparisons across diverse classification paradigms (binary, multiclass, multilabel, and ordinal).

Methodology
This study conducted a rigorous, standardized benchmarking of KANs against MLPs across twelve publicly available datasets. The experimental design prioritized fairness and reproducibility by adhering to the following protocols:

• Datasets: Twelve datasets were selected from UCI, Kaggle, and the MULAN library, covering four task types: binary, multiclass, multilabel, and ordinal classification.
• Preprocessing: A uniform pipeline was applied, including handling missing values (imputation or deletion), one-hot encoding for nominal attributes, and standardization of numerical features. Data was split into training, validation, and testing sets using stratified sampling.
• Model Configuration: Both models utilized "vanilla" configurations with fixed, untuned hyperparameters to isolate architectural differences. MLPs used ReLU activations, while KANs employed learnable spline functions (B-splines). Architectures were matched in terms of layer depth and width, with specific adjustments for multilabel tasks to manage computational complexity.
• Evaluation Metrics: Performance was assessed using Test Accuracy and F1-Score (weighted for class imbalance). To ensure robustness, models underwent five independent execution runs.
• Statistical Analysis: Paired hypothesis testing (t-tests or Wilcoxon signed-rank tests) was conducted at $\alpha = 0.05$ to determine statistical significance. Effect sizes were calculated using Cohen's $d$ and rank-biserial correlation ($r_{rb}$).

Key Results
The empirical results reveal a nuanced trade-off between predictive performance and computational cost:

1. Performance Superiority:

   • Aggregate: KANs statistically outperformed MLPs across all datasets combined ($p=0.001$), achieving higher test accuracy in 9 out of 12 datasets and higher F1-Scores in 10 out of 12.
   • Task-Specific: KANs demonstrated significant superiority in binary ($p=0.001$, $d=-1.09$) and multiclass ($p=0.001$, $r_{rb}=-1.00$) domains. However, for multilabel and ordinal tasks, the performance differences were not statistically significant, with effect sizes ranging from small to medium.
   • Convergence: KANs generally reached performance limits faster and stabilized earlier than MLPs in most datasets, except for the multilabel Emotions, Birds, and Enron datasets.

2. Computational Cost:

   • Parameters: KANs require substantially more parameters than MLPs. Due to the replacement of scalar weights with spline functions (grid size $G$ and order $k$), KANs consistently required approximately 16 times more parameters than their MLP counterparts in standard tasks.
   • Efficiency: The computational overhead is significant. KAN training times frequently exceeded 30 seconds (often reaching minutes), whereas MLP training rarely exceeded five seconds. Similarly, KAN inference latency frequently exceeded 100 ms, compared to ~3 ms for MLPs.

Key Contributions

• Standardized Benchmarking: This study provides one of the first comprehensive comparisons of KANs and MLPs across a diverse spectrum of tabular classification tasks (including under-explored multilabel and ordinal settings) under strictly controlled, non-tuned conditions.
• Statistical Rigor: Beyond raw accuracy, the paper establishes the statistical significance of KAN's advantages and quantifies the magnitude of these gains through effect size analysis.
• Cost-Benefit Framework: The work explicitly weighs the predictive gains against the computational penalties, offering a data-driven framework for architectural selection.

Significance and Claims
The paper concludes that while KANs offer a statistically significant and theoretically superior approach to function approximation for structured data, the practical utility of this advantage is context-dependent.

• High-Precision Applications: For scenarios where predictive precision is critical and computational resources are abundant (e.g., medical diagnosis, financial forecasting), the statistical advantage of KANs justifies the additional computational cost.
• Resource-Constrained Environments: For real-time edge computing or low-latency systems, the "medium" effect size ($d = -0.46$) of the aggregate improvement does not outweigh the efficiency losses. In these cases, MLPs remain the robust and efficient choice.

The authors emphasize that the decision to adopt KANs should be driven by specific operational requirements regarding the tolerance for computational overhead versus the need for marginal accuracy gains. The study acknowledges limitations, including the use of fixed hyperparameters (which may not reflect optimal tuned performance) and reduced training epochs for multilabel tasks, suggesting future work should explore additional data modalities and deeper hyperparameter optimization.