Monotonic Kolmogorov-Arnold Networks: A Theoretical and Empirical Study of Monotonicity as an Inductive Bias

Problem Statement

Monotonicity is a critical inductive bias in domains such as tabular data, scientific modeling, and economics, where outputs are known to respond monotonically to specific inputs (e.g., higher debt implies higher default risk). While existing monotone neural networks (MNNs) exist—ranging from min-max architectures to deep lattice networks and normalizing flows—they typically rely on Multi-Layer Perceptrons (MLPs), lattices, or autoregressive flows. These architectures lack per-edge functional transparency, a key advantage of Kolmogorov–Arnold Networks (KANs), which expose how every input–output coordinate pair contributes to the output via learnable univariate splines.

The only prior attempt to combine KANs with monotonicity, MonoKAN, enforces constraints only on a restricted subset of parameters and requires a projection-style training procedure. Furthermore, MonoKAN has only been evaluated in supervised settings. There is a gap in the literature for a KAN variant that:

1. Guarantees hard monotonicity for all parameter values.
2. Is trainable via standard unconstrained gradient descent.
3. Is supported by a representation-theoretic guarantee regarding the cost of imposing monotonicity.

Methodology: MKAN

The authors propose MKAN (Monotonic Kolmogorov–Arnold Network), a KAN variant that enforces monotonicity by construction through three specific mechanisms:

1. Exponential Reparameterization of B-spline Coefficients: To ensure each spline $\phi'_{ij}$ is monotonically increasing, the authors enforce a strict ordering of B-spline coefficients ($\gamma^{(1)} < \gamma^{(2)} < \dots$). Instead of learning constrained coefficients directly, they introduce unconstrained parameters $\omega$ and define the coefficients via an exponential mapping: $\gamma^{(k)} = \omega^{(1)} + \sum_{t=2}^k \exp(\omega^{(t)})$. This guarantees the ordering condition is satisfied for all $\omega \in \mathbb{R}$.
2. Positive Edge Weights: The scaling weights for the spline outputs and the skip connections are exponentiated ($\exp(w)$) to ensure they remain strictly positive.
3. Monotone Base Activation: The skip connection uses a monotonically increasing activation function (ReLU).

The resulting layer output is defined as:
$$F'^{(j)}(x) = \sum_{i=1}^{N_{in}} \left( \exp(w^{(s)}_{ij}) \phi'_{ij}(x_i) + \exp(w^{(b)}_{ij}) \text{ReLU}(x_i) \right) + b_j$$
Because the composition of monotonically increasing functions is monotonically increasing, stacking these layers yields a globally monotone feature extractor. Crucially, this formulation allows training via standard unconstrained gradient descent without projection steps.

Key Contributions

1. Hard, Projection-Free Monotonicity in KANs (C1)
MKAN provides the first KAN variant where monotonicity is guaranteed for every parameter value. This eliminates the need for complex projection-based training or restricted parameter subsets, closing the gap left by MonoKAN.

2. Representation-Cost Theorem for Monotonicity (C2)
The paper presents a theoretical result (Theorem 3) characterizing the "cost" of enforcing monotonicity.

• Theorem: Any $C^K$ feature extractor $F: [-1, 1]^N \to \mathbb{R}^{N^*}$ that induces a ball-shaped semantic-neighborhood partition admits a monotone realization $F': [-1, 1]^N \to \mathbb{R}^{N'}$ that induces the same partition.
• Bound: The dimension of the monotone extractor is $N' = N^* + k$, where $k$ is the number of non-monotone coordinates in the original extractor. Since $0 \le k \le N^*$, the worst-case bound is $N' \le 2N^*$.
• Implication: This is an architecture-agnostic sizing rule, suggesting that a monotone encoder needs at most twice the feature dimension of a non-monotone one to preserve semantic neighborhood structures.

3. Empirical Validation (C3)
The authors validate MKAN across supervised and self-supervised regimes:

• Supervised: MKAN is competitive with state-of-the-art monotone networks (SMM, LMN) on the SMM/ICML-2024 benchmark suite (classification and regression tasks) while uniquely offering per-edge functional transparency.
• Self-Supervised:
  • On a controlled synthetic "monotone-generative" dataset (ellipses), MKAN recovers ground-truth factors with substantially higher Spearman alignment than KAN, MLP, and linear baselines.
  • On four real-world datasets (MNIST, Fashion MNIST, Dry Bean, MAGIC), a feature-size sweep validates the $2N^*$ prediction of Theorem 3, showing that MKAN with $2N$ features matches or exceeds the performance of KAN with $N$ features.

Results

• Function Approximation: On synthetic monotonic multivariate functions, MKAN achieves Median MSE comparable to the Smooth Min-Max (SMM) baseline and significantly outperforms vanilla KAN (which suffers from catastrophic overfitting as dimensionality increases).
• Tabular Benchmarks: On the SMM/ICML-2024 suite, MKAN achieves top or near-top performance. For example, it leads on COMPAS (69.6% vs. SMM 69.5%) and LoanDefaulter (65.49% vs. SMM-mini 65.47%).
• Self-Supervised Alignment: In the DIP-VAE framework on the synthetic ellipse dataset, MKAN achieves a Spearman correlation of 83.0 with ground-truth factors, significantly outperforming KAN (49.4) and MLP (40.1). While unconstrained models (KAN) achieve lower reconstruction error, they fail to recover the monotone structure of the data.
• Feature Size Validation: Experiments on real datasets confirm that MKAN with $2N$ features generally outperforms KAN with $N$ features, and MKAN with $N$ features often performs comparably to KAN with $N$ features, validating the theoretical bound.

Significance and Claims

The paper claims to recast monotonicity not as a categorical restriction that degrades performance, but as a quantifiable representation-cost trade-off. The primary significance lies in:

1. Uniqueness: MKAN is the only method that combines hard, unconstrained monotonicity (guaranteed for all parameters) with KAN's per-edge functional transparency.
2. Theoretical Grounding: It provides the first representation-cost theorem for monotonicity, offering a principled guideline for sizing monotone encoders (at most a 2x increase in dimension).
3. Practical Utility: By enabling standard gradient descent, MKAN makes monotone KANs easier to train and deploy than previous projection-based approaches.

The authors note limitations, including the lack of an adaptive grid range in MKAN, the fact that Theorem 3 is an existence result rather than a learning guarantee, and the assumption of ball-shaped neighborhoods. Future work suggested includes partial monotonicity via per-edge masks and extending theorems to ellipsoidal neighborhoods.