⚡ electrical engineering

Ultrafast On-Chip Online Learning via Spline Locality in Kolmogorov-Arnold Networks

This paper demonstrates that Kolmogorov-Arnold Networks (KANs) enable ultrafast, model-free online learning on FPGAs with sub-microsecond latencies by leveraging B-spline locality for sparse updates and inherent robustness to fixed-point quantization, outperforming conventional MLPs in resource-constrained, high-frequency control systems.

Original authors: Duc Hoang, Aarush Gupta, Philip Harris

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Duc Hoang, Aarush Gupta, Philip Harris

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Learning at the Speed of Light

Imagine you are trying to steer a spaceship that is moving so fast that if you wait even a fraction of a second to check your instruments, you've already crashed. This is the reality for systems like quantum computers and nuclear fusion reactors. They change so quickly (in microseconds) that traditional computers are too slow.

Usually, when a computer learns, it sends data to a big "brain" (like a server or a cloud computer), waits for the brain to do the math, and then sends the answer back. By the time the answer arrives, the situation has already changed.

The Goal: The researchers wanted to build a "brain" that lives entirely inside the machine (on a chip) and can learn and adapt instantly, without ever leaving the device.

The Old Way vs. The New Way

To solve this, the team compared two types of learning models:

The Old Way (MLPs - Multi-Layer Perceptrons): Think of this like a giant, dense web. To learn something new, the whole web has to shake and rearrange itself. Every single thread in the web is connected to every other thread.
- The Problem: On a tiny chip with limited space and power, trying to shake this whole web is too slow and uses too much energy. It's like trying to fix a single loose thread in a massive tapestry by rewiring the whole thing. Also, when you try to do the math with limited precision (like using a ruler with only big markings), this web gets wobbly and breaks.
The New Way (KANs - Kolmogorov-Arnold Networks): Think of this like a modular Lego wall made of flexible, stretchy rubber bands (called B-splines).
- The Magic Trick: When the wall needs to learn something new, you don't have to touch the whole wall. You only touch the few specific rubber bands right where the change is happening.
- The Paper's Claim: Because the rubber bands only affect their immediate neighbors (a property called "locality"), the computer only has to do a tiny amount of math for each update. This makes it incredibly fast and efficient.

Why KANs are Better for Tiny Chips

The paper highlights three main reasons why this "Lego wall" approach wins on hardware chips:

The "Sparse" Update: Imagine you are painting a mural.
- MLP: To fix a smudge, you have to repaint the entire wall.
- KAN: You only repaint the tiny square where the smudge is.
- Result: The KAN uses far less computing power and memory, allowing it to fit on small chips (FPGAs) that can't handle the heavy load of the MLP.
The "Stable" Math:
- MLP: If you try to do math with a low-precision ruler (fixed-point numbers), the MLP's calculations can spiral out of control, like a microphone screeching when it gets too close to a speaker.
- KAN: The rubber bands naturally keep the math within safe, predictable limits. Even with a low-precision ruler, the KAN stays stable and doesn't crash.
Growing Without Breaking:
- MLP: To make the MLP smarter, you have to add more threads to the whole web, which makes it slower and heavier.
- KAN: To make the KAN smarter, you just add more "grid points" to the rubber bands. The amount of math needed for each update stays the same, but the model gets much better at guessing. It's like adding more rungs to a ladder without making the ladder heavier to climb.

The Experiment: Putting it to the Test

The researchers built these models on a FPGA (a type of chip you can reprogram on the fly) and tested them in three high-speed scenarios:

Drifting Sensors: Imagine a sensor that slowly changes its calibration over time. The KAN tracked the changes instantly, while the MLP got confused and lost track.
Quantum Reading: They tried to read the state of a quantum bit (qubit) which is very noisy and changes shape. The KAN learned to read it correctly in real-time, while the MLP failed or needed huge amounts of power.
Robot Control: They tried to control a double-pendulum robot (Acrobot) where the weights kept changing randomly. The KAN learned to balance it quickly, while the MLP struggled to adapt.

The Bottom Line

The paper claims to be the first to show that a machine can learn and adapt in less than a microsecond (a millionth of a second) entirely on a chip, without needing a supercomputer.

By using the "Lego wall" approach (KANs) instead of the "dense web" (MLPs), they achieved:

Speed: Updates happen in under 100 nanoseconds.
Efficiency: They used 3 to 4 times less hardware resources.
Stability: The system didn't crash even when using simplified math.

In short, they found a way to make AI "learn on the fly" so fast that it can keep up with the fastest physical systems in the universe, something that was previously impossible with standard AI models.

Technical Summary: Ultrafast On-Chip Online Learning via Spline Locality in Kolmogorov–Arnold Networks

1. Problem Statement

The paper addresses the critical challenge of ultrafast model-free online learning in high-frequency systems where adaptation must occur on sub-microsecond timescales. Such requirements are essential for domains like quantum computing control, nuclear fusion plasma diagnostics, and high-speed communications.

Current approaches face significant bottlenecks:

Host-Accelerator Latency: Traditional training loops involving off-chip gradient computation and parameter updates (e.g., via PCIe) are too slow; by the time parameters return, the system's operating conditions may have already shifted.
MLP Limitations: Conventional Multi-Layer Perceptrons (MLPs) are inefficient and numerically unstable under the strict constraints of on-chip training:
- Resource Scaling: MLPs require dense gradient updates over all parameters, leading to linear growth in on-chip resources (DSP, LUT, FF) and latency as model capacity increases.
- Fixed-Point Instability: Reduced precision (fixed-point arithmetic) often destabilizes gradient-based optimization in MLPs due to unbounded activation and gradient scaling.
- Memory Constraints: Storing dense activations and weights for backpropagation exceeds the limited on-chip memory of FPGAs.

2. Methodology

The authors propose leveraging Kolmogorov–Arnold Networks (KANs) as a hardware-native alternative to MLPs, specifically exploiting the locality of B-spline basis functions.

Theoretical Motivation:
- Sparse Updates: Unlike MLPs which update all weights, KANs utilize B-splines with local support. For a spline order $S$ , only $S+1$ coefficients are active per input sample, regardless of the total grid size $G$ . This results in a per-sample update complexity of $O(S+1)$ rather than $O(N)$ , where $N$ is the total parameter count.
- Capacity Scaling: Increasing the grid resolution $G$ improves approximation quality without increasing the active computation per sample. This decouples capacity from computational cost.
- Quantization Robustness: KAN activations are convex combinations of learned coefficients, bounded by $\min(W_i) \leq \phi(x) \leq \max(W_i)$ . Similarly, gradients are bounded by the B-spline envelope $[0, 1]$ . This inherent bounding prevents the numerical instability common in MLPs under aggressive fixed-point quantization.
Hardware Implementation:
- Platform: The system is implemented on an AMD Virtex UltraScale+ XCVU13P FPGA.
- Architecture: Custom kernels are synthesized using Vitis HLS with a target clock of 200 MHz.
- KAN Kernel:
  - Inputs are mapped to grid cell indices ( $k$ ) and LUT indices ( $u$ ).
  - Pre-computed B-spline values and derivatives are stored in small ROM LUTs.
  - Forward and backward passes update only the active coefficients ( $S+1$ ) per edge, avoiding global memory access.
  - All state (parameters, context) resides on-chip (LUT/FF/BRAM); no external DRAM is used.
- MLP Baseline: Implements dense matrix multiplications and global weight updates with the same streaming interface and fixed-point constraints for fair comparison.

3. Key Contributions

First Sub-Microsecond Model-Free Online Learning: The work demonstrates the first implementation of model-free online learning with deterministic latencies below 100 ns (sub-microsecond) on-chip.
Hardware-Aware KAN Analysis: It identifies and exploits B-spline locality to achieve sparse updates, challenging the perception that KANs are hardware-inefficient due to recursive evaluation.
Fixed-Point Stability: It provides theoretical proofs and empirical evidence that KANs are inherently robust to fixed-point quantization, enabling stable training where MLPs diverge.
Custom FPGA Implementation: A fully on-chip training pipeline (forward, backward, and parameter update) is realized, eliminating host-accelerator communication bottlenecks.

4. Experimental Results

The authors evaluate the system on three fully online benchmarks using fixed-point arithmetic:

Adaptive Function Approximation (Drifting Regression):
- KANs track regime changes (concept drift) with significantly lower cumulative regret than parameter-matched MLPs.
- Under fixed-point quantization (2 integer bits), KANs remain stable, while MLPs exhibit a "precision cliff," requiring much higher bitwidths to stabilize.
Adaptive Single-Shot Qubit Readout:
- In a non-linearly separable, drifting IQ classification task, KANs maintain high running accuracy (>90%) with grid size scaling.
- MLPs fail to converge or diverge under similar fixed-point constraints, even with increased parameter counts.
Non-Stationary Acrobot Control:
- In a reinforcement learning setting with randomized dynamics per episode, a fixed-point KAN Actor-Critic agent reaches the solved regime in ~300 episodes.
- MLP-based Actor-Critics fail to adapt, even with 10x the parameter budget, unless using a more stable (but slower) DQN-style update rule.
Scalability:
- KANs achieve sub-100 ns forward and backward latencies across tasks.
- Resource usage (DSP/LUT/FF) scales with the active set size ( $S+1$ ) rather than total parameters, allowing capacity increases via grid size $G$ with near-constant latency.
- Post-route validation confirms that designs meet timing constraints with positive slack, achieving end-to-end latencies well below 1 µs.

5. Significance and Claims

The paper claims that Kolmogorov–Arnold Networks are uniquely suited for ultrafast on-chip online learning, a regime where MLPs fundamentally struggle.

Architectural Advantage: By replacing dense, interference-prone gradient updates with sparse, targeted coefficient adjustments, KANs enable favorable resource scaling.
Hardware-Native Efficiency: The work argues that modern ML stacks are optimized for dense linear algebra (GPUs), overlooking the benefits of localized basis-function learning. Custom hardware supporting spline evaluation makes KAN inference and sparse updates "hardware-native."
Impact: This approach enables reliable, model-free adaptation for emerging applications requiring sub-microsecond response times, such as quantum control, plasma systems, and high-speed networking, without the latency penalties of host-accelerator loops.

The authors remain modest regarding optimization, noting that while High-Level Synthesis (HLS) was used for rapid prototyping, manual RTL refinement could further improve efficiency, and the current focus is on architectural merits and order-of-magnitude comparisons rather than extreme low-level hardware tuning.