FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing

This paper introduces FS-KAN, a principled framework that constructs permutation equivariant and invariant Kolmogorov-Arnold Networks via function sharing, offering superior data efficiency and interpretability while maintaining the expressive power of standard parameter-sharing networks.

The Gist

Imagine you are trying to teach a robot to recognize patterns in a chaotic room full of scattered toys.

The Problem: The "Symmetry" Puzzle

In the real world, data often comes in groups where the order doesn't matter.

• If you have a bag of 5 red marbles, it doesn't matter if you pull them out one by one or all at once; the bag is still the same.
• If you have a photo of a crowd, swapping two people standing next to each other doesn't change the fact that it's a crowd.

In math and AI, this is called Permutation Symmetry. Traditional AI models (like standard neural networks) are like a robot that memorizes specific seats: "If a red marble is in seat #1, it's a win." If you move that marble to seat #2, the robot gets confused. To fix this, we usually force the robot to use Parameter Sharing. This is like telling the robot: "You must use the exact same rule for seat #1, seat #2, and seat #3." It works, but it's rigid.

The New Idea: KANs (The "Shape-Shifting" Robot)

Recently, a new type of AI called KAN (Kolmogorov-Arnold Network) became popular.

• Old AI (MLP): Think of this as a robot with fixed gears. It multiplies numbers by fixed weights. It's fast, but it's hard to understand why it made a decision.
• New AI (KAN): Think of this as a robot with shape-shifting arms. Instead of fixed gears, it uses flexible, learnable curves (functions) that can bend and twist to fit the data perfectly.
  • Benefit: It's much better at learning with very little data, and you can actually see the curves it learned, making it very transparent (interpretable).

The Gap: The Missing Link

Scientists realized: "Hey, KANs are great, but they don't know how to handle those 'order doesn't matter' groups (symmetries) yet."
Previous attempts to combine them were like trying to fit a square peg in a round hole—they only worked for specific, simple cases (like just images or just graphs). There was no general rulebook for making a "Symmetry-Aware KAN."

The Solution: FS-KAN (The "Shared Blueprint" Robot)

This paper introduces FS-KAN (Function Sharing KAN).

The Analogy: The Shared Blueprint
Imagine you are building a massive house with 1,000 identical rooms.

• Standard AI: You hire 1,000 different architects. Each one designs their room from scratch. If you swap Room 1 and Room 2, the house looks weird because the designs don't match.
• Old Symmetry AI (Parameter Sharing): You hire one architect and tell them, "Use the exact same blueprint for every room." This works, but the blueprint is rigid (just straight lines and fixed angles).
• FS-KAN (Function Sharing): You hire one master architect who creates a flexible, shape-shifting blueprint. They say, "I will use the same flexible curve for every room, but I'll let the curve bend slightly depending on the room's position."

How it works:
Instead of sharing fixed numbers (weights), FS-KAN shares entire mathematical functions.

• If the robot sees a red marble in position A, it applies "Curve X."
• If it sees a red marble in position B, it applies "Curve X" again.
• Because it's the same curve, the robot automatically understands that A and B are interchangeable.

Why is this a Big Deal?

1. It's a Universal Translator: The authors proved mathematically that FS-KANs are just as powerful as the old rigid methods. They can learn anything the old methods could, but they do it with the flexibility of KANs.
2. Data Starvation Superpower: In the real world, we often don't have millions of examples (like in medical imaging or rare event detection).
   • The Experiment: The authors tested FS-KAN on tasks like classifying 3D shapes (point clouds) and predicting movie ratings.
   • The Result: When data was scarce (low-data regime), FS-KAN crushed the competition. It learned the patterns with far fewer examples than traditional models because it didn't waste time learning that "swapping two items doesn't matter"—it was built into the architecture from day one.
3. It's Transparent: Because KANs use visible curves, you can look at the FS-KAN and see exactly how it's grouping the data. It's like looking at the robot's brain and seeing the logic flow, rather than a black box.

The Trade-off

There is a small catch. Because these "shape-shifting curves" are more complex to calculate than simple fixed gears, FS-KANs are a bit slower and use more computer memory.

• Verdict: If you have infinite data and a supercomputer, the old way might be fine. But if you have limited data and need a model that learns fast, understands itself, and respects the natural order of your data, FS-KAN is the clear winner.

Summary

FS-KAN is a new type of AI that combines the flexibility and transparency of the latest "shape-shifting" networks with the smart symmetry rules needed to handle unordered data. It's like giving a robot a flexible, shared blueprint that allows it to learn complex patterns from very few examples, making it perfect for real-world problems where data is scarce and messy.

Technical Summary

Here is a detailed technical summary of the paper "FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing".

1. Problem Statement

The paper addresses the challenge of designing neural network architectures that respect permutation symmetries (where the order of input elements does not matter, or specific permutations lead to predictable output transformations) while leveraging the recent advantages of Kolmogorov-Arnold Networks (KANs).

• Context: Traditional equivariant networks (e.g., for sets, graphs, or images) rely on parameter-sharing schemes (tying weights $W_{i,j} = W_{\sigma(i),\sigma(j)}$) to enforce symmetry. While effective, these are typically based on Multilayer Perceptrons (MLPs) with fixed scalar weights.
• The Gap: KANs, which replace scalar weights with learnable univariate functions, have shown superior interpretability and expressivity in standard settings. However, a principled framework for constructing equivariant KANs for arbitrary permutation groups was missing. Existing attempts were limited to specific data types (e.g., sets or continuous groups) and lacked a unified theoretical foundation for general permutation symmetries.
• Goal: To develop a general framework that constructs permutation-equivariant and invariant KAN layers using a function-sharing mechanism, ensuring the model is both theoretically sound and practically efficient.

2. Methodology: Function Sharing KAN (FS-KAN)

The authors propose FS-KAN, a framework that generalizes parameter-sharing to the KAN architecture by constraining the learnable functions rather than scalar weights.

A. Core Concept: Function Sharing

In a standard KAN layer $\Phi: \mathbb{R}^{n_{in}} \to \mathbb{R}^{n_{out}}$, the output is computed as:
$$\Phi(x)_q = \sum_{p=1}^{n_{in}} \phi_{q,p}(x_p)$$
where $\phi_{q,p}$ are learnable univariate functions.

For a permutation group $G \leq S_n$, the authors define a $G$-equivariant Function Sharing (FS) KA layer by imposing the constraint:
$$\phi_{q,p} = \phi_{\sigma(q), \sigma(p)}, \quad \forall \sigma \in G$$
This means that functions are "tied" across the matrix indices according to the group action. If two index pairs $(q, p)$ and $(q', p')$ belong to the same orbit under the group action, they share the exact same learnable function.

B. Theoretical Foundations

• Universality of FS: The paper proves that restricting attention to FS layers does not lose generality. Proposition 2 states that any $G$-equivariant KA layer can be represented by an equivalent $G$-equivariant FS-KA layer.
• Expressive Power Equivalence: A key theoretical contribution is Proposition 6 and 7, which establish that FS-KANs and standard parameter-sharing MLPs have equivalent expressive power in the uniform approximation sense.
  • Any parameter-sharing MLP can be exactly implemented by an FS-KAN (using splines).
  • Any FS-KAN can be approximated arbitrarily well by a parameter-sharing MLP.
  • Implication: This allows the transfer of well-known expressivity results (e.g., universality of DeepSets, discriminative power of $k$-WL tests for graphs) directly to FS-KANs.

C. Efficient FS-KA Layers

Standard FS-KANs can be computationally expensive because they apply functions independently to every input-output pair before summing. To address this, the authors introduce Efficient FS-KA layers:

• Mechanism: They commute the summation (pooling) with the function application. Instead of applying functions to individual elements and then summing, they aggregate inputs (e.g., sum or mean pooling) first and then apply shared functions to the aggregated values.
• Benefit: This significantly reduces the number of function evaluations and memory usage (especially during backpropagation) while preserving equivariance.

D. Generalization to Complex Symmetries

The framework extends to:

• Direct Product Symmetries ($G \times H$): Handling data like matrices where rows and columns have independent symmetries (e.g., user-item interactions). This involves "external sharing" of sub-layers and "internal sharing" of functions within sub-layers.
• High-Order Tensors: Extending the logic to $k$-order tensors (e.g., hypergraphs, 3D meshes) by defining function sharing over tensor index orbits.

3. Key Contributions

1. FS-KAN Framework: A principled method for constructing equivariant and invariant KAN layers for arbitrary permutation symmetry groups via function sharing.
2. Theoretical Unification: Proof that FS-KANs possess the same expressive power as parameter-sharing MLPs, enabling the transfer of universality theorems and discriminative power bounds (like $k$-WL) to the KAN domain.
3. Efficient Variant: An optimized layer architecture that reduces computational complexity by aggregating inputs before function application, making FS-KANs viable for larger datasets.
4. Enhanced Interpretability: Demonstrated that FS-KANs not only maintain the interpretability of standard KANs (visualizable learned functions) but enhance it by making the equivariant structure explicit (shared functions across symmetric edges).

4. Experimental Results

The authors evaluated FS-KANs on three distinct tasks involving permutation symmetries:

A. Signal Classification (Multiple Measurements)

• Task: Classifying periodic signals from noisy, permuted measurements ($S_n \times C_T$ symmetry).
• Result: FS-KANs significantly outperformed standard parameter-sharing baselines (DeepSets) in low-data regimes (60–1200 samples), achieving higher accuracy with fewer parameters. The efficient variant was faster to train than the full FS-KAN.

B. Point Cloud Classification (ModelNet40)

• Task: 3D object classification from point sets ($S_n$ symmetry).
• Result: FS-KANs consistently outperformed DeepSets, Point Transformers, and non-invariant KANs, particularly when the number of training samples or points per cloud was limited.
• Continual Learning: In a continual learning experiment (learning Task A then Task B), FS-KANs demonstrated superior robustness against catastrophic forgetting compared to DeepSets, retaining knowledge of the first task better while adapting to the second.

C. Semi-Supervised Rating Prediction (Matrix Completion)

• Task: Predicting missing user-item ratings ($S_n \times S_m$ symmetry).
• Result: On sparse datasets (extreme data scarcity), FS-KANs achieved lower RMSE than baselines (SSEM), highlighting their data efficiency.

5. Significance and Impact

• Bridging Two Paradigms: The paper successfully bridges the gap between Geometric Deep Learning (symmetry-aware models) and Kolmogorov-Arnold Networks (interpretable, function-based models).
• Data Efficiency: The results strongly suggest that FS-KANs are the preferred architecture for low-data regimes where data symmetries are present. They learn the underlying structure more efficiently than standard MLPs.
• Interpretability: By enforcing function sharing, the learned models become more interpretable. Researchers can visualize the specific univariate functions shared across symmetric parts of the data, providing insights into how the model respects the data's geometry.
• Theoretical Guarantee: The equivalence proof ensures that adopting FS-KANs does not sacrifice the theoretical guarantees (universality, expressivity) established for decades in equivariant MLP research.

In conclusion, FS-KAN offers a robust, theoretically grounded, and empirically superior alternative to traditional equivariant MLPs, particularly excelling in scenarios with limited data and requiring high interpretability.