🔢 mathematics

Explicit Construction of Approximate Kolmogorov Superpositions with C2 Smoothness

This paper presents an explicit construction of approximate Kolmogorov superpositions using $C^2$ -smooth inner and outer functions to approximate arbitrary $\alpha$ -Hölder continuous functions with an error rate of $N^{-\alpha}$ , thereby overcoming the pathological behaviors of classical representations while retaining their exact representation strategy.

Original authors: Lunji Song, Zilan Cheng, Juan Diego Toscano, Li-Lian Wang

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lunji Song, Zilan Cheng, Juan Diego Toscano, Li-Lian Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Unraveling a 3D Puzzle with 1D Strings

Imagine you have a complex, multi-dimensional object, like a giant, intricate 3D sculpture (representing a function with many variables, like $x_1, x_2, \dots, x_d$ ). For decades, mathematicians have known a "magic trick" (the Kolmogorov Superposition Theorem) that says you can completely describe this 3D sculpture using only a stack of simple, one-dimensional strings (functions of a single variable).

However, there was a catch. The original "strings" used in this magic trick were wild and jagged. They were so rough and broken that they had no smooth curves, making them impossible to use in modern tools like computer simulations or neural networks (which prefer smooth, flowing lines).

This paper presents a new, smoother version of the magic trick. The authors have explicitly built a set of "smooth strings" that can still reconstruct the complex 3D sculpture, but without the jagged, broken edges. They prove that these new strings work well and can approximate any smooth or slightly bumpy shape with high accuracy.

The Problem: The "Wild" Strings

In the original theory, the "inner strings" (the ones that take the input) were like staircases made of broken glass.

They went up and down in tiny, sharp steps.
They were so jagged that if you tried to measure their slope (derivative) at most points, it was zero or undefined.
Because they were so "wild," computers couldn't use them effectively for learning or prediction.

The paper asks: Can we replace these broken-glass strings with smooth, polished ones without losing the ability to reconstruct the 3D shape?

The Solution: Building Smooth "Inner" Strings

The authors designed a new type of "inner string" (called an inner function) that is $C^2$ -smooth.

What does $C^2$ -smooth mean? Imagine driving a car.
- A jagged string is like hitting a pothole: you jerk forward, then stop, then jerk again.
- A $C^1$ -smooth string is like a smooth road: you don't jerk, but the steering wheel might still turn sharply.
- A $C^2$ -smooth string is like a perfectly banked racetrack. Not only is the road smooth, but the curvature of the road changes smoothly too. You can drive on it without any sudden jolts or sharp turns.

How did they build it?
Instead of using broken steps, they used a "glue" made of special mathematical shapes (polynomials and sine waves).

The Gaps: They created tiny gaps in the road where the string gently curves up (using a "smeared-out" shape).
The Flat Parts: Between the gaps, the string stays mostly flat but still moves forward.
The Result: A single, continuous, smooth line that never stops moving forward (strictly increasing) and has no sharp corners.

The "Map" Problem: Organizing the Chaos

Once you have smooth strings, you have to combine them to map the 3D world into a 1D line. This is like trying to pack a messy room into a single long hallway.

If you pack things randomly, items from different parts of the room might end up next to each other in the hallway, causing a mess (mathematicians call this "dislocation").

The Authors' Fix:
They figured out a precise recipe for how to mix the strings.

They assigned different "weights" (called $\lambda$ ) to each dimension.
They proved that if you choose these weights just right (based on the size of the gaps), the "hallway" will stay organized.
The Analogy: Imagine sorting books by height. If you just throw them in a pile, a short book might end up next to a tall one. But if you use a specific sorting rule (their mathematical recipe), every short book stays near other short books, and every tall book stays near tall books. This ensures the 1D map preserves the structure of the 3D object.

The "Outer" Strings: Reading the Map

Once the 3D object is flattened into a 1D line (the "inner" part), you need to read the value at that point to get the final answer. This is the outer function.

The authors built these outer functions by looking at the "centers" of the packed 3D blocks.
They used a smooth interpolation method (connecting the dots with smooth curves) to create a function that can read the 1D line and output the correct value for the original 3D shape.

The Results: How Good Is It?

The paper proves two main things:

Accuracy: The new smooth construction can approximate any function with a certain level of smoothness (called $\alpha$ -Hölder continuous) with an error that shrinks predictably as you add more "strings" (increasing $N$ ). The error gets smaller at a rate of $O(N^{-\alpha})$ .
Verification: They ran computer tests on functions with up to 9 dimensions. The results matched their mathematical predictions perfectly, showing that the error decreased exactly as fast as they said it would.

Why This Matters (According to the Paper)

It's Practical: Unlike the original "wild" functions, these new smooth functions can actually be used in neural networks (computer learning systems).
It's Explicit: The authors didn't just say "it exists"; they gave the exact formulas and steps to build these functions.
It Solves a Long-Standing Question: For years, researchers wondered if you could make these smooth versions without breaking the magic of the original theorem. This paper says "Yes," and shows exactly how.

In summary: The authors took a mathematical magic trick that used jagged, broken strings and replaced them with smooth, polished strings. They proved that these new strings can still reconstruct complex 3D shapes perfectly, making the theory ready for real-world computer applications.

Technical Summary: Explicit Construction of Approximate Kolmogorov Superpositions with $C^2$ -Smoothness

Problem Statement
The Kolmogorov Superposition Theorem (KST) establishes that any continuous multivariate function can be represented as a superposition of continuous univariate functions. However, existing constructive proofs (e.g., by Sprecher, Kahane, and Lorentz) rely on inner functions that exhibit "wild" and "pathological" behaviors, such as staircase-like structures with derivatives vanishing almost everywhere. These non-smooth functions are ill-suited for modern applications, particularly in neural network architectures (e.g., Kolmogorov-Arnold Networks or KANs), where differentiability and smoothness are critical for optimization and convergence analysis. While approximate versions have been proposed to bypass the non-differentiability of exact representations, a rigorous, explicit construction of smooth ( $C^2$ ) approximate superpositions that retains the essence of Kolmogorov's strategy has remained an open question.

Methodology
The authors propose a novel, explicit construction of approximate Kolmogorov superpositions that achieves $C^2$ -smoothness for both inner and outer functions. The methodology proceeds in three main stages:

Construction of Smooth Inner Functions:
Instead of using staircase functions, the authors define a generating function $\phi(t)$ constructed by gluing two specific shape functions:
- A "smeared-out" $C^2$ -Heaviside function $S(z)$ (derived from a trigonometric polynomial) used on small "gaps" of size $\delta = N^{-2}$ .
- A quintic polynomial $P(z)$ used on the main sub-intervals of size $h = N^{-1}$ .
  These pieces are glued to form a strictly increasing, piecewise $C^2$ function. The inner functions $\psi_q(x)$ are then generated by applying suitable translations and dilations to this single generating function $\phi$ . A tuning parameter $\nu$ controls the behavior at the junctions, ensuring the functions remain strictly increasing and $C^2$ -smooth with controllable derivatives.
Separation of Hypercubes and Parameter Selection:
The domain $[0, 1]^d$ is partitioned into hypercubes separated by gaps. The authors define Kolmogorov maps $\Psi_q(x) = \sum_{p=1}^d \lambda_p \psi_q(x_p)$ . A critical challenge is ensuring that these maps separate the hypercubes (i.e., map distinct hypercubes to disjoint intervals in the 1D domain) to allow for global interpolation.
The paper derives explicit conditions on the scaling parameters $\lambda = (\lambda_1, \dots, \lambda_d)$ and the tuning parameter $\nu$ to guarantee strict monotonicity of the mapped hypercube corners. By solving a quadratic equation for $\nu$ (denoted $\nu^*$ ) and defining a recursive sequence for the partial sums of $\lambda$ , the authors provide an explicit set of parameters $\lambda^*$ that ensures the mapped data points are well-ordered and separated, mitigating "dislocations" common in previous constructions.
Construction of Outer Functions:
The outer functions $g_q$ are constructed via row-wise interpolation of the sampled target function values at the centers of the hypercubes. The authors utilize the same shape function $S(z)$ used in the inner construction to perform piecewise $C^2$ interpolation between the mapped data points. This ensures the outer functions are also $C^2$ -smooth.

Key Contributions

Explicit $C^2$ Construction: The paper provides the first explicit construction of approximate Kolmogorov superpositions where both inner and outer functions are strictly increasing and $C^2$ -smooth. This overcomes the "wild" behavior of inherent functions in exact KST representations.
Rigorous Parameter Selection: The authors derive a specific set of scaling parameters $\lambda^*$ and a tuning parameter $\nu^*$ that guarantee the separation of hypercubes under the Kolmogorov maps. This resolves the issue of data dislocation, which is crucial for the validity of the interpolation.
Convergence Analysis: The paper proves that the constructed approximation $f_N$ converges to any $\alpha$ -Hölder continuous function $f \in H^\alpha([0, 1]^d)$ with an accuracy of $O(N^{-\alpha})$ . The proof explicitly accounts for the error contributions from both points inside hypercubes and points falling within the gaps.
Numerical Verification: The authors implement the algorithm and verify the theoretical convergence rates numerically for dimensions $d$ ranging from 2 to 9, demonstrating that the observed error decay matches the predicted $O(N^{-\alpha})$ rate.

Results
The main theoretical result (Theorem 4.1) states that for any $\alpha$ -Hölder continuous function $f$ , the error between $f$ and the constructed approximation $f_N$ satisfies:
$\|f - f_N\|_\infty \leq C \cdot N^{-\alpha}$
where $C$ depends on the dimension $d$ and the Hölder constant. The numerical experiments confirm this convergence order for various test functions, including those with singularities and high-dimensional inputs ( $d \leq 9$ ). The results indicate that the convergence rate is primarily dictated by the smoothness of the target function ( $\alpha$ ) and the number of outer summations ( $N$ ), with the dimension $d$ influencing the constants but not the asymptotic order.

Significance and Claims
The authors claim that this work resolves an open question regarding the construction of smooth approximate Kolmogorov superpositions. The significance of this construction lies in several areas:

Neural Network Applications: The smoothness and strict monotonicity of the constructed functions make them suitable for implementation in neural networks (specifically Kolmogorov-Arnold Networks or KANs). The construction provides a theoretical framework where the inner functions can be parameterized by smooth ansatz classes (e.g., splines, MLPs) without the theoretical obstruction of non-differentiability.
Convergence Theory: By providing an explicit, smooth intermediate representation, the paper paves the way for establishing rigorous convergence theories for KANs. It allows the decomposition of the total error into the approximation error of the KST representation and the approximation error of the neural network ansatz.
Theoretical Insight: The construction reveals that as the dimension $d$ increases, the inner functions become nearly flat within the hypercubes (with derivatives approaching zero), a natural consequence of encoding high-dimensional information into one dimension. This insight suggests that relaxing the requirement of mapping to a single dimension (mapping to $d-1$ or lower dimensions instead) might be a fruitful direction for future research.

The paper emphasizes that while the exact KST representation is mathematically elegant, its practical utility is limited by non-smoothness. This work bridges the gap between the theoretical existence of such representations and their practical, smooth implementation in computational frameworks.