Neural Operators as Efficient Function Interpolators

This paper reframes neural operators as efficient function interpolators by introducing an auxiliary base-space, demonstrating through analytic benchmarks and a nuclear mass model application that they achieve state-of-the-art accuracy with significantly fewer parameters and faster training times compared to standard neural networks.

The Gist

The Big Idea: Changing the Game

Imagine you are trying to guess the shape of a hidden landscape based on a few scattered pebbles you've found on the ground. This is what scientists call "function interpolation."

For a long time, the standard tool for this job has been Neural Networks (specifically MLPs). Think of these like a student taking a test: they memorize the specific answers to the questions they practiced on. If you ask them a question slightly different from the practice set, they might stumble. They learn point-by-point.

The authors of this paper propose a new way of thinking using Neural Operators (NOs). Instead of memorizing individual points, NOs learn the rules of the terrain itself. They treat the data not as a list of answers, but as a continuous map.

The paper asks a simple question: Can we use these powerful "map-makers" (NOs), which were originally designed for complex physics equations, to simply fill in the blanks on a standard graph?

The answer is a resounding yes. In fact, they found that NOs can do this job better, faster, and with less "brainpower" (parameters) than the standard tools.

---

The Secret Sauce: The "Auxiliary Base-Space"

How do they make a "map-maker" work on a simple list of numbers? They use a clever trick called an auxiliary base-space.

The Analogy: The Shadow Puppet
Imagine you have a complex 3D sculpture (the function you want to learn).

• Standard Method (MLP): You take a photo of the sculpture from one angle, then another, then another. You try to memorize every single photo.
• The Paper's Method (NO): You put the sculpture on a rotating stage (the base-space). You shine a light on it and look at the shadow it casts on the wall. Even though the shadow is just a 2D line, by rotating the stage and watching how the shadow changes, you can reconstruct the entire 3D shape in your mind.

In the paper, they take a simple list of data points and arrange them into a "shadow" (a function on a base-space). They train the Neural Operator to understand how the shadow moves. Once it understands the movement rules, it can predict the shape of the sculpture perfectly, even for parts of the shadow it has never seen before.

---

The Tests: How Did They Do?

The team put this new method through a series of "gym workouts" to see how it compared to the old champions (MLPs) and a new contender called KANs (Kolmogorov–Arnold Networks).

1. The Smooth Curves: They tested on wavy, mathematical functions.
   • Result: The NOs were just as accurate as the others but used far fewer resources.
2. The Sharp Edges: They tested on functions with sudden jumps (like a cliff).
   • Result: The NOs handled the sharp edges surprisingly well, whereas standard networks often get "blurry" around the jumps.
3. The Noise: They tested on pure random static (noise).
   • Result: This is where NOs shined. While standard networks tried to "smooth out" the noise (like trying to iron a crumpled shirt), the NOs learned the chaotic pattern efficiently.
4. The High Dimensions: They tested on complex, multi-variable functions.
   • Result: As the data got more complex, the NOs stayed stable and accurate, while others started to struggle.

The Takeaway: The NOs are like a Swiss Army knife that is just as good as a specialized screwdriver, but it's lighter, faster to pack, and doesn't need to be tuned as much.

---

The Real-World Test: The Nuclear Chart

To prove this wasn't just a math trick, they applied it to a real-world problem: Nuclear Physics.

The Problem:
Scientists have a massive chart of all known atomic nuclei (defined by their number of protons and neutrons). They have a very good formula (called WS4) to predict how heavy these nuclei are. But the formula isn't perfect; it has small errors.

• Imagine the WS4 formula is a rough sketch of a mountain range.
• The "error" is the difference between the sketch and the real mountain.
• The goal is to fill in the missing details of the real mountain using only a few known measurements.

The Challenge:
In this field, you cannot cheat. You cannot let the computer "peek" at the answer before it guesses. It must predict the weight of a nucleus it has never seen before, based only on the surrounding landscape.

The Result:
The team used a 2D version of their Neural Operator (a TFNO) to learn the "error map" of the nuclear chart.

• The Old Way (WS4 alone): Had an error of about 282 keV (a unit of energy).
• The New Way (WS4 + Neural Operator): Dropped the error to 198 keV.

This puts them in the top tier of recent methods. But here is the kicker: The Neural Operator model was tiny and trained in minutes on a single computer card. Other top-performing models in the field required massive computer clusters and days of training.

Summary

The paper claims that by rethinking how we feed data into Neural Operators—treating a list of numbers as a continuous "shadow" rather than a list of points—we get a tool that is:

1. More Accurate: It fills in the blanks better.
2. More Efficient: It needs less memory and training time.
3. More Robust: It handles messy, noisy, or complex data without breaking a sweat.

They successfully demonstrated this on both abstract math problems and a critical real-world physics problem (predicting the mass of atomic nuclei), proving that this "map-maker" approach is ready for prime time.

Technical Summary

Technical Summary: Neural Operators as Efficient Function Interpolators

Problem Statement

Interpolating unknown functions from sparse evaluations is a fundamental challenge in science and engineering. While classical methods (linear, polynomial, spline) struggle with high-dimensional or highly oscillatory targets, standard neural networks (MLPs) often depend sensitively on data discretization and are prone to overfitting. Alternative architectures like Kolmogorov–Arnold Networks (KANs) offer interpretability but can be computationally expensive.

Neural Operators (NOs), originally designed to learn maps between infinite-dimensional function spaces (e.g., for solving parametric PDEs), possess "discretization invariance," allowing evaluation at arbitrary resolutions without retraining. However, their application to the simpler, ubiquitous task of finite-dimensional function approximation/interpolation remains under-explored. This paper investigates whether NOs can be repurposed to learn finite-dimensional functions more efficiently than standard point-wise learning approaches.

Methodology

The authors propose a novel reframing of function approximation by introducing an auxiliary base-space ($B$).

Theoretical Framework

Instead of approximating a target function $f: D_{in} \to \mathbb{R}^{d_{out}}$ directly, the method defines an operator $\mathcal{F}$ that acts on functions $x: B \to D_{in}$ via composition:
$$\mathcal{F}[x](s) = f(x(s))$$
By learning the operator $\mathcal{F}$ using a Neural Operator, the system effectively learns the target function $f$.

Implementation Strategy

1. Data Construction: Training data $\{(x_i, f(x_i))\}$ is rearranged into discretized input functions $x(s)$ on a grid of $r$ points within the base-space $B$.
2. Learning Strategy: The NO learns to map these input functions to output functions. This allows the model to learn $f$ across higher-dimensional subspaces "non-locally" rather than point-wise.
3. Architectural Variants:
   • 0D-NO: The base-space $B$ is a single point. This collapses the NO architecture to a standard Multi-Layer Perceptron (MLP) but with tensorized linear layers (Tensorized MLP).
   • 1D-NO: The base-space is one-dimensional, learning functions along curves.
   • 2D-NO: The base-space is two-dimensional, used for the nuclear physics application.
4. Inference: Predictions are made by evaluating the trained NO on input functions constructed similarly to the training data. The output is a function containing $r$ evaluations, leveraging the NO's zero-shot super-resolution capabilities.

Key Contributions

1. Reformulation: A conceptual shift recasting finite-dimensional function approximation as an operator learning problem via an auxiliary base-space.
2. Benchmarking: Comprehensive evaluation of 0D-NOs, 1D-NOs, MLPs, and KANs on analytic functions of varying complexity (partial-wave expansions, Heaviside steps, piecewise Gaussians, noise, and hypergeometric functions).
3. Real-World Application: Application to nuclear physics, specifically learning corrections to the Weizsacker–Skyrme version-4 (WS4) nuclear mass model using a 2D Tensorized Fourier Neural Operator (TFNO).

Results

Analytic Benchmarks

• Performance: The 1D-TFNO consistently emerged as a top performer, often outperforming or matching MLPs and KANs in accuracy (RMSE) while requiring significantly fewer parameters and training time.
• Stability: The 1D-TFNO demonstrated superior stability across different test set sizes and resolutions, a feature attributed to the zero-shot super-resolution properties of FNOs.
• Complexity: The 1D-TFNO successfully learned high-frequency features and random noise structures where MLPs struggled (due to spectral bias) and where KANs sometimes produced large residuals.
• 0D-NO Efficiency: The tensorized MLP (0D-NO) generally outperformed standard MLPs, suggesting that tensorized layers alone offer efficiency gains in function approximation.

Nuclear Binding Energy Application

• Task: The model learned the residual field $\Delta E_b = E_b^{exp} - E_b^{WS4}$ on the $(Z, N)$ nuclear chart, treating the problem as completing a partially observed 2D field.
• Protocol: Evaluation was strictly out-of-sample (pooled five-fold out-of-fold) to prevent data leakage, a critical requirement for nuclear mass modeling.
• Performance:
  • A single TFNO member achieved a Root-Mean-Square (RMS) error of 208.3 ± 2.7 keV.
  • A 30-member ensemble reached 198.2 keV, representing a 30% reduction in error compared to the raw WS4 baseline (282.5 keV).
• Efficiency: The ensemble (4.4M total parameters) was trained "embarrassingly in parallel" on single GPUs in minutes per member, maintaining high parameter efficiency compared to other recent neural network approaches.
• Comparison: The TFNO+WS4 approach outperformed most coordinate-only single-task models in the literature, though it was surpassed by multi-task or physics-informed models (e.g., NuCLR, LightGBM variants) that utilized engineered features or multiple baselines.

Significance and Claims

The paper claims that Neural Operators offer a scalable framework for finite-dimensional function interpolation. The primary significance lies in demonstrating that:

1. Non-local learning is superior: Learning functions across higher-dimensional subspaces (via the auxiliary base-space) is more effective than point-wise learning for sparse, structured scientific data.
2. Efficiency: NOs can achieve state-of-the-art accuracy in scientific interpolation tasks (like nuclear mass correction) with fewer parameters and shorter training times than standard MLPs or KANs.
3. Robustness: The approach maintains high performance without excessive hyperparameter tuning and handles high-frequency structures and noise effectively.

The authors position this work as a motivation for the systematic use of NOs as function approximators, particularly in high-dimensional settings where training data is necessarily sparse. They do not claim to have solved the nuclear mass problem entirely but demonstrate that NOs are a competitive, efficient tool for learning structured residuals in physics.