Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications

This paper introduces Neural Operator-Grounded Continuous Tensor Function Representation (NO-CTR), a novel framework that replaces discrete, linear mode-$n$ products with continuous, nonlinear neural operators to more faithfully represent complex real-world data across various grid structures and point clouds, while theoretically guaranteeing universal approximation and demonstrating superior performance in multi-dimensional data completion tasks.

The Gist

Imagine you are trying to describe a complex 3D object, like a statue of a frog or a colorful video of a horse running, to a friend.

The Old Way (Discrete & Linear):
Traditionally, scientists have tried to describe these objects by taking a grid of photos or a list of coordinates. Think of this like a Minecraft world. Everything is made of square blocks. If you want to describe a smooth curve, you have to use tiny, tiny blocks. But no matter how small the blocks are, the curve still looks a little jagged and pixelated.

Furthermore, the old math used to connect these blocks was very rigid. It was like using a stencil. If you wanted to change the shape of the frog's leg, you could only stretch or shrink it in straight lines. You couldn't bend it naturally or add complex curves because the "stencil" was too simple. This is what the paper calls "discrete and linear mode-n product." It works okay for simple things, but it struggles with the messy, smooth, and complex reality of the real world.

The New Way (NO-CTR):
The authors of this paper, led by Ruoyang Su and Xi-Le Zhao, came up with a brilliant new idea. Instead of using a grid of blocks and a rigid stencil, they decided to use smooth, flowing paint and a smart, flexible artist.

Here is how their new system, called NO-CTR, works, broken down into simple parts:

1. The "Smooth Paint" (Continuous Tensor Functions)

Instead of a grid of blocks, imagine the data (like an image or a 3D shape) is a continuous sheet of paint. You can ask for the color or shape at any point, even between the pixels. This means you can zoom in infinitely, and the image never gets pixelated. It's like having a high-definition video that never loses quality, no matter how much you zoom in.

2. The "Smart Artist" (Neural Operators)

This is the magic trick. In the old days, the "artist" (the math) was a robot that could only draw straight lines and copy-paste patterns. It was rigid.

The authors replaced this robot with a Neural Operator. Think of a Neural Operator as a master chef or a skilled sculptor.

• The Old Robot: If you gave it a lump of clay, it could only flatten it or cut it into squares.
• The New Master Chef: If you give it a lump of clay (the core data), it can twist, stretch, smooth, and mold it into any complex shape it needs to be. It understands that real-world data is curvy, bumpy, and full of hidden patterns.

3. How They Work Together

The paper proposes a system where:

1. You start with a simple "core" shape (the lump of clay).
2. You pass it through a series of "Smart Artists" (the Neural Operators).
3. Each artist takes the shape and applies a complex, non-linear transformation—bending it, twisting it, and adding detail—until it perfectly matches the real-world object you are trying to describe.

Why is this a Big Deal?

The authors tested this on three very different types of data:

• Multispectral Images: Like seeing the world in more colors than human eyes can see (e.g., seeing heat or chemical composition).
• Color Videos: Moving pictures with complex motion.
• Point Clouds: 3D data that isn't even on a grid (like a cloud of dust or a 3D scan of a car).

The Result:
In every test, the new "Smart Artist" system (NO-CTR) did a much better job than the old "Minecraft block" methods.

• Sharper Details: It could recover the tiny stripes on a frog's skin or the texture of a horse's mane that the old methods blurred out.
• No Pixelation: Because it uses "smooth paint" instead of blocks, it works perfectly even when the data is missing huge chunks (like a photo with 90% of the pixels missing).
• Flexibility: It works on standard grids, weird grids, and even data that doesn't fit on a grid at all.

The Bottom Line

Think of this paper as upgrading from a pixelated, blocky video game to a realistic, fluid simulation.

The authors realized that the math used to describe complex data was too rigid (like a stencil). They introduced a new tool (Neural Operators) that acts like a flexible, intelligent artist. This allows computers to represent the real world not as a collection of jagged blocks, but as a smooth, continuous, and highly detailed masterpiece.

They proved mathematically that this new method can approximate any continuous shape, and their experiments showed it is currently the best tool we have for filling in missing data and reconstructing complex 3D worlds.

Technical Summary

1. Problem Statement

Multi-dimensional data (e.g., images, videos, point clouds) is fundamental to modern applications, but representing it effectively remains a challenge.

• Limitations of Discrete Methods: Traditional tensor decompositions (CP, Tucker, t-SVD) rely on discrete, linear mode-$n$ products. These methods map discrete core tensors to discrete target tensors using linear matrix multiplications. They fail to capture complex, nonlinear structures inherent in real-world data and suffer from discretization artifacts.
• Limitations of Existing Continuous Methods: Recent Continuous Tensor Function Representations (e.g., LRTFR, FunBaT) aim to represent data as continuous functions of coordinates, unifying mesh-grid and non-mesh-grid data. However, these methods still rely on discrete and linear mode-$n$ products to map a discrete core tensor to the target. This linearity constrains their ability to model complex nonlinear relationships, leaving the full potential of continuous representations untapped.

Core Bottleneck: The mapping mechanism in current continuous tensor representations is fundamentally discrete and linear, preventing them from faithfully capturing the nonlinear complexity of real-world data.

2. Methodology

The authors propose a novel framework called Neural Operator-Grounded Continuous Tensor Function Representation (NO-CTR) to overcome these limitations.

A. Continuous and Nonlinear Mode-$n$ Operators

The core innovation is replacing the traditional discrete/linear mode-$n$ product with a continuous and nonlinear mode-$n$ operator.

• Concept: Instead of operating on discrete fiber vectors, the new operator acts on mode-$n$ univariate fiber functions of a continuous tensor function.
• Mechanism: It utilizes Neural Operators (specifically DeepONets) to map the continuous core tensor function directly to the continuous target tensor function.
• Mathematical Formulation:
  Let $G \in C([0, 1]^N)$ be a continuous core tensor function. A continuous and nonlinear mode-$n$ operator $F^{\langle n \rangle}$ induced by a neural operator $F_n: C([0, 1]) \to C([0, 1])$ is defined as:
  $$F^{\langle n \rangle}(G)(y_1, \dots, y_N) = F_n\left(G(y_1, \dots, y_{n-1}, \cdot, y_{n+1}, \dots, y_N)\right)(y_n)$$
  This allows the model to learn nonlinear transformations of the fiber functions directly.

B. NO-CTR Architecture

The NO-CTR represents a target continuous tensor function $X$ as a composition of a continuous core function and a series of these nonlinear operators:
$$X = F_N^{\langle N \rangle} \circ \dots \circ F_2^{\langle 2 \rangle} \circ F_1^{\langle 1 \rangle}(G)$$

• Core Function ($G$): Implemented using SIREN (Sinusoidal Representation Networks), a type of Implicit Neural Representation (INR) known for capturing high-frequency details.
• Operators ($F_n$): Implemented using DeepONets (Deep Operator Networks), which consist of a branch network (encoding input function values at sensor points) and a trunk network (outputting basis functions).

C. Theoretical Guarantee

The paper provides a Universal Approximation Theorem (Theorem 1), proving that any continuous tensor function can be approximated arbitrarily well by the NO-CTR framework, provided the core function and inducing operators have sufficient capacity.

D. Application: Multi-Dimensional Data Completion

To validate the method, the authors formulate a data completion model. Given an incomplete observation $O$ on a domain $\Omega$, the model minimizes the reconstruction error:
$$\arg\min_{\{\theta_n\}} \sum_{y \in \Omega} \left( O(y) - \text{NO-CTR}(y) \right)^2$$
This approach recovers missing data (pixels, voxels, or point cloud coordinates) by learning the underlying continuous function.

3. Key Contributions

1. Novel Operator Definition: Introduced continuous and nonlinear mode-$n$ operators as a replacement for discrete/linear products, enabling genuine continuous representation of real-world data.
2. NO-CTR Framework: Proposed the NO-CTR architecture, which integrates neural operators into tensor decomposition, bridging the gap between neural operators and tensor representations.
3. Theoretical Proof: Proved that NO-CTR is a universal approximator for continuous tensor functions.
4. Empirical Superiority: Demonstrated state-of-the-art performance in multi-dimensional data completion across diverse datasets, including regular grids, multi-resolution grids, and unstructured point clouds.

4. Experimental Results

The authors evaluated NO-CTR against traditional tensor methods (TR-ALS) and continuous representation methods (SIREN, MFN, FR-INR, LRTFR) on four datasets:

• Multi-Spectral Images (MSI): NO-CTR achieved the highest PSNR and SSIM across all sampling rates (5%–20%). It significantly outperformed LRTFR, recovering fine textures (e.g., stripes on clothes) with higher fidelity.
• Color Videos: NO-CTR consistently outperformed competitors in restoring dynamic scenes, showing sharper edges and clearer motion details.
• Sentinel-2 Images (Multi-Resolution): Tested on images with varying spatial resolutions (10m, 20m, 60m). NO-CTR excelled in recovering geographical details and urban boundaries, crucial for remote sensing.
• Point Clouds (Beyond Mesh Grids): In tasks involving 3D point clouds (Frog, Mario, Rabbit), where traditional tensor methods fail, NO-CTR achieved the lowest NRMSE and highest $R^2$, successfully reconstructing surface details and distinctive features.

Ablation Studies:

• Operator Type: Replacing continuous/nonlinear operators with discrete/linear ones caused a significant drop in performance, confirming the necessity of nonlinearity.
• Neural Operator Choice: DeepONet outperformed Fourier Neural Operators (FNO) in this context due to its flexible architecture.
• Core Representation: SIREN as the core function yielded better results than LRTFR or PE-MLP.
• Parameters: While NO-CTR has a higher parameter count than some baselines, the performance gain justifies the complexity.

5. Significance

• Breaking the Linearity Bottleneck: The paper fundamentally shifts tensor representation from linear mappings to nonlinear functional mappings, unlocking the ability to model complex, real-world data structures.
• Unification of Domains: By operating on continuous functions, NO-CTR seamlessly handles data on regular grids, irregular grids, and unstructured point clouds within a single framework.
• Discretization Artifact Reduction: The continuous nature of the representation eliminates aliasing and discretization errors common in grid-based methods.
• Future Impact: This work opens new avenues for applying neural operators to tensor decomposition, potentially impacting fields like scientific computing, computer vision, and remote sensing where high-fidelity continuous data representation is critical.