Neural Network Field Theory at Finite Width

Technical Summary: Neural Network Field Theory at Finite Width

Problem Statement
The paper investigates the limitations of representing Quantum Mechanics (QM) and Quantum Field Theories (QFT) using neural networks (NNs) with a finite number of parameters ($N < \infty$). While previous work established that any QM or QFT with a non-negative measure admits an exact representation via a neural network with a countably infinite number of random parameters, the properties of such representations when truncated to finite width remain unclear. The central question is: which fundamental properties of conventional Euclidean QFTs (such as reflection positivity, cluster decomposition, and the specific nature of field configurations) can be preserved exactly at finite $N$, and which must be violated?

A key subtlety addressed is that, via the Borel isomorphism theorem, a single random parameter can formally encode an infinite amount of information. Therefore, the authors distinguish between "finite" in a parameter-counting sense versus "finite" in a computational or architectural sense. The analysis focuses on natural finite-width architectures, such as single-layer networks with fixed features or random input weights, which cannot hide infinite computations within a finite set of parameters.

Methodology
The authors employ a combination of functional analysis, spectral theory, and constructive QFT techniques to analyze finite-width neural network ensembles. The methodology proceeds in two main stages:

1. Quantum Mechanics (1D): The authors analyze stochastic processes $x(t)$ representing quantum trajectories. They utilize the Kosambi-Karhunen-Loève (KKL) expansion, which provides an optimal orthogonal decomposition of a stochastic process based on the eigenfunctions of its covariance operator. They compare the infinite-rank nature of the covariance kernel in standard QM (derived from the Osterwalder-Schrader axioms) against the finite-rank nature of any finite-width neural network.
2. Quantum Field Theory (d $\ge$ 2): The analysis shifts to generalized random fields (Schwartz distributions). The authors examine three specific obstructions:
   • Coincident-Point Limits: They analyze the behavior of two-point functions as points coincide ($x \to y$), which typically diverge in QFT due to ultraviolet (UV) singularities.
   • Momentum-Space Analysis: They investigate the support of the Fourier transform of field configurations, utilizing the cluster decomposition property (mixing) to determine the momentum content of typical field draws.
   • Distributional Neurons: They test whether allowing the neurons themselves to be Schwartz distributions (rather than ordinary functions) can resolve the mismatch between finite networks and QFT field configurations.

Key Contributions and Results

1. Obstruction in Quantum Mechanics (Finite N)
The paper proves that a finite-width neural network cannot exactly reproduce the correlation functions of a conventional quantum mechanical system satisfying the Osterwalder-Schrader (OS) axioms.

• Dimension-Counting Argument: Standard QM path integrals are supported on continuous, nowhere-differentiable paths (e.g., Brownian motion). A finite sum of piecewise differentiable functions (standard neurons) generates paths that are differentiable almost everywhere, failing to capture the nowhere-differentiable nature of the true measure.
• KKL Optimality Argument: The covariance kernel of a non-trivial QM system has infinitely many strictly positive eigenvalues (infinite rank). A finite-width network corresponds to a projection onto a finite-dimensional subspace. By the optimality of the KKL expansion, any finite truncation incurs a non-zero mean-square error. Specifically, the authors show that for any finite feature space, there exists a direction in the function space where the network has zero variance, whereas the true QM process has strictly positive variance in every non-zero direction (Lemma 2).

2. Obstructions in Quantum Field Theory (Finite N)
For $d \ge 2$, the authors identify that finite-width networks generically violate at least one OS axiom.

• Coincident-Point Divergences: Conventional QFTs with non-trivial Källén-Lehmann representations exhibit divergent two-point functions at coincident points (logarithmic in $d=2$, power-law in $d>2$).
  • Result: Any finite-width architecture where the neuron outputs have bounded pointwise variance (a condition satisfied by standard activations like ReLU or bounded functions with finite-moment parameters) yields a finite smeared variance. Consequently, such networks cannot reproduce the required UV divergences of a non-trivial QFT.
  • Exception: The authors show that if one relaxes the finite-variance condition (e.g., using heavy-tailed parameter distributions or singular amplitudes), a finite-width (even width-1) network can reproduce the correct two-point function and reflection positivity. However, this requires "unphysical" parameter statistics for practical Monte Carlo sampling.
• Momentum-Space Obstruction (Cluster Decomposition):
  • Result: Under the assumption of cluster decomposition (mixing), a typical field draw in a QFT must have Fourier support over the entire momentum space $\mathbb{R}^d$.
  • Obstruction: A finite-width single-layer network $\phi_\theta(x) = \sum_{i=1}^N w_i \sigma_i(a_i \cdot x + b_i)$ has a Fourier transform supported only on the union of $N$ lines (proportional to the input weights $a_i$). Since a finite union of lines is a measure-zero subset of $\mathbb{R}^d$ for $d \ge 2$, the network cannot reproduce the full momentum support required by a clustered QFT. This leads to a violation of cluster decomposition or reflection positivity in higher-point functions.
• Distributional Neurons: Even if the neurons are generalized to be Schwartz distributions (allowing the network output to be a distribution rather than a function), the finite-rank obstruction persists. A finite sum of distributions still leaves an infinite-dimensional kernel where the field is deterministic, contradicting the non-trivial Källén-Lehmann covariance which requires fluctuations in all directions.

Significance and Claims
The paper concludes that finite-width neural network representations of QM and QFT are fundamentally incompatible with the full set of Osterwalder-Schrader axioms.

• Exact Realization: To exactly realize a conventional QFT via a neural network, one must either take the infinite-width limit ($N \to \infty$) or employ architectures/parameter densities that explicitly violate standard physical assumptions (e.g., using infinite-variance parameters or sacrificing cluster decomposition).
• Nature of Finite-N Theories: Theories defined by finite-$N$ neural networks are not merely "approximations" of standard QFTs; they are distinct, "exotic" theories that necessarily lack properties such as Euclidean covariance, reflection positivity, or cluster decomposition.
• Practical Implications: While finite-width networks cannot exactly reproduce standard QFTs, the authors note that truncated approximations are still used in numerical studies (e.g., Liouville theory, Maxwell theory). However, the statistical properties of these approximations are not guaranteed to match the target theory, and the errors are not merely numerical but structural.

The work does not propose new experimental applications but serves as a rigorous theoretical characterization of the limitations of finite-width neural networks as representations of continuum quantum systems. It suggests that future research should focus on quantifying the degree of violation of OS axioms as a function of $N$ and classifying which combinations of axioms can be preserved at finite width.