🔢 mathematics

Random Neural Network Expressivity for Non-Linear Partial Differential Equations

This paper establishes rigorous error bounds and dimension-free approximation rates for Random Neural Networks (RaNNs) in solving non-linear partial differential equations, demonstrating their theoretical and practical efficacy for complex systems like Porous Medium and Compressible Navier-Stokes equations.

Original authors: Muhammed Ali Mehmood, Lukas Gonon

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Muhammed Ali Mehmood, Lukas Gonon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Solving Nature's Toughest Puzzles

Imagine the natural world is full of complex puzzles, like predicting how a storm moves, how gas flows through a sponge, or how a shockwave travels through air. Scientists describe these puzzles using Partial Differential Equations (PDEs). These are mathematical recipes that tell us how things change over time and space.

For simple puzzles, we have standard tools to solve them. But for the messy, non-linear ones (like turbulent air or flowing gas), these traditional tools often break down or take too long to compute.

In recent years, scientists have started using Artificial Neural Networks (a type of AI) to solve these puzzles. Think of a neural network as a giant, flexible Lego structure. By adjusting the connections between the Legos, the structure can learn to mimic almost any shape or pattern.

The Problem: Training is Hard and Slow

Usually, to make a neural network solve a specific puzzle, you have to "train" it. This is like trying to tune a massive orchestra by adjusting the pitch of every single instrument one by one. It takes a long time, requires a lot of computer power, and sometimes the orchestra just can't find the right tune.

The Solution: The "Random" Orchestra (RaNNs)

This paper focuses on a clever shortcut called Random Neural Networks (RaNNs).

Imagine you have that same giant Lego structure, but instead of carefully tuning every connection, you randomly scatter the pieces and lock them in place. The only thing you are allowed to adjust are the final "output knobs" (the weights at the very end).

The Analogy: Think of a piano where the strings and hammers are randomly placed and fixed. You can't change the strings, but you can press the keys (the output weights) to create music. Surprisingly, if you have enough strings (enough "width" in the network), you can still play almost any song, even though the internal mechanics are random.

The authors wanted to know: Can this "random" approach actually solve the hardest physics puzzles (non-linear PDEs), and how good is it?

The Main Discovery: Speed and Accuracy Without the "Curse"

The researchers proved two main things:

It Works for Complex Physics: They showed that these random networks can accurately approximate the solutions to very difficult equations, specifically the Porous Medium Equation (gas flowing through a sponge) and the Compressible Navier-Stokes Equations (shockwaves in air).
The "Magic" Rate: They proved that as you add more random pieces (increase the network width, $N$ $N$ ), the error drops at a steady, predictable rate of $1/\sqrt{N}$ .
- The Analogy: Imagine you are trying to guess the average height of people in a city. If you ask 4 people, you might be off by a lot. If you ask 100 people, you get much closer. If you ask 10,000, you are very close. The paper proves that for these specific physics problems, the "random" method gets better at this exact mathematical speed, regardless of how many dimensions (variables) the problem has.
- Usually, adding more variables (like adding time, plus 3D space, plus temperature, plus pressure) makes math problems explode in difficulty (the "Curse of Dimensionality"). This paper claims their method avoids that trap.

How They Proved It: The "Ridgelet" Lens

To prove this mathematically, the authors used a tool called the Ridgelet Transform.

The Analogy: Imagine you have a complex, crumpled piece of paper (the solution to the physics equation). To understand it, you shine a special light through it (the Ridgelet Transform). This light breaks the crumpled paper down into simple, straight lines (ridges).
The authors showed that even though the neural network's internal weights are random, they act like a bucket of these "straight lines." By mixing enough of these random lines together, you can reconstruct the crumpled paper perfectly. They proved that the "crumpled paper" (the solution to the PDE) is smooth enough to be rebuilt this way.

The Experiments: Does it Work in Real Life?

Theory is great, but does it work on a computer?

The authors ran simulations on the two tough equations mentioned above.

The Test: They tried to solve the equations using random networks of different sizes (small, medium, large).
The Result: As they made the networks bigger, the error dropped exactly as their math predicted ( $1/\sqrt{N}$ ).
The Surprise: Their math required the random numbers to follow a very specific, unusual distribution (heavy-tailed). However, in their computer experiments, they used standard Gaussian (Bell Curve) distributions, which are much more common and easier to use. The method still worked perfectly. This suggests the theory is robust and practical.

Summary

This paper is a "proof of concept" that Random Neural Networks are not just a lucky guess. They are a mathematically sound, efficient way to solve some of the most complex physics equations in the universe.

Old Way: Carefully tune every part of the machine (slow, expensive).
New Way: Randomize the inside, tune only the outside (fast, cheap).
The Guarantee: The more random parts you add, the closer you get to the right answer, and this holds true even for very high-dimensional, complex problems.

The authors conclude that this method is a powerful tool for scientists who need to simulate complex natural phenomena without waiting days for a computer to finish the calculation.

Technical Summary: Random Neural Network Expressivity for Non-Linear Partial Differential Equations

Problem Statement
Partial Differential Equations (PDEs), particularly non-linear ones such as the Porous Medium Equation (PME) and Compressible Navier-Stokes Equations, model complex physical phenomena but often exhibit disorderly behavior that renders classical analytic and numerical approaches intractable. While deep learning methods, specifically Physics-Informed Neural Networks (PINNs), have emerged as powerful tools for solving PDEs, the loss landscapes for non-linear problems can be exceptionally complex. To mitigate this, Random Neural Networks (RaNNs)—networks with randomly generated hidden weights where only the output weights are trainable—are frequently employed either as standalone solvers or as initialization schemes. However, despite their practical utility, a rigorous theoretical understanding of the approximation properties of RaNNs for time-dependent, non-linear PDE solutions remains limited. Existing theoretical bounds often rely on strict assumptions (e.g., finiteness in Barron-ridgelet norms) or apply only to fixed time points, failing to capture the distinct temporal and spatial behaviors characteristic of evolutionary PDEs.

Methodology
The authors develop a theoretical framework to analyze the expressivity of RaNNs for functions in time-dependent Sobolev spaces, which encompass the solution spaces of many important non-linear PDEs. The core methodology involves:

Ridgelet Transform Representation: The paper extends the ridgelet transform framework to time-dependent Sobolev spaces. It establishes an integral representation for functions $u(t, x)$ using a ridgelet transform $R_\psi u$ and an activation function $\sigma$ . This representation allows the function to be expressed as an integral over the parameter space $(\tau, a, b)$ .
Unbiased Estimator Construction: By defining a specific probability distribution $\pi$ over the parameters (involving heavy-tailed distributions for $\tau$ and $a$ ), the authors construct a RaNN $u_N$ as an unbiased Monte Carlo estimator of the integral representation.
Error Analysis via Plancherel-type Estimates: A key technical step involves deriving a higher-order Plancherel-type estimate (Lemma 3.3). This lemma connects the Sobolev regularity of the target function $u$ in Cartesian space to the regularity of its ridgelet transform in parameter space. This connection is crucial for bounding the approximation error.
Application to PDE Residuals: The derived approximation bounds for the function $u$ are propagated to the PDE residual loss. The authors analyze how the approximation error in mixed Sobolev norms ( $H^p_t H^q_x$ ) translates to the training error of PINNs for specific non-linear PDEs.

Key Contributions
The paper makes the following specific contributions:

Dimension-Independent Approximation Rate: The authors derive error bounds for RaNN approximations of sufficiently regular time-dependent Sobolev functions. They prove that the approximation error decays at a rate of $O(N^{-1/2})$ with respect to the network width $N$ , independent of the spatial dimension $d$ . This holds for functions in mixed Sobolev spaces $H^p_t H^q_x$ , requiring only minimal extra regularity in time and space.
Theoretical Guarantees for Non-Linear PDEs: The results are applied to two significant classes of non-linear PDEs:
- Porous Medium Equations (PME): The authors show that RaNNs can approximate PME solutions with high probability, providing quantitative bounds on the PINN training error.
- Compressible Navier-Stokes Equations: The framework is extended to the compressible Navier-Stokes system (specifically in 1D with periodic boundary conditions), demonstrating that RaNN-based PINNs can drive the training residual to zero with high probability.
Ridgelet-Based Representation: The work provides a specific ridgelet-based representation for $L^2$ functions and establishes a novel inequality linking Sobolev regularity to the ridgelet transform, serving as a potential building block for generalization error analyses in RaNN-based learning.

Results

Theoretical Bounds: Theorem 3.4 establishes that for a function $u \in H^{p+s_1}(0, T; H^{q+s_2}(D))$ , there exists a RaNN $u_N$ such that the expected squared error in the mixed Sobolev norm is bounded by $C/N$ , where $C$ depends on the domain, the function's regularity, and the activation function, but the rate $N^{-1/2}$ is dimension-free.
Numerical Validation: The theoretical findings are supported by numerical experiments on the PME (in dimensions $d=1$ $d = 1$ to $5$) and the 1D Compressible Navier-Stokes equations.
- The experiments utilize RaNNs with Fourier feature embeddings and Gaussian-distributed weights (noting that the theory assumes heavy-tailed distributions, but experiments test robustness).
- The observed convergence rates in the numerical experiments closely match the theoretical $O(N^{-1/2})$ rate (e.g., slopes of $-0.47$ to $-0.58$ in log-log plots), validating the upper bounds even when the specific theoretical weight distribution is replaced by a standard Gaussian.
- The results suggest that the convergence rates may hold beyond the strict theoretical setting, including for solutions with lower regularity (e.g., the Barenblatt profile which is non-differentiable at support edges).

Significance and Claims
The paper claims to provide the first rigorous, dimension-independent approximation guarantees for RaNNs applied to time-dependent, non-linear PDEs. By moving beyond fixed-time analysis and addressing mixed Sobolev spaces, the work bridges a gap between the empirical success of RaNN-based PINNs and their theoretical justification. The authors assert that their results offer quantitative approximation guarantees for learning complex non-linear PDEs, suggesting that RaNNs are capable of efficiently approximating solutions to these problems without suffering from the curse of dimensionality in the approximation rate. The work positions RaNNs not merely as heuristic tools but as methods with provable convergence properties for evolutionary PDEs.