Adaptive-Growth Randomized Neural Networks for Level-Set Computation of Multivalued Nonlinear First-Order PDEs with Hyperbolic Characteristics

This paper introduces an Adaptive-Growth Randomized Neural Network (AG-RaNN) method combined with adaptive collocation and layer-growth mechanisms to efficiently compute multivalued solutions of nonlinear first-order PDEs with hyperbolic characteristics by solving their high-dimensional level-set formulations in an augmented phase space.

The Gist

Imagine you are trying to predict the path of a crowd of people running through a maze. At first, everyone is running in a straight line, and it's easy to draw a single map showing where they are. But what happens when the maze gets tricky? Some people hit dead ends, others turn corners, and eventually, the paths cross over each other.

In the world of physics and math, this is called a singularity. When paths cross, the "solution" (the map of where everyone is) stops being a single line and becomes a messy, overlapping tangle of many different possibilities. This is what happens with multivalued solutions in complex equations like those describing light bending, seismic waves, or quantum particles.

Traditional math tools usually try to "smooth over" these tangles or pick just one path (like the shortest one), but that loses important information. The real world often needs to know all the paths at once.

This paper introduces a new, clever way to solve these tangled problems using Adaptive-Growth Randomized Neural Networks (AG-RaNN). Here is how it works, broken down into simple concepts:

1. The Problem: The "Spaghetti" Tangle

Think of the solution to these equations as a piece of spaghetti.

• Before the tangle: It's a straight noodle. Easy to track.
• After the tangle: The noodle folds over itself, creating a ball of spaghetti where multiple strands occupy the same space.
• The Challenge: If you try to draw this on a 2D piece of paper, it looks like a mess. If you try to track every single strand individually, the math gets so complicated (high-dimensional) that even supercomputers struggle. This is the "Curse of Dimensionality."

2. The Trick: The "Shadow" Method (Level-Set)

Instead of trying to track the messy spaghetti strands directly, the authors use a trick called the Level-Set Method.

Imagine you have a giant, invisible 3D block of Jell-O. You embed the spaghetti inside it. Instead of looking at the spaghetti, you look at the surface of the Jell-O where the level is exactly zero.

• In this higher-dimensional "shadow world," the messy tangle of spaghetti becomes a clean, smooth surface.
• The complex, non-linear problem turns into a simple, linear one (like a gentle wind blowing across the Jell-O surface).
• The Catch: To make this shadow work, you have to add extra dimensions. A 1D problem becomes 3D; a 2D problem becomes 5D. This makes the computer's job much harder because it has to calculate in a much bigger space.

3. The Solution: The "Smart Net" (AG-RaNN)

To solve this huge, multi-dimensional Jell-O problem without the computer crashing, the authors use a Randomized Neural Network.

• The Random Net: Imagine a fishing net where the knots (the complex parts) are tied randomly and never moved. You only adjust the ropes (the simple linear parts) to catch the fish. This is much faster and easier than trying to tie every knot perfectly from scratch.
• Adaptive Growth: The net starts small. If it misses a fish (the solution isn't accurate enough), the net automatically grows a new layer of mesh to catch the tricky spots. It "grows" only where it needs to.
• Adaptive Collocation (The Spotlight): Instead of shining a light on the entire ocean to find the fish, the computer uses a spotlight. It only looks closely at the "tube" of space where the zero-level surface actually exists. It ignores the empty ocean around it. This saves a massive amount of computing power.

4. The Result: Unraveling the Tangle

By combining the "Shadow Method" (to turn the mess into a smooth surface) with the "Smart Net" (to solve the huge space efficiently), the authors can:

1. See the whole picture: They recover all the overlapping paths (the multivalued solution) instead of just picking one.
2. Handle the mess: They can accurately predict where the "shocks" and "caustics" (the tangles) happen.
3. Do it fast: Because they only focus on the important areas and use a simple, linear training process, they can solve problems that are too big for traditional methods.

In a Nutshell

Imagine trying to map a traffic jam where cars are driving over each other in a 3D pile-up.

• Old way: Try to draw every car's path on a flat map. It's impossible; the lines cross and confuse you.
• This paper's way: Lift the traffic jam into a 3D cloud. The cars are now on different layers of a smooth hill. Use a smart, self-growing drone swarm to only scan the specific layer where the cars are, ignoring the empty sky.
• Outcome: You get a perfect, clear map of the entire traffic jam, including every car's position, without the computer getting overwhelmed.

This method is a game-changer for fields like seismology (predicting earthquake waves), optics (designing better lenses), and quantum physics, where understanding the "messy" overlapping paths is crucial.

Technical Summary

1. Problem Statement

The paper addresses the computational challenge of solving nonlinear first-order partial differential equations (PDEs) with hyperbolic characteristics, specifically:

• Quasilinear hyperbolic balance laws (e.g., Burgers' equation).
• Hamilton–Jacobi (HJ) equations.

The Core Difficulty:
In many physical applications (geometric optics, seismic waves, semiclassical limits of quantum dynamics), solutions to these equations develop singularities (shocks or caustics) in finite time. After singularity formation, the solution is no longer a single-valued function but becomes a multivalued structure (a union of multiple branches).

• Traditional methods often seek "viscosity" or "entropy" solutions, which select a single physical branch but discard the multivalued nature required for certain physical descriptions (e.g., wavefronts in optics).
• Directly tracking the evolving number of solution branches is computationally difficult because the number of phases is generally unknown a priori.

2. Methodology

The authors propose a framework combining Level-Set Methods with Adaptive-Growth Randomized Neural Networks (AG-RaNN).

A. Level-Set Formulation (Dimensional Augmentation)

To handle multivalued solutions, the authors reformulate the original nonlinear PDE into a linear transport equation posed in an augmented phase space.

• Original Problem: $u(t, x)$ satisfies a nonlinear PDE.
• Level-Set Function: A function $\phi(t, x, z, p)$ is introduced such that the zero level set ($\phi=0$) corresponds to the solution manifold ($z=u, p=\nabla u$).
• Result: The nonlinear evolution is transformed into a linear transport equation:
  $$\partial_t \phi + \mathbf{a}(X) \cdot \nabla_Y \phi = 0$$
  where $X = (t, Y)$ and $Y$ includes physical and auxiliary variables.
• Trade-off: While the equation becomes linear, the dimensionality increases significantly (often doubling the original dimensions), leading to the "curse of dimensionality."

B. Adaptive-Growth Randomized Neural Networks (AG-RaNN)

To solve the high-dimensional linear transport equation efficiently, the authors utilize AG-RaNNs, which offer a convex optimization landscape compared to standard deep learning.

1. Randomized Neural Networks (RaNN):

   • Unlike standard Deep Neural Networks (DNNs) where all weights are trained, RaNNs fix the nonlinear parameters (weights and biases of hidden layers) after random initialization.
   • Only the linear output coefficients are trained via a least-squares method. This converts the non-convex optimization problem into a convex one, ensuring robustness and avoiding local minima.

2. Layer-Growth Strategy:

   • The network is not static. It starts with a coarse approximation.
   • Iterative Enrichment: Based on error indicators (residuals), new hidden layers are added. The parameters for new layers are constructed using the error from the previous approximation, progressively enriching the feature space to capture complex structures (like shocks).

3. Adaptive Collocation:

   • Solving the equation over the entire high-dimensional domain is inefficient.
   • Tube Sampling: The method identifies a "tubular neighborhood" around the zero level set ($\{|\phi| \le \epsilon\}$).
   • Strategy: Collocation points are concentrated exclusively in this narrow tube where the solution manifold exists, rather than uniformly sampling the entire augmented space. This drastically reduces the number of required samples.

3. Key Contributions

1. Unified Framework for Multivalued Solutions: The paper provides a systematic approach to compute multivalued solutions for both scalar hyperbolic laws and Hamilton-Jacobi equations by embedding them in a linear transport framework.
2. AG-RaNN Integration: It introduces the first application of Adaptive-Growth RaNNs to level-set equations, combining:
   • Layer Growth: To handle the increasing complexity of the solution manifold without pre-defining network size.
   • Adaptive Sampling: To mitigate the curse of dimensionality inherent in the level-set formulation.
3. Theoretical Convergence: The authors establish a rigorous error analysis under standard regularity assumptions. They prove a graph norm equivalence for the level-set operator and decompose the total error into:
   • Approximation error (due to network capacity).
   • Statistical error (due to finite sampling).
   • Optimization error (due to least-squares solver precision).
     The analysis confirms that the method converges as the network grows and sampling density increases.
4. Efficient Zero-Level Set Extraction: A practical algorithm is proposed to locate the zero level set in high dimensions using iterative refinement and local perturbation, avoiding the need for interpolation.

4. Numerical Results

The method was tested on various 1D and 2D problems, including:

• 1D/2D Inviscid Burgers' Equation: Handling continuous data, rarefaction waves, and shock waves.
• Hamilton-Jacobi Equations: Solving for both the gradient field ($\nabla S$) and the full solution ($S$) in cases involving caustics (focusing) and harmonic oscillators.
• High-Dimensional Case: A 2D HJ equation transformed into a 5D augmented phase space problem.

Performance Highlights:

• Accuracy: The AG-RaNN method successfully recovered multivalued structures and resolved nonsmooth features (shocks/caustics) with high precision.
• Efficiency: Compared to standard collocation (uniform sampling), the adaptive collocation strategy significantly reduced computational time while maintaining accuracy near the solution manifold.
• Comparison: The method outperformed traditional finite difference methods in terms of speed and ability to handle long-time horizons without numerical dissipation smearing the multivalued branches.
• Scalability: The method successfully solved problems in 5D augmented spaces, demonstrating its ability to overcome the curse of dimensionality.

5. Significance

This work represents a significant advancement in the numerical solution of hyperbolic PDEs where multivaluedness is physically relevant.

• Bridging Physics and AI: It effectively bridges the gap between classical level-set methods (which handle topology changes well) and modern randomized neural networks (which handle high dimensions well).
• Overcoming Limitations: By fixing nonlinear parameters and using adaptive growth, it avoids the training instability of standard DNNs while retaining the expressive power needed for complex, singular solutions.
• Broad Applicability: The framework is applicable to diverse fields including seismic imaging, geometric optics, and semiclassical quantum mechanics, where understanding the full multivalued wave structure is crucial.

In summary, the paper presents a robust, theoretically grounded, and computationally efficient method for recovering multivalued solutions of nonlinear first-order PDEs, overcoming the dual challenges of singularity formation and high dimensionality.