Radial Müntz-Szász Networks: Neural Architectures with Learnable Power Bases for Multidimensional Singularities

This paper introduces Radial Müntz-Szász Networks (RMN), a highly parameter-efficient neural architecture that utilizes learnable radial power bases and a log-primitive to accurately model multidimensional singular fields like $1/r$and$\log r$, achieving significantly lower error rates than standard MLPs and SIREN on benchmark tasks while providing closed-form gradients for physics-informed learning.

The Gist

Imagine you are trying to teach a robot to draw a picture of a black hole or a crack in a piece of glass. These things have a very specific, dangerous property: right at the center, the numbers go to infinity. In math, we call these "singularities."

For a long time, the standard way to teach robots (using neural networks) to draw these shapes was like trying to paint a perfect circle using only a grid of square tiles. You can get close, but you'll always end up with jagged, diamond-shaped edges, and it takes millions of tiles to make it look smooth.

This paper introduces a new kind of robot brain called Radial Müntz-Szász Networks (RMN). Here is the simple breakdown of what they did and why it's a big deal.

1. The Problem: The Wrong Tool for the Job

Standard AI models (called MLPs) are like Lego builders. They build complex shapes by stacking small, square blocks (neurons) on top of each other.

• The Issue: If you try to build a perfect circle (or a radial singularity like gravity pulling toward a center) using only square Legos, you are fighting against the geometry. You need millions of Legos to get a smooth curve, and even then, it looks a bit blocky.
• The Paper's Discovery: The authors proved a mathematical rule: You cannot build a perfect circle using only straight, grid-aligned Lego lines. It's physically impossible to get it right without using a massive amount of resources.

2. The Solution: The "Magic Compass"

Instead of using square Legos, the authors gave the robot a Magic Compass.

• How it works: The robot stops looking at "Left/Right" and "Up/Down" coordinates. Instead, it looks at Distance from the Center.
• The Secret Sauce: The robot learns a special set of "power laws." Instead of just drawing lines, it learns to draw curves like $1/r$(which gets huge as you get close to the center) or$\log(r)$ (which grows slowly).
• The "Learnable" Part: Usually, scientists have to guess the formula. This robot learns the formula. It asks, "Is the curve $1/r$? Or is it$1/r^{0.5}$?" and it adjusts its internal dials until it finds the perfect answer.

3. The Analogy: The Swiss Army Knife vs. The Hammer

• Old AI (MLP): Imagine trying to fix a watch with a giant sledgehammer. It can eventually smash the watch into the shape you want, but you need a huge hammer, it takes a lot of force, and you might break other things.
• New AI (RMN): This is a Swiss Army Knife with a tiny, perfect screwdriver. It is designed specifically for the job. It doesn't need to be big or heavy; it just needs the right shape.

4. Why This is Amazing (The Results)

The authors tested this new robot against the old ones on 10 different difficult physics problems (like gravity, cracks in metal, and electric fields).

• Accuracy: The new robot was 1.5 to 51 times more accurate than the old giant models.
• Efficiency: This is the real shocker.
  • The old models needed 33,537 parameters (think of these as the robot's "brain cells" or memory slots).
  • The new RMN model did the same job with only 27 parameters.
  • Analogy: It's like the old robot needed a library of 33,000 books to solve a riddle, while the new robot solved it with a single, perfectly written index card.

5. What Can It Do Now?

Because the robot is so efficient and understands the "shape" of the problem, it can do things the old ones struggled with:

• Find the Source: If you show it a messy gravitational field, it can figure out exactly where the planet is hiding, even if it's buried in noise.
• Understand Physics: It doesn't just guess numbers; it learns the actual math of the universe (like how gravity works). If you ask it, "What is the power of this singularity?" it can tell you, "It's -1," which corresponds to real-world physics.
• Handle Cracks: It can model how metal cracks, which is crucial for building safer bridges and planes.

Summary

The authors realized that trying to force a square-peg AI to fit a round-hole problem was a waste of time. They built a new AI that is round by design.

By teaching the AI to think in terms of distance from a center and power laws (like $1/r$), they created a model that is:

1. Tiny: Uses almost no computer memory.
2. Fast: Learns quickly.
3. Accurate: Perfectly captures the dangerous, infinite points that other models miss.

It's a shift from "brute force" (throwing more data at the problem) to "smart design" (building the right tool for the job).

Technical Summary

Here is a detailed technical summary of the paper "Radial Müntz-Szász Networks: Neural Architectures with Learnable Power Bases for Multidimensional Singularities."

1. Problem Statement

Many physical phenomena in multiple spatial dimensions exhibit radial singularities, where the field depends primarily on the distance $r = \|x\|$ from a source. Examples include:

• Coulomb and Gravitational Potentials: Scaling as $1/r$(inverse) in 3D or$\log r$ in 2D.
• Fracture Mechanics: Stress fields scaling as $r^{-1/2}$ near crack tips.
• Fluid Dynamics: Boundary layers with steep fractional-power behavior.

The Challenge: Standard neural network architectures, such as Multilayer Perceptrons (MLPs) and even specialized networks like SIREN (Sinusoidal Representation Networks), struggle to approximate these singularities efficiently.

• Spectral Bias: MLPs with smooth activations preferentially learn low-frequency components, requiring massive parameter counts to capture sharp gradients or divergences.
• Structural Mismatch: The paper identifies a fundamental theoretical obstruction: coordinate-separable architectures (which model functions as sums of univariate functions, e.g., $f(x,y) = g(x) + h(y)$) cannot represent non-quadratic radial functions. This forces coordinate-wise models to introduce axis-aligned artifacts (e.g., diamond-shaped level sets instead of circles) and fail to capture the true physics, regardless of model capacity.

2. Methodology: Radial Müntz-Szász Networks (RMN)

The authors propose RMN, a structure-matched architecture that explicitly parameterizes the radial power-law structure of the target function.

Core Architecture (RMN-Direct)

Instead of learning weights for fixed activation functions, RMN learns the exponents of power-law bases. The network output is defined as:
$$\phi_{RMN}(x) = \sum_{k=1}^{K} a_k r^{\mu_k} + c_0 \psi_{log}(r; \mu_{log}) + b_0$$
Where:

• $r = \|x\|$ is the radial distance.
• $\{ \mu_k \}$ are learnable exponents (including negative values for singularities like $r^{-1}$).
• $\{ a_k \}$ are learnable coefficients.
• Log-Primitive ($\psi_{log}$): A specialized term handling logarithmic singularities ($\log r$) via the limit $\lim_{\mu \to 0} (r^\mu - 1)/\mu = \log r$. This ensures numerical stability and exact representation of logarithmic behavior.
• Closed-Form Derivatives: The architecture provides analytical expressions for gradients ($\nabla \phi$) and Laplacians ($\Delta \phi$), which are crucial for Physics-Informed Neural Networks (PINNs) without relying on automatic differentiation.

Theoretical Foundation

The paper establishes a Separability Obstruction Theorem (Theorem 14):

Any $C^2$ function that is both radial and additively separable must be quadratic ($f(x) = c\|x\|^2 + b$).

This proves that coordinate-wise models (like coordinate-wise MSN or standard MLPs) are fundamentally incapable of representing singular radial functions (like $1/r$or$\log r$) without massive inefficiency.

Extensions

• RMN-Angular: Incorporates spherical harmonics ($Y_\ell^m$) to handle angular dependence (e.g., crack-tip fields with $\cos(\theta/2)$).
• RMN-MC (Multi-Center): Learns multiple singularity centers $\{c_j\}$ simultaneously, allowing the network to model superpositions of sources (e.g., multi-charge systems) and recover source locations with high precision.

3. Key Contributions

1. Theoretical Obstruction: Formal proof that coordinate-separable architectures fail on radial singularities, explaining why increasing capacity in standard models does not solve the problem.
2. Novel Architecture: Introduction of RMN, which uses learnable power bases and a log-primitive to exactly match the mathematical structure of singular fields.
3. Parameter Efficiency: RMN achieves state-of-the-art accuracy with orders of magnitude fewer parameters than MLPs or SIREN.
4. Interpretability: The learned exponents $\mu_k$ directly correspond to physical singularity orders (e.g., converging to $-1$ for Coulomb potentials), offering scientific insight unavailable from black-box models.
5. Physics-Informed Learning: The provision of closed-form gradients and Laplacians enables efficient training on punctured domains (domains excluding the singularity point).

4. Experimental Results

The authors evaluated RMN across ten 2D and 3D benchmarks, comparing it against MLPs (33,537 parameters), SIREN (8,577 parameters), and coordinate-wise MSN.

• Accuracy:
  • Radial Singularities: RMN achieved 1.5x to 51x lower RMSE than MLPs and 10x to 100x lower RMSE than SIREN.
  • Specific Example: On the 3D Coulomb potential ($1/r$), RMN (27 params) was 51x more accurate than the MLP (33k params).
  • Coordinate-Separable Failure: RMN outperformed coordinate-wise MSN by factors of 72x to 1,652x, validating the separability obstruction theorem.
• Parameter Efficiency:
  • RMN uses only 27 parameters (for $K=12$) compared to 33,537 for MLPs and 8,577 for SIREN.
  • This represents a 1,242x reduction in parameters for comparable or superior accuracy.
• Singularity Recovery:
  • RMN successfully learned the correct singularity order (e.g., $\mu \approx -0.997$ for a $1/r$ target).
  • RMN-MC recovered source centers with errors $< 10^{-4}$.
• Failure Modes:
  • RMN underperforms on smooth, non-radial, or highly oscillatory functions (e.g., $\sin(\pi x)\sin(\pi y)$), where standard MLPs remain superior. This delineates the operating regime of the architecture.
• PINN Performance: In a 3D Poisson equation with a point charge, RMN maintained stable performance under pure physics-based training (no ground truth), whereas the MLP suffered from gradient pathologies and high variance.

5. Significance and Impact

• Scientific Machine Learning (SciML): RMN bridges the gap between classical approximation theory (Müntz-Szász theorem) and deep learning, providing a principled way to handle singularities that are ubiquitous in physics but difficult for generic neural networks.
• Efficiency: The extreme parameter efficiency makes RMN ideal for resource-constrained applications, real-time simulations, and scenarios where ground truth data is scarce (relying on physics constraints).
• Interpretability: Unlike "black box" networks, RMN outputs physically meaningful parameters (singularity orders), allowing researchers to extract scientific insights directly from the model weights.
• Future Directions: The work suggests a paradigm shift toward structure-matched architectures where the network topology is designed to reflect the known mathematical properties of the physical system (e.g., radial symmetry, conservation laws) rather than relying solely on universal approximation via capacity.

In conclusion, the paper demonstrates that for problems dominated by radial singularities, matching the architecture to the physics yields dramatic improvements in accuracy, efficiency, and interpretability compared to generic deep learning approaches.