Discovering Scaling Exponents with Physics-Informed Müntz-Szász Networks

Technical Summary: Discovering Scaling Exponents with Physics-Informed Müntz-Szász Networks

Problem Statement
Physical systems near singularities, interfaces, and critical points frequently exhibit power-law scaling (e.g., $r^{-1/2}$ for crack tips, $k^{-5/3}$ for turbulent spectra). Identifying the governing scaling exponents is a primary scientific goal, as they encode symmetries, conservation laws, and solution regularity. However, standard neural networks, including Physics-Informed Neural Networks (PINNs), treat these exponents as implicit features hidden within millions of weights. While PINNs can solve forward and inverse problems by minimizing PDE residuals, they lack the architectural capacity to explicitly output the scaling structure. Standard architectures (e.g., MLPs with ReLU or smooth activations) require vast numbers of units to approximate power laws ($N = \Omega(\epsilon^{-2})$ for $x^\alpha$) and fail to provide interpretable exponents, often necessitating fragile post-hoc analysis to extract them.

Methodology: MSN-PINN
The authors introduce Physics-Informed Müntz-Szász Networks (MSN-PINN), a framework that treats scaling exponents as explicit, trainable parameters.

1. The Ansatz: Instead of standard basis functions, the model utilizes a Müntz-Szász expansion:
   $$u_\theta(x) = \sum_{k=1}^K c_k x^{\mu_k}$$
   where both the coefficients $\{c_k\}$ and the exponents $\{\mu_k\}$ are optimized. This allows the network to represent power-law targets exactly with a single term if the exponent is learned correctly.

2. Physics-Informed Training: The network is trained to satisfy Partial Differential Equations (PDEs) and Boundary Conditions (BCs) without labeled solution data. The total loss function combines:

   • PDE Residual Loss ($L_{res}$): Measures violation of the governing equation.
   • Boundary Condition Loss ($L_{BC}$): Enforces data at boundaries.
   • Sparsity Loss ($L_{sparse}$): An $\ell_1$ penalty on coefficients to suppress spurious terms and aid identifiability.

3. Constraint-Aware Training: The authors identify a critical failure mode in "naive" MSN-PINN training for certain problems (e.g., Laplace's equation on a wedge). In these cases, the PDE residual is flat with respect to the exponent $\mu$ (any harmonic function satisfies the PDE), leading to "exponent drift" where the network converges to physically incorrect values (e.g., 14.6% error).
   To resolve this, the authors introduce constraint-aware losses that encode specific physical requirements, such as edge compatibility conditions. For a wedge with angle $\omega$, the constraint $L_{constraint} = \sum |c_k| \sin^2(\mu_k \omega)$ is added. This acts as a quantization force, driving exponents toward the discrete set of physically valid values ($\mu = n\pi/\omega$) and providing the necessary gradient signal for convergence.

4. Optimization Strategy: A two-timescale optimization scheme is employed. Coefficients ($c$) are updated rapidly (fast timescale) to solve the linear least-squares problem for fixed exponents, while exponents ($\mu$) are updated slowly (slow timescale) to navigate the complex, non-linear landscape of the loss function without destabilization.

Key Contributions

• Framework Integration: Embedding MSN within a physics-informed framework to discover exponents directly from PDE residuals without labeled data.
• Theoretical Guarantees:
  • Identifiability: Proving that power-law sums are uniquely determined by their exponents; if a function is represented exactly, the exponents must match the true physical values.
  • Stability: Deriving stability bounds showing that recovery accuracy degrades gracefully with noise, governed by a separation condition analogous to the Rayleigh Limit ($O(\sigma / \Delta^2 \sqrt{N})$).
  • Approximation Rates: Showing that squared error scales as $O(|\mu - \alpha|^2)$, offering superior efficiency compared to standard networks for power-law targets.
• Constraint-Aware Mechanism: Introducing loss terms that encode physical constraints (e.g., BC compatibility) to resolve degeneracies where PDE residuals alone provide insufficient gradient signals.
• Empirical Validation: Demonstrating the method across singular ODEs, corner singularities, and singular forcing problems.

Results
The paper reports high-precision recovery of scaling exponents across several benchmarks:

• Singular ODE: Recovered the exponent $\alpha=0.5$ from physics alone with 1.31% error.
• Corner Singularity (2D Laplace): Recovered the Kondrat'ev exponent ($\mu=2/3$) for a $270^\circ$ wedge. Naive training failed with 14.6% error, while constraint-aware training achieved 0.009% error, matching the analytical result to four significant figures.
• Singular Poisson Problem: Recovered dual exponents induced by singular forcing ($x^{-1/2} \to u \sim x^{3/2}$) with errors of 0.03% and 0.05%.
• Wedge Benchmark: On a 40-configuration wedge benchmark, the method achieved a 100% success rate with a mean error of 0.022%.
• Constraint Impact: Constraint-aware training improved accuracy by three orders of magnitude over naive training in cases where PDE residuals were insufficient.

Significance and Claims
The paper claims that MSN-PINN bridges the gap between the expressiveness of neural networks and the interpretability of asymptotic analysis. By making scaling exponents explicit trainable parameters, the method produces learned parameters with direct physical meaning. The authors state that these learned exponents can validate existing theory, suggest new physics, diagnose model misspecification, and enable extrapolation beyond the training domain. The work positions itself not as a general-purpose solver replacing standard PINNs, but as a specialized tool for problems where identifying the underlying scaling laws is the primary scientific objective.