Modified Wavenumber Analysis Extended to Physics-Informed Neural Networks

Technical Summary: Modified Wavenumber Analysis Extended to Physics-Informed Neural Networks

Problem Statement
Physics-Informed Neural Networks (PINNs) have emerged as a versatile tool for solving partial differential equations (PDEs) by embedding physical laws into the training process. However, their application to high-frequency wave problems and multiscale regimes is hindered by "spectral bias," a tendency to learn low-frequency components more easily than high-frequency ones. While various architectural enhancements (e.g., Fourier feature encoding, specialized activation functions) have been proposed to mitigate this, there is a lack of rigorous, frequency-dependent metrics to quantify PINN accuracy. Unlike classical numerical methods (e.g., finite difference schemes), which possess well-established frameworks like modified wavenumber analysis to characterize dispersion and dissipation errors, PINNs lack an analogous diagnostic tool. This gap makes it difficult to systematically compare PINN performance against traditional methods or to understand how specific network components affect spectral resolution.

Methodology
The authors extend the classical modified wavenumber analysis to PINNs to create a quantitative framework for evaluating spectral accuracy. The methodology involves three core components:

1. Theoretical Extension: The study adapts the concept of the modified wavenumber ($k_{eff}$), traditionally used to analyze finite difference schemes, to neural network approximations. For a periodic wave function, the exact derivative is compared against the PINN output. The effective wavenumber is extracted by fitting the PINN solution to a sinusoidal basis function via orthogonal projection.
2. Spectral Decomposition: The computed $k_{eff}$ is decomposed into:
   • Dispersion ($\Re[k_{eff}]$): The real part, governing phase velocity errors.
   • Dissipation ($\Im[k_{eff}]$): The imaginary part, governing amplitude decay or artificial damping.
     These metrics are normalized against the Nyquist frequency ($k_{max} = \pi/\Delta x$) to allow comparison across different resolutions.
3. Numerical Experiments: The framework is first validated against classical finite difference schemes (central differences C2, C4, C6 and upwind schemes U1, U2) on a periodic domain. Subsequently, it is applied to PINNs to assess the impact of:
   • Architecture: Varying network depth (2 vs. 4 layers) and width (64 vs. 128 neurons).
   • Activation Functions: Comparing standard hyperbolic tangent (tanh) against sinusoidal and tanh-Gaussian (TG) activations.
   • Fourier Features: Testing different numbers of frequency modes ($F = 2, 4, 6$) in the input encoding.

Key Results
The study yields several specific findings regarding the spectral behavior of PINNs:

• Validation of Framework: The modified wavenumber analysis successfully reproduced known spectral characteristics of classical finite difference schemes (e.g., the purely dispersive nature of central differences and the dissipative nature of upwind schemes), confirming the framework's viability as a "black-box" diagnostic tool.
• Depth vs. Width: Network depth is more effective than width for improving spectral resolution. A 4-layer network with 64 neurons ($4\times64$) maintained dispersion accuracy up to a normalized wavenumber of $k^* \approx 0.6$, whereas a 2-layer network with 64 neurons ($2\times64$) degraded significantly earlier at $k^* \approx 0.45$.
• Activation Functions: Sinusoidal and tanh-Gaussian (TG) activations demonstrated superior spectral fidelity compared to standard tanh. While all activations performed well at low frequencies ($k^* < 0.6$), tanh exhibited significant deviation and oscillatory dissipation at higher frequencies, whereas sinusoidal and TG activations maintained near-zero dissipation up to $k^* \approx 0.6$.
• Fourier Feature Sensitivity: The number of Fourier modes ($F$) critically influences accuracy, independent of training loss.
  • $F=4$ provided near-exact solutions across most wavenumbers.
  • $F=2$ and $F=6$ configurations suffered substantial accuracy degradation at higher wavenumbers, despite achieving low training losses. This highlights that low training error does not guarantee high-frequency spectral accuracy.
• Training Efficiency: The inclusion of Fourier features significantly accelerated training convergence (by a factor of approximately 50), though optimal accuracy required careful tuning of the number of modes.

Significance and Claims
The paper claims to bridge a theoretical gap between neural network approximations and classical numerical analysis. By establishing modified wavenumber analysis as a diagnostic tool for PINNs, the authors provide a rigorous, quantitative framework for:

1. Comparative Analysis: Enabling systematic comparison of PINN performance with finite difference and spectral methods on equivalent analytical grounds.
2. Architectural Design: Offering frequency-dependent metrics to guide the selection of network depth, activation functions, and Fourier feature parameters for wave-dominated applications.
3. Validation: Demonstrating that conventional loss metrics can mask deficiencies in high-frequency accuracy, thereby advocating for spectral validation in PINN workflows.

The authors position this work not as a definitive guide to the "optimal" PINN architecture for all problems, but as a methodological contribution that introduces a new analytical perspective for understanding and characterizing the spectral behavior of physics-informed learning.