On $L^\infty$ stability for wave propagation and for linear inverse problems

1. Problem Statement

The paper addresses a fundamental limitation in the stability analysis of forward models (PDEs) and inverse problems. While stability is often guaranteed in $L^2$ norms (based on energy conservation principles), many physical phenomena and applications require stability with respect to the $L^\infty$ norm (supremum norm). The $L^\infty$ norm is crucial for detecting localized features, such as sharp spikes or discontinuities, which $L^2$ norms may smooth over or fail to bound.

The core problems identified are:

• Wave Propagation: The linear wave equation is not stable in $L^\infty$. Small perturbations in initial data (specifically those with high-frequency components or steep gradients) can lead to arbitrarily large deviations in the solution at later times when measured in the $L^\infty$ norm.
• Inverse Problems: Classical regularization techniques (e.g., Tikhonov regularization) are bounded in Hilbert spaces ($L^2$) but often fail to be bounded in $L^\infty$. This leads to adversarial perturbations: small, imperceptible changes in the input data can cause massive, unbounded errors in the reconstructed solution in the $L^\infty$ sense.
• Deep Learning Connection: The instability of Deep Neural Networks (DNNs) against adversarial examples is linked to these mathematical properties. Since residual CNNs can be modeled as discretized PDEs, the lack of $L^\infty$ stability in the underlying continuous models suggests a theoretical basis for DNN vulnerability.

2. Methodology

The authors propose a unified approach to restore $L^\infty$ stability through spectral filtering and regularization.

A. Wave Equation Stabilization

For the linear wave equation, the solution operator is a Fourier multiplier. The authors introduce a double-filtered solution operator $B_{\alpha, \beta}$:

1. Spatial Filtering: Apply a spatial filter $h_\beta$ to the initial data.
2. Frequency Filtering: Apply a frequency filter $\kappa_\alpha$ to the Fourier transform of the filtered data.
3. Operator Definition: The regularized solution is defined as:
   $$B_{\alpha, \beta} f = \mathcal{F}^{-1} \left( \mu \cdot \kappa_\alpha \cdot \mathcal{F}(h_\beta f) \right)$$
   where $\mu$ is the original wave equation multiplier.
4. Approximation: As the filter parameters $\alpha, \beta \to 0$, the operator $B_{\alpha, \beta}$ approximates the true solution $B$ in the $L^\infty$ norm, provided the initial data belongs to a suitable subspace (e.g., functions with integrable Fourier transforms).

B. Inverse Problem Stabilization

For compact linear operators $A: X \to Y$ (where $X=L^2$ and $Y$ is arbitrary, but we seek solutions in $X' = L^\infty$), the authors modify the standard Tikhonov regularization.

• Standard Tikhonov: $T_\alpha^{Tikh} y = \sum \frac{\sigma_n}{\sigma_n^2 + \alpha} \langle y, u_n \rangle v_n$. This is unbounded in $L^\infty$ if the right singular vectors $v_n$ grow too fast relative to the singular values $\sigma_n$.
• Proposed Spectral Filtering: Introduce a sequence of weights $c_n$ to modify the denominator:
  $$T_\alpha^c y = \sum_{n=1}^\infty \frac{\sigma_n}{\sigma_n^2 + \alpha c_n} \langle y, u_n \rangle v_n$$
• Weight Selection: The weights $c_n$ are chosen to grow sufficiently fast (e.g., $c_n \sim \sigma_n^{-\beta}$) to counteract the growth of $\|v_n\|_{L^\infty}$. This effectively penalizes directions in the solution space that correspond to "adversarial" singular vectors.

3. Key Contributions

1. Proof of $L^\infty$ Instability: The paper rigorously demonstrates that the wave equation and standard Tikhonov regularization admit adversarial sequences. These are sequences of bounded inputs (in $L^2$) that produce unbounded outputs (in $L^\infty$) under the respective operators.
2. Construction of $L^\infty$-Stable Solvers:
   • For the wave equation, they prove that the double-filtered operator $B_{\alpha, \beta}$ is bounded from $L^\infty \to L^\infty$ and converges to the true solution as parameters vanish.
   • For inverse problems, they design a spectral filtering method $T_\alpha^c$ that is bounded from $Y \to L^\infty(M)$ under specific conditions on the singular value decomposition (SVD) of the operator $A$.
3. Convergence Rates: The authors derive convergence rates for the proposed spectral filtering method. They establish that if the data satisfies a "source condition" (related to the decay of Fourier coefficients relative to singular values), the error $\|A^\dagger y - T_\alpha^c y\|_{L^\infty}$ converges to zero with a rate of $O(\alpha^{\min\{1, \eta/4\}})$, where $\eta$ depends on the smoothness of the data and the growth of the singular vectors.
4. Connection to Adversarial Examples: The paper formally links the mathematical concept of "adversarial perturbations" in Banach spaces to the phenomenon of adversarial examples in Deep Learning, suggesting that $L^\infty$-stable regularization is a theoretical pathway to stabilizing neural networks modeled by hyperbolic PDEs.

4. Results

• Theoretical Bounds: The paper provides explicit bounds for the operator norms $\|B_{\alpha, \beta}\|_{L^\infty \to L^\infty}$ and $\|T_\alpha^c\|_{Y \to L^\infty}$.
• Numerical Simulations:
  • Wave Equation: Simulations show that while standard wave propagation amplifies small, steep perturbations into massive spikes (adversarial artifacts), the regularized operator $B_{\alpha, \beta}$ suppresses these spikes, maintaining a bounded $L^\infty$ norm.
  • Inverse Problems: In a periodic convolution example with a singular kernel:
    • Tikhonov Regularization: Fails to suppress spikes caused by adversarial noise, leading to unbounded errors in $L^\infty$.
    • Truncated SVD: Works well for smooth signals but suffers from Gibbs phenomena for discontinuous signals.
    • Proposed Method ($T_\alpha^c$): Successfully reconstructs both smooth and discontinuous signals with significantly lower $L^\infty$ error compared to Tikhonov, effectively suppressing the adversarial spikes while preserving the signal structure.

5. Significance

• Mathematical Rigor: The work bridges the gap between $L^2$-based stability (standard in physics and PDEs) and $L^\infty$-based stability (critical for signal processing and image reconstruction). It provides a rigorous framework for understanding why certain physical models and algorithms fail under localized perturbations.
• Robustness in Inverse Problems: The proposed spectral filtering offers a practical alternative to Tikhonov regularization for problems where the solution must be bounded pointwise (e.g., image reconstruction, medical imaging), ensuring that noise does not create unphysical artifacts.
• Deep Learning Implications: By modeling CNNs as PDEs, the paper suggests that the instability of DNNs against adversarial attacks may be an inherent property of the underlying continuous dynamics if they are not $L^\infty$-stable. The proposed regularization techniques offer a theoretical blueprint for designing more robust neural architectures or training objectives that prioritize $L^\infty$ stability.
• Future Directions: The authors note that extending these results to nonlinear PDEs (which fully capture DNN behavior) is a necessary but challenging next step, opening a new avenue for research in the mathematical theory of deep learning.