Inexact Uzawa-Double Deep Ritz Method for Weak Adversarial Neural Networks

Technical Summary: Inexact Uzawa-Double Deep Ritz Method for Weak Adversarial Neural Networks

Problem Statement
The paper addresses the numerical instability inherent in Weak Adversarial Neural Network (WAN) approaches for solving Partial Differential Equations (PDEs). WANs formulate PDE solution as a min-max saddle-point problem where one neural network approximates the trial solution ($u_\theta$) and a second network acts as an adaptive test function ($v_\eta$) to maximize the residual. While conceptually appealing, the adversarial min-max optimization in WANs often suffers from numerical instability, particularly as the residual approaches zero and the identification of the supremizer becomes ill-posed. Previous attempts to stabilize WANs, such as the Double Deep Ritz Method, replaced the adversarial structure with coupled energy minimizations but introduced computationally expensive nested inner loops. The authors aim to develop a robust, iterative solver grounded in classical numerical analysis that retains the mesh-free flexibility of neural networks while avoiding the instability of direct adversarial training.

Methodology
The proposed Uzawa Double Deep Ritz Method integrates the classical Uzawa iteration for saddle-point problems with Deep Ritz neural network approximations. The method is derived from the observation that the minimal residual (MinRes) formulation of a PDE is equivalent to a saddle-point problem.

1. Theoretical Framework: The authors establish that the WAN formulation is structurally equivalent to a constrained Ritz minimization problem. They propose replacing the adversarial min-max optimization with a sequence of two coupled minimization steps performed at each outer iteration:

   • Residual Update: Minimizing a Ritz functional associated with the dual problem to update the residual variable ($r_\eta$).
   • Solution Update: Minimizing a Ritz functional driven by the current residual to update the solution variable ($u_\theta$).
     Both variables are represented by separate neural networks.

2. Inexact Implementation: Recognizing that exact minimization of the Ritz functionals is computationally prohibitive in high-dimensional settings, the method is formulated as an inexact Uzawa scheme. The inner minimization problems are solved approximately using gradient-based optimization. The paper analyzes two specific training strategies:

   • Block-Gradient Training: Updates linear coefficients exactly (or via specific solvers) while updating nonlinear parameters via gradient descent.
   • Full Gradient Training: Updates all parameters simultaneously using standard gradient descent steps.

3. Convergence Analysis: The authors prove that the proposed method acts as an iterative realization of the WAN formulation. They establish a comprehensive convergence theory for the inexact scheme, demonstrating that convergence is guaranteed provided the approximation errors in the inner steps are suitably controlled. This analysis covers both single-gradient-step updates (structurally equivalent to the Arrow-Hurwicz method) and multi-step inner iterations.

Key Contributions

• Theoretical Unification: The paper formally proves (Theorem 3.1) that the Uzawa Double Deep Ritz method is an iterative realization of the WAN objective, thereby connecting adversarial residual minimization with classical saddle-point iterations.
• Inexact Convergence Theory: The authors establish convergence for an inexact Uzawa scheme where both the residual and solution updates are computed approximately (Theorem 4.1). They further prove convergence for practical gradient-based implementations, including single-step and multi-step regimes (Theorems 4.2 and 4.4).
• Algorithmic Design: The proposal of a fully neural network variational solver that replaces unstable adversarial dynamics with sequential, energy-based optimization steps, preserving the mesh-free nature of neural approximations.

Results
Numerical experiments were conducted on a 1D transport problem and a 2D Poisson problem to validate the theoretical findings.

• Convergence: The method demonstrated consistent decay in both the residual norm and the solution error across iterations.
• Robustness of Inexactness: The experiments confirmed that the outer Uzawa iteration remains stable even when the inner Ritz minimization problems are not solved to full convergence. Specifically, the Block-Gradient Training strategy showed high robustness, where the final solution error was largely insensitive to the number of inner iterations, provided the updates moved in the correct descent direction.
• Sensitivity of Full Gradient: In contrast, the Full Gradient Training strategy showed significant sensitivity to the number of inner iterations, requiring more accurate inner solves to achieve comparable convergence rates.
• Step Size Stability: The experiments validated the theoretical stability condition for the step size $\tau$ ($0 < \tau < 2/M^2$), showing slow convergence for small $\tau$ and instability for values exceeding the bound.

Significance and Claims
The paper claims to bridge the gap between classical saddle-point theory and modern neural approximation. By replacing the adversarial min-max training of WANs with a robust, sequential residual-correction procedure, the proposed framework offers a theoretically grounded alternative for solving PDEs. The authors emphasize that their work provides rigorous stability and convergence guarantees for gradient-based implementations, addressing the numerical instabilities that have historically plagued adversarial neural PDE solvers. The method is presented as a "fully neural counterpart" to residual-Uzawa approaches, offering a pathway toward the systematic design of consistent deep learning methods for PDEs without relying on mesh-based discretizations.