A Learning-Based Ansatz Satisfying Boundary Conditions in Variational Problems

Technical Summary: A Learning-Based Ansatz Satisfying Boundary Conditions in Variational Problems

Problem Definition
Variational problems, which seek to find critical points of functionals (often leading to Partial Differential Equations via Euler-Lagrange equations), are fundamental in physics and engineering. When exact solutions are unavailable, approximation techniques like the Ritz method are employed. Recently, the Deep Ritz Method (DRM) has integrated deep learning into this framework by using neural networks as trial functions. However, a critical limitation of the DRM is that standard neural networks do not inherently satisfy the boundary conditions of the problem. To compensate, the DRM introduces a penalty term into the action functional. This approach presents two significant drawbacks:

1. Hyperparameter Sensitivity: The penalty term is weighted by a hyperparameter ($\beta$) that must be tuned; improper tuning can lead to misleading optimization results.
2. Non-Convexity and Approximation: The penalty term can render the action functional non-convex, complicating the search for global minima. Furthermore, the boundary conditions are only approximately satisfied, not exactly, which can result in solutions that minimize the penalized action but fail to represent the true physical solution.

Methodology
The authors propose a novel ansatz (trial function) that inherently satisfies the boundary conditions, thereby eliminating the need for penalty terms. The proposed trial function $\hat{y}(u)$ is constructed as:
$$\hat{y}(u, \theta) = B(u) + p(u) \cdot N_{net}(u, \theta)$$
Where:

• $B(u)$ is a function that exactly matches the prescribed boundary values ($\hat{b}(u)$) on the domain boundary $\partial \Omega$.
• $p(u)$ is a polynomial (or suitable function) chosen to vanish at the boundary ($p(u)=0$ for $u \in \partial \Omega$).
• $N_{net}(u, \theta)$ is a single-hidden-layer neural network with parameters $\theta$.

This construction ensures that for any parameters $\theta$, the trial function satisfies the boundary conditions exactly. The optimization process then minimizes the original action functional $S$ directly, rather than a penalized version $S_{DR}$. The minimization is performed using gradient descent on the parameters $\theta$.

Theoretical Contributions
A primary contribution of this work is the rigorous theoretical justification of the proposed ansatz within the framework of Sobolev spaces, which are the natural setting for variational problems involving derivatives.

• Universal Approximation in Sobolev Spaces: The authors extend the Universal Approximation Theorem to products of functions and neural networks. They prove that if $f$ and $g$ are in $C^m(\Omega)$ and $f/g \in C^m(\Omega)$, then $f$ can be approximated arbitrarily well in the Sobolev norm $\|\cdot\|_{m,p,\Omega}$ by the product $g(x) \cdot N_{net}(x)$.
• Admissibility: By establishing that the product of a polynomial vanishing at the boundary and a neural network is dense in the relevant Sobolev space, the paper proves that the proposed ansatz constitutes a valid set of admissible functions for the Ritz method. This provides a mathematical guarantee of the ansatz's expressiveness and ability to approximate the true solution.

Results
The paper validates the method through three numerical examples, comparing the proposed approach against the standard Deep Ritz Method:

1. 1D Variational Problem: The proposed method (using $N=2$ neurons) converged to four significant digits in 4,200 iterations. The DRM required significantly more iterations (29,200 for $N=2$ and 29,400 for $N=30$) to reach similar precision. Notably, while the DRM with $N=30$ achieved a lower final action value, the resulting solution was less accurate than the proposed method's solution, demonstrating that minimizing the penalized action can lead to misleading outcomes. The proposed method achieved an absolute error 100 times smaller than the DRM.
2. 2D Poisson-like Problem: The proposed method ($N=5$) converged in 2,800 iterations, whereas the DRM (requiring an additional hidden layer, $N=10$) took 9,800 iterations. The DRM solution exhibited non-monotonic convergence behavior due to the penalty term, while the proposed method showed monotonic decrease. The proposed method achieved errors $<0.0005$, compared to errors bounded by $0.09$ for the DRM.
3. Quantum Oscillator (Eigenvalue Problem): The proposed method estimated the minimum eigenvalue with a relative error of 0.3% ($\lambda \approx 3.0096$), while the DRM yielded a relative error of 19.8% ($\lambda \approx 3.5939$). The DRM also exhibited spurious oscillations during convergence.

Significance and Claims
The paper claims that the proposed ansatz offers a robust alternative to the Deep Ritz Method by:

• Eliminating Hyperparameter Dependence: Removing the need to tune penalty weights for boundary conditions.
• Ensuring Exact Boundary Satisfaction: Preventing the "misleading" optimization outcomes where the action is minimized at the cost of violating boundary constraints.
• Reducing Complexity: Achieving higher accuracy with simpler neural network architectures (fewer neurons and layers) compared to the DRM.
• Providing Rigorous Justification: Unlike previous works that proposed similar ansatzes without theoretical backing, this study establishes the density of the proposed function class in Sobolev spaces, ensuring the method is mathematically sound for variational problems.

The authors conclude that satisfying boundary conditions exactly is critical for obtaining physically meaningful solutions and that their approach effectively balances accuracy, complexity, and theoretical rigor.