Variational formulation based on duality to solve partial differential equations: Use of B-splines and machine learning approximants

This paper proposes a dual variational formulation based on convex duality and Lagrange multipliers to solve partial differential equations lacking a primal variational structure, demonstrating that such problems can be accurately solved via Galerkin discretization using smooth B-splines and RePU neural networks to minimize a convex dual functional.

The Gist

Imagine you are trying to solve a massive, complex puzzle. In the world of physics and engineering, these puzzles are called Partial Differential Equations (PDEs). They describe how things move, heat up, or flow—like water in a river, heat spreading through a metal rod, or air swirling around a plane wing.

For many of these puzzles, there is a "golden key" called a Variational Principle. Think of this key as a master map. If you have this map, you don't need to solve the puzzle piece by piece; you just follow the path of "least resistance" (minimizing energy) to find the answer. This is how most standard computer simulations work.

The Problem:
However, some of the most important puzzles in physics (like the flow of viscous fluids or certain heat problems) do not have this golden key. They are "non-variational." Trying to solve them with standard methods is like trying to open a locked door with a screwdriver. You can force it, but it's messy, unstable, and often requires adding artificial "glue" (stabilization tricks) to keep the solution from falling apart.

The Paper's Big Idea: The "Mirror" Trick
The authors, N. Sukumar and Amit Acharya, propose a clever workaround. Instead of trying to force a key into a lock that doesn't exist, they build a mirror.

1. The Primal Problem (The Real World): This is the messy, difficult equation we actually want to solve (e.g., "How does the heat move?").
2. The Dual Problem (The Mirror): They create a completely new, imaginary problem. In this mirror world, the rules are flipped. The difficult equation becomes a simple "constraint" (a rule that must be followed), and they introduce a new, smooth, convex "energy" function that does have a golden key.

How the Mirror Works (The Analogy):
Imagine you are trying to balance a wobbly, jagged rock on a table (the Primal Problem). It's hard to find a stable spot.

• The Old Way: You try to sand down the rock or use glue to make it fit.
• The New Way (Dual Approach): You look at the shadow the rock casts on the wall (the Dual Problem). The shadow is smooth and round. You find the perfect spot to balance the shadow. Once you know where the shadow is balanced, you use a special "translation map" (called the DtP mapping) to instantly know exactly where the jagged rock should be to cast that specific shadow.

By solving the smooth, easy "shadow" problem, they automatically get the correct solution for the jagged, difficult "rock" problem.

The Tools: B-Splines and Neural Networks
To solve these mirror problems on a computer, the authors use two high-tech tools:

1. B-Splines: Think of these as ultra-smooth, flexible plastic rulers. Unlike standard computer grids that look like a staircase (jagged steps), B-splines are like a smooth, continuous curve. They are perfect for mimicking the smooth nature of the "shadow" (dual) problem.
2. Machine Learning (RePU Neural Networks): They also use a type of simple neural network. Imagine a network of tiny, flexible springs that can bend into any shape needed. These "springs" (specifically using a function called RePU) help approximate the solution without needing the massive "training" time usually required by AI. They just solve a set of linear equations, which is fast and precise.

Why This is a Game-Changer:

• No More "Glue": Because they solve the smooth mirror problem, they don't need the messy stabilization tricks that usually cause errors in fluid dynamics.
• Symmetry: The math behind this method creates perfectly symmetrical equations, which computers love because they are easier and faster to solve.
• Accuracy: The authors tested this on heat flow and fluid convection. They found that their method was incredibly accurate, even when the solutions had sharp, sudden changes (like a sudden temperature spike).

The "Time Travel" Quirk
One interesting quirk of this method is how it handles time. Usually, physics problems start at time zero and move forward. But this "mirror" method treats time like a spatial dimension (like width or height). It sets a "boundary condition" at the end of the time period (like a deadline) and solves the whole timeline at once.

• The Catch: Sometimes, the solution gets a little "fuzzy" right at the very end of the timeline (the deadline).
• The Fix: The authors suggest a trick: pretend the timeline goes a tiny bit past the deadline, solve the whole thing, and then just throw away the extra bit. This cleans up the fuzziness perfectly.

In Summary
This paper introduces a brilliant mathematical "hack." When physics gives you a problem that is too jagged to solve directly, don't fight the jaggedness. Build a smooth mirror image of the problem, solve the easy version, and use a translation map to get the answer to the hard version. By combining this with smooth mathematical curves (B-splines) and smart neural networks, they can solve complex fluid and heat problems with high accuracy and without the usual headaches.

Technical Summary

1. Problem Statement

Many critical Partial Differential Equations (PDEs) in physics and engineering—such as the Navier-Stokes equations, inelastic deformation models, and transient convection-diffusion or wave equations—lack a natural, exact primal variational structure.

• The Challenge: Standard variational methods (like the Finite Element Method or Deep Ritz) rely on minimizing a functional derived from the PDE. For equations without this structure (often due to non-self-adjoint operators like convection terms), standard Galerkin methods require stabilization techniques (e.g., SUPG, GLS) involving user-defined tuning parameters, or they suffer from spurious oscillations.
• The Goal: To develop a robust, variational-based solution strategy for these non-variational PDEs that avoids ad-hoc stabilization parameters and leverages modern approximants like neural networks and B-splines.

2. Methodology: The Dual Variational Principle

The authors utilize a dual variational formulation where the original PDE is treated as a constraint rather than the primary objective.

Core Concept

1. Constraint Formulation: The given PDE $G(U) = 0$ (where $U$ is the primal field) is treated as a constraint.
2. Auxiliary Potential: An arbitrarily chosen, strictly convex auxiliary potential $H(U)$ is introduced.
3. Lagrangian Construction: A Lagrangian $\mathcal{L}(U, \lambda) = H(U) + \lambda \cdot G(U)$ is formed, where $\lambda$ represents the dual fields (Lagrange multipliers).
4. Dual-to-Primal (DtP) Mapping: By minimizing the Lagrangian with respect to the primal variables $U$ (setting $\nabla_U \mathcal{L} = 0$), an explicit mapping $U = U_H(\lambda)$ is derived. This expresses the primal solution in terms of the dual variables and their derivatives.
5. Dual Functional: Substituting the DtP map back into the Lagrangian yields a convex dual functional $S(\lambda)$ that depends only on the dual variables.
6. Optimization: The problem is transformed into maximizing this convex dual functional subject to Dirichlet boundary conditions on the dual variables.
   • Key Insight: Even if the primal PDE is non-elliptic or hyperbolic, the resulting dual problem is locally degenerate elliptic, ensuring well-posedness for standard Galerkin discretization without upwinding.

Discretization Strategy

The paper employs a Galerkin discretization for the dual weak form using two types of smooth approximants:

• Machine Learning Approximants: Shallow neural networks using Rectified Power Unit (RePU) activation functions ($\text{ReLU}^p$). These allow for high-order continuity ($C^{p-1}$) and are constructed with fixed weights/biases in the hidden layer, making the system linear in the output weights.
• B-splines: Univariate B-splines for 1D problems and tensor-product B-splines for space-time (2D) problems. These provide high-order smoothness and compact support.

3. Key Contributions

• Generalized Dual Framework: The authors extend the dual variational principle to transient convection-diffusion and heat equations, deriving the specific dual weak forms and DtP mappings for these problems.
• Symmetric Stiffness Matrices: Unlike standard Galerkin methods for convection-diffusion which often yield non-symmetric matrices requiring stabilization, the dual formulation naturally produces symmetric stiffness matrices, simplifying the linear algebra.
• Handling Transient Problems: The method treats transient initial-value problems (IVPs) as boundary-value problems (BVPs) in space-time. It imposes a terminal Dirichlet condition on the dual field, effectively converting the time-marching problem into a single global optimization problem.
• Convergence Analysis: The paper establishes rigorous convergence rates in the $L^2$ norm and $H^1$ seminorm for steady-state problems, demonstrating optimal rates of $O(h^p)$ and $O(h^{p+1})$ respectively.
• Boundary Condition Flexibility: The method shows that the dual variables can satisfy homogeneous Dirichlet conditions easily, and non-homogeneous primal boundary conditions appear as natural boundary terms in the dual functional.

4. Numerical Results

The authors validated the method on several benchmark problems:

• Laplace Equation: Demonstrated that the method recovers exact solutions using low-order polynomial ansatzes for the dual fields, confirming the theoretical consistency.
• Steady-State Convection-Diffusion:
  • Tested with high Peclet numbers ($\alpha = 10, 50$) where boundary layers form.
  • Neural Networks: RePU-based networks achieved high accuracy ($10^{-3}$ to $10^{-5}$ errors) without spurious oscillations, even with coarse discretizations.
  • B-splines: Showed superior convergence rates. For $p=2$ (quadratic) and $q=3$ (cubic) dual fields, the flux ($u'$) was more accurate than the solution ($u$), with errors decreasing as $O(h^p)$ and $O(h^{p+1})$.
• Transient Convection-Diffusion & Heat Equations:
  • Solved using space-time tensor-product B-splines.
  • Accuracy: The method produced accurate solutions for both temperature and flux.
  • Terminal Time Issue: The authors identified that errors tend to concentrate near the terminal time ($t=T$) due to the interaction between the primal initial/boundary conditions and the dual terminal boundary condition. They proposed a "buffer-zone" strategy (solving in $[0, T+\delta]$ and discarding the extension) to mitigate this, a technique validated in prior work.

5. Significance and Impact

• Unified Variational Approach: This work provides a unified framework to solve a broad class of PDEs (including those without primal variational structures) using standard, stable Galerkin methods.
• Elimination of Stabilization Parameters: By moving to the dual space, the need for user-defined stabilization parameters (like $\tau$ in SUPG) is removed, making the method more robust and automated.
• Synergy of ML and FEM: The successful integration of RePU neural networks and B-splines within a dual variational framework bridges the gap between traditional numerical methods and modern machine learning approaches (like PINNs). Unlike PINNs, which minimize the strong form residual, this approach minimizes a convex dual functional, ensuring better physical consistency and avoiding the training difficulties associated with non-convex loss landscapes.
• Computational Efficiency: The resulting linear systems are symmetric and positive definite (for the dual), allowing for efficient solvers. The method avoids the iterative training loops typical of deep learning, solving the system directly via matrix inversion.

In conclusion, the paper demonstrates that the dual variational principle, combined with high-order smooth approximants, offers a powerful, accurate, and theoretically sound alternative for solving challenging PDEs that traditionally require complex stabilization techniques.