Monolithic Multi-level Overlapping Schwarz Solvers for Fluid Problems

Technical Summary: Monolithic Multi-level Overlapping Schwarz Solvers for Fluid Problems

Problem Statement
The paper addresses the numerical simulation of incompressible fluid dynamics, specifically solving the discretized Stokes equations which result in large-scale saddle point systems. As geometric complexity increases and mesh resolutions become finer to resolve boundary layers, direct solvers become infeasible due to prohibitive computational costs and memory consumption. The authors focus on the scalability limitations of standard two-level domain decomposition methods, specifically the generalized Dryja–Smith–Widlund (GDSW) approach. In a two-level scheme, as the number of subdomains increases, the coarse problem ($A_0$) grows in size, eventually rendering its direct factorization computationally intractable.

Methodology
The authors propose and implement a monolithic multi-level overlapping Schwarz preconditioner to overcome the scalability limits of the two-level GDSW approach.

• Mathematical Formulation: The method extends the two-level GDSW preconditioner by applying it recursively. Instead of directly inverting the coarse problem $A_0$, the authors replace $A_0^{-1}$ with an approximation derived from a coarser level (a three-level or multi-level hierarchy). The preconditioner is defined as:
  $$B_{GDSW, 3-level} = \Phi^{(2)} \left( \Phi^{(3)} A_0^{(3)-1} \Phi^{(3)T} + \sum_{i=1}^{N^{(2)}} R_i^{(2)T} A_i^{(2)-1} R_i^{(2)} \right) \Phi^{(2)T} + \sum_{i=1}^{N^{(1)}} R_i^{(1)T} A_i^{(1)-1} R_i^{(1)}$$
  This structure allows the coarse problem to be solved via a further level of domain decomposition rather than a direct solver.

• Implementation: The solver is implemented within the FROSch (Fast and Robust Overlapping Schwarz) library, part of the ShyLU package in the Trilinos ecosystem.

  • Coupling: A scalable interface connects FROSch with the FEATFLOW finite element library. FEATFLOW handles the assembly of the system matrix and right-hand side vectors using $Q_2$ elements for velocity and discontinuous $P_1$ elements for pressure.
  • Algebraic Nature: The implementation is purely algebraic, requiring only information about the parallel distribution of degrees of freedom (dofs). It constructs local subdomain matrices, restriction/interpolation operators, and the coarse problem automatically.
  • Coarse Space Construction: For the pressure variable, which is discontinuous across subdomains, the authors utilize constant functions over the whole subdomain as coarse basis functions, avoiding the need for a pressure interface $\Gamma_p$. For the velocity, energy-minimizing extensions of the null space restrictions are used.

Key Contributions

1. Extension to Multi-level: The work successfully extends the monolithic two-level GDSW preconditioner (previously introduced in [7]) to a three-level (and potentially multi-level) framework specifically for saddle point fluid problems.
2. Scalable Coupling: The development of an interface between the FEATFLOW library and the Trilinos/FROSch framework, enabling efficient coupling of geometric assembly with algebraic domain decomposition solvers.
3. Large-Scale Parallel Results: The authors demonstrate weak scaling results on high-performance computing (HPC) architectures, achieving scalability up to 32,768 MPI ranks for a Poiseuille flow on a unit cube and up to 4,096 MPI ranks for a complex extrusion die geometry.

Numerical Results
The authors evaluated the solver using two test cases: a Poiseuille flow on a unit cube and a Poiseuille flow on a complex extrusion die geometry.

• Unit Cube (Structured Decomposition):

  • Two-Level Solver: At 32,768 subdomains, the two-level solver failed because the coarse problem (407,837 dofs) could not be factorized. The setup time for the coarse problem factorization became the bottleneck.
  • Three-Level Solver: Successfully solved the problem at 32,768 subdomains. While the number of Krylov iterations increased from 41 to 66 as subdomains grew by a factor of 64, the solver remained robust. The three-level approach reduced the setup time significantly compared to the two-level approach for large subdomain counts, avoiding the infeasible factorization of the massive coarse problem.

• Extrusion Die (Unstructured Decomposition):

  • Two-Level Solver: At 4,096 subdomains, the setup time was dominated by the coarse problem factorization (84.3s).
  • Three-Level Solver: Demonstrated a performance gain. The numerical setup time decreased by a factor of approximately 6 compared to the two-level solver, and the solution time decreased by nearly a factor of 2. The three-level approach successfully handled the unstructured decomposition where the two-level approach faced scalability limits.

Significance and Claims
The paper claims that the primary significance of this work is the successful demonstration that multi-level monolithic overlapping Schwarz preconditioners can overcome the scalability bottlenecks inherent in two-level GDSW methods for fluid problems. Specifically:

• The three-level approach allows for the solution of problems with significantly larger numbers of subdomains (up to 32,768) where the two-level coarse problem would otherwise be infeasible to factorize.
• The method provides a performance gain in both setup and solve times for large-scale problems, particularly when the coarse space grows large.
• The work represents a step toward efficient Computational Fluid Dynamics (CFD) simulations on future exascale architectures with heterogeneous CPU-GPU systems, as part of the German "StroemungsRaum" project.

The authors remain modest regarding the increase in Krylov iterations at very high subdomain counts, noting that the preconditioner may not yet be in its asymptotic range, but emphasize that the iteration growth is acceptable given the massive increase in parallelism.