Shape Derivative-Informed Neural Operators with Application to Risk-Averse Shape Optimization

The Big Picture: Designing in a Foggy World

Imagine you are an architect trying to design the perfect bridge. But there's a catch: you don't know exactly how strong the wind will be, or if the ground beneath the bridge will shift slightly. This is Optimization Under Uncertainty. You aren't just looking for a good bridge; you are looking for a bridge that won't collapse even if the wind is stronger than expected or the ground is softer than predicted.

Traditionally, to find this perfect design, engineers have to run thousands of computer simulations. They change the shape of the bridge, simulate the wind, check if it breaks, change the shape again, and repeat.

The Problem: These simulations are like trying to solve a massive, complex math puzzle. Doing it once takes hours. Doing it 10,000 times to account for uncertainty? That would take years. It's too slow and too expensive.

The Old Solution: The "Guess and Check" Robot

To speed things up, scientists started using Neural Networks (a type of AI). Think of these as "surrogate models" or "cheat sheets."

Instead of solving the hard math puzzle every time, the AI looks at a picture of the bridge and the wind conditions, and guesses the answer based on patterns it learned from previous puzzles.
The Flaw: Standard AI is great at guessing the result (e.g., "The bridge will sway 2 meters"). But it is terrible at guessing the direction to improve the design (e.g., "If I make the bridge 1% wider, the sway drops by 10%").
In optimization, knowing the direction is crucial. If your AI gives you a wrong direction, you might walk off a cliff thinking you are walking toward safety.

The New Solution: Shape-DINO (The "Super-Teacher" AI)

The authors of this paper created a new framework called Shape-DINO. Think of it as a "Super-Teacher" AI that doesn't just give you the answer; it also teaches you how to get a better answer.

Here is how it works, broken down into three simple concepts:

1. The "Magic Trampoline" (Diffeomorphic Mappings)

Designing shapes is hard because every time you change the shape, the computer has to redraw the entire grid (mesh) of the simulation. It's like trying to measure a room every time you move the furniture, but the floor tiles keep changing size.

The Fix: Shape-DINO uses a "Magic Trampoline." It stretches and squashes the new shape back into a standard, fixed rectangle (the reference domain).
The Analogy: Imagine you have a piece of clay. Instead of reshaping the clay and then trying to measure it, you stretch the clay back into a perfect square every time you want to measure it. The AI learns the rules of the square, and the "Magic Trampoline" handles the stretching. This makes the learning process much faster and consistent.

2. The "Teacher's Notes" (Derivative-Informed Learning)

This is the secret sauce.

Standard AI: Learns by looking at the final score. "Oh, this bridge design got a score of 80/100. I'll remember that."
Shape-DINO: Learns by looking at the score and the teacher's notes on how to improve it. "This bridge got 80/100. If I thicken the left pillar, the score goes up. If I thin the right pillar, it goes down."
The Metaphor: Imagine learning to play golf.
- Standard AI watches you hit a ball and says, "That was a 70-yard shot."
- Shape-DINO watches you, then says, "That was a 70-yard shot, but if you tilt your club 2 degrees to the left, you'll hit 80 yards."
By forcing the AI to learn these "sensitivity" rules (derivatives), it becomes incredibly accurate at finding the best design, not just guessing random ones.

3. The "Compression" (Reduced Basis)

The data from these simulations is huge (like a 4K movie file). Storing and processing it all is slow.

The Fix: Shape-DINO uses a "compression algorithm" (like turning a 4K movie into a high-quality MP4). It finds the most important features of the bridge and the wind, ignoring the tiny, unimportant details.
This allows the AI to run on a standard computer chip (GPU) in milliseconds, whereas the original simulation might take hours on a supercomputer.

The Results: Why It Matters

The paper tested this on three difficult problems:

Heat flow through a shape with a wiggly top.
Airflow around a 2D object (like a car or plane wing).
Wind force on a 3D tower (like a skyscraper).

The Findings:

Speed: Shape-DINO was 1,000 to 100,000,000 times faster than the traditional method when making predictions.
Accuracy: Because it learned the "directions" (derivatives), it found better designs much faster. Standard AI often got stuck in local loops, while Shape-DINO found the global optimum.
Efficiency: To find a good design, Shape-DINO needed to run the expensive physics simulation only 10 to 100 times to train. The traditional method needed to run it 1,000 to 10,000 times just to get a similar result.

The Bottom Line

Shape-DINO is like giving an architect a crystal ball that not only shows them the future (what the wind will do) but also gives them a map of exactly how to tweak their design to survive that future.

It solves the "too slow" problem of designing safe, efficient structures in an uncertain world. Instead of waiting years to simulate thousands of possibilities, engineers can now do it in days, or even hours, ensuring that the bridges, planes, and towers we build are robust against the unexpected.

1. Problem Statement

The paper addresses Shape Optimization under Uncertainty (OUU) for systems governed by Partial Differential Equations (PDEs). In many engineering applications (e.g., aerodynamics, fluid-structure interaction), the geometry of the domain is a design variable ( $z$ ), while physical parameters (e.g., material properties, boundary conditions) are uncertain ( $m$ ).

The core challenges in solving these problems are:

Computational Cost: Traditional methods require repeated high-fidelity PDE solves for thousands of uncertainty realizations (sampling) across varying geometries.
Geometric Variability: Changing the shape $z$ typically requires remeshing or complex mesh deformation, which is computationally expensive.
Derivative Accuracy: Risk-averse optimization (minimizing worst-case scenarios or Conditional Value-at-Risk) requires accurate gradients with respect to the shape variables. Standard neural operator surrogates often fail to provide accurate derivatives, leading to poor optimization convergence or spurious local minima.
Risk Aversion: Optimizing a scalar risk measure (e.g., CVaR, Entropic Risk) rather than just the mean requires capturing the tail behavior of the solution distribution, which is highly sensitive to gradient errors.

2. Methodology: Shape-DINO

The authors propose Shape-DINO (Shape Derivative-Informed Neural Operator), a framework that learns the PDE solution operator and its Fréchet derivatives simultaneously.

A. Mathematical Formulation

Reference Domain Mapping: To handle geometric variability, the authors map all varying domains $\Omega_z$ to a fixed reference domain $\Omega_0$ using diffeomorphic mappings $\chi_z$ . This allows the learning problem to be posed on a fixed mesh.
Solution Operator: The goal is to learn the map $(m, z) \mapsto u(m, z)$ , where $u$ is the PDE state.
Derivative-Informed Learning: Unlike standard operators that minimize only the $L^2$ $L^{2}$ error of the state, Shape-DINO minimizes a Sobolev-type loss ( $H^1_\mu$ $H_{μ}^{1}$ ) that includes errors in:
1. The state $u$ .
2. The derivative with respect to uncertain parameters $D_m u$ .
3. The derivative with respect to shape variables $D_z u$ (shape sensitivities).
  The loss function is:
  $\mathcal{L}(\theta) = \mathbb{E} \left[ \|u - u_\theta\|^2 + \|D_m u - D_m u_\theta\|^2 + \|D_z u - D_z u_\theta\|^2 \right]$

B. Architecture and Dimension Reduction

Reduced Basis Neural Operators (RBNO): To handle high-dimensional inputs/outputs, the authors use a reduced-basis architecture.
- State Reduction: Uses Proper Orthogonal Decomposition (POD) (PCA) on the state variable $u$ to project it into a low-dimensional latent space.
- Parameter Reduction: Uses Active Subspaces (AS) for the uncertain parameter $m$ . AS is chosen because it utilizes derivative information to find directions in the parameter space that most significantly impact the output, making it more efficient than standard PCA for inputs.
Neural Network: A multi-input dense neural network maps the reduced coefficients of $(m, z)$ to the reduced coefficients of $u$ and its Jacobians.

C. Training Workflow

Data Generation: Solve high-fidelity PDEs for sampled $(m, z)$ pairs.
Sensitivity Computation: Compute $D_m u$ and $D_z u$ using adjoint methods (solving linearized PDEs).
Basis Construction: Compute POD for states and AS for parameters from the training data.
Training: Train the neural network to minimize the joint loss of states and derivatives in the latent space.

3. Key Contributions

Theoretical Contributions

A Priori Error Bounds: The authors prove that for strongly convex objectives, the optimization error of the surrogate-based solution is bounded by the error in the derivatives of the objective function. Specifically, they show that controlling the $L^2$ error of the state and the $L^2$ error of the shape derivative is sufficient to guarantee convergence to the true optimum.
Universal Approximation: They establish universal approximation theorems for multi-input reduced-basis neural operators in $C^1$ norms. This proves that such networks can approximate both the solution operator and its derivatives arbitrarily well, justifying their use in gradient-based optimization.

Algorithmic Contributions

Unified Framework: A single framework that handles simultaneous variations in geometry ( $z$ ) and stochastic parameters ( $m$ ) via diffeomorphic mappings.
Efficient Derivative Generation: Demonstrates that generating Jacobian data for training adds negligible computational cost (often reusing matrix factorizations from the state solve) compared to the gains in optimization accuracy.

4. Numerical Results

The framework was tested on three challenging problems:

Poisson Equation (2D): Flux-tracking with uncertain permeability and a variable top boundary.
- Result: Shape-DINO achieved near-reference accuracy with 512 training samples, whereas standard operators (Shape-NO) and PDE-based Sample Average Approximation (SAA) required significantly more samples or failed to converge to the same accuracy.
Navier-Stokes (2D): Dissipation reduction around an object under uncertain inflow (East/North directions).
- Result: Shape-DINO reduced the optimal dissipation value using 10x fewer PDE solves than the PDE-based approach. Shape-NO failed to match the performance of the PDE baseline.
Navier-Stokes (3D): Drag and bending-moment reduction on a tower structure.
- Result: In this computationally prohibitive 3D setting, Shape-DINO produced designs with significantly lower force and moment variability compared to deterministic PDE optimization and Shape-NO.

Performance Metrics

Accuracy: Shape-DINO consistently outperformed Shape-NO (trained without derivatives) in both state prediction and, crucially, in the quality of the final optimized design.
Speedup:
- Online Evaluation: Shape-DINO provided 3–8 orders of magnitude speedup in state and gradient evaluations compared to PDE solvers.
- Offline Cost: Including training data generation, Shape-DINO reduced the total number of required PDE solves by 1–2 orders of magnitude compared to a strictly PDE-based OUU approach for a single problem.
Generalization: Shape-DINO demonstrated robustness in out-of-distribution scenarios (e.g., different target flux profiles) where Shape-NO failed.

5. Significance

This work bridges a critical gap between Scientific Machine Learning and Risk-Averse Engineering Design.

Reliability: It demonstrates that for optimization tasks, learning the derivative is not just a "nice-to-have" but a necessity for reliable convergence. Standard $L^2$ -trained surrogates can lead to suboptimal or unstable designs in risk-averse settings.
Scalability: By combining diffeomorphic mappings, reduced bases, and derivative-informed training, the method makes large-scale, 3D, risk-averse shape optimization computationally feasible, a task previously intractable due to the "curse of dimensionality" in both geometry and uncertainty.
Theoretical Foundation: It provides the first rigorous theoretical justification (error bounds and universal approximation) for using neural operators in PDE-constrained optimization under uncertainty, moving beyond heuristic applications.

In summary, Shape-DINO offers a computationally efficient, theoretically grounded, and highly accurate framework for designing complex systems under uncertainty, significantly outperforming both traditional PDE-based methods and standard neural operator surrogates.