Knowledge-guided generative surrogate modeling for high-dimensional design optimization under scarce data

This paper introduces RBF-Gen, a knowledge-guided generative surrogate modeling framework that integrates scarce experimental data with subject matter expert domain knowledge to significantly enhance predictive accuracy and physical meaningfulness in high-dimensional mechanical and process design optimization.

The Gist

Imagine you are a master chef trying to recreate a famous, complex dish, but you've only tasted it four times. You don't have the recipe, and you can't afford to buy more ingredients to experiment. You need to guess the recipe so perfectly that you can cook it for a banquet tomorrow.

This is the exact problem engineers face when designing things like airplane wings, car parts, or semiconductor chips. They have high-fidelity computer simulations (the "perfect recipe"), but running them takes days or costs thousands of dollars. So, they only have a handful of data points (the "four tastes").

If they try to guess the recipe using standard math tools, they often get it wrong because there isn't enough data to fill in the blanks. They might guess the dish needs salt when it actually needs sugar.

Enter "RBF-Gen": The Chef's Secret Mentor.

This paper introduces a new method called RBF-Gen. Think of it as a smart assistant that doesn't just look at your four taste samples; it also listens to a Master Chef (the Subject Matter Expert) who whispers hints like, "The sauce gets thicker the longer you cook it," or "It should never be bitter."

Here is how RBF-Gen works, broken down into simple concepts:

1. The Problem: The "Blank Canvas" Dilemma

Imagine you have a canvas with only four dots of paint on it. You need to draw a picture that connects them.

• Standard Method (RBF): You try to draw a single line connecting the dots. But because you have so few dots, you could draw a million different lines that all fit those four points. Most of them look like scribbles and don't make sense as a real picture.
• The Issue: In high-dimensional problems (like designing a car with 50 different parts), the "canvas" is huge, and the "dots" are tiny. Standard math gets lost in the blank space.

2. The Solution: The "Infinite Possibilities" Box

RBF-Gen changes the rules. Instead of forcing the computer to draw just one line, it creates a box of infinite possible lines that all fit your four dots perfectly.

• Think of this as a "What If?" machine. It generates thousands of different versions of the recipe that all match your four taste tests.
• Some versions say, "Add more sugar!" Others say, "Add more salt!" All of them match the four tastes you have.

3. The Secret Sauce: The Generator Network

Now, we have too many options. How do we pick the right one?

• This is where the Generator Network comes in. It's like a filter or a sieve.
• We tell the sieve: "Keep the recipes that get thicker as they cook" (Monotonicity) and "Throw away any recipe that tastes bitter" (Positivity).
• The sieve uses Domain Knowledge (the expert's hints) to filter out the nonsense options and keep only the ones that make physical sense.

4. The Result: A "Physically Meaningful" Guess

By combining the few data points you have with the expert's rules, RBF-Gen finds the "Goldilocks" solution. It doesn't just guess; it guesses intelligently.

The Analogy in Action:

• Without RBF-Gen: You guess the recipe based only on the four tastes. You might guess the cake is chocolate when it's actually vanilla, because you didn't have enough samples to tell the difference.
• With RBF-Gen: You guess based on the four tastes plus the expert's rule: "This cake is always vanilla." Even with limited data, your guess is much closer to the truth because you used the expert's wisdom to guide you.

Why Does This Matter?

The paper tested this on three real-world scenarios:

1. A Cantilever Beam: Like a diving board. RBF-Gen figured out the best shape to hold weight with very few simulations.
2. A Shell Structure: Like a thin metal roof. It handled complex 2D shapes better than standard methods.
3. Semiconductor Etching: This is the big one. Making computer chips is incredibly expensive and complex. The researchers used RBF-Gen on real factory data where they only had 34 experiments for 17 different variables.
   • The Result: RBF-Gen predicted the outcome of new experiments much better than standard methods. It helped engineers optimize the chip-making process without needing to run hundreds of expensive tests.

The Bottom Line

RBF-Gen is a bridge between "Not Enough Data" and "Expert Wisdom."

In a world where data is expensive and time is short, this method allows engineers to build better, safer, and more efficient designs by teaching computers to listen to human experts, not just look at numbers. It turns a "best guess" into an "educated prediction."

Technical Summary

1. Problem Statement

In mechanical design and manufacturing process optimization, high-fidelity physics-based simulations are often too expensive or proprietary to run frequently. Consequently, engineers rely on surrogate models (approximations) trained on limited experimental or simulation data.

• The Core Challenge: In high-dimensional design spaces, data is often scarce (the number of data points $N$ is much smaller than the number of design variables $D$).
• Limitations of Current Methods:
  • Standard Data-Driven Surrogates (e.g., RBF, Kriging): These struggle with high-dimensional, low-data regimes, often producing overfitted or physically meaningless interpolants because they lack mechanisms to incorporate engineering intuition.
  • Physics-Informed Neural Networks (PINNs): While they can embed governing equations, they often require large datasets for stable training and struggle to incorporate qualitative expert knowledge (e.g., monotonicity, convexity) without explicit differential equations.
• The Gap: Subject Matter Experts (SMEs) possess valuable qualitative knowledge (e.g., "compliance decreases as thickness increases," or "outputs must be positive"), but existing frameworks cannot systematically integrate this knowledge with scarce data to guide the surrogate model.

2. Methodology: RBF-Gen

The authors propose RBF-Gen, a framework that combines Radial Basis Function (RBF) interpolation with generative modeling to integrate domain knowledge. The method consists of three main stages:

A. Relaxed RBF Centers (Overcomplete Basis)

Instead of tying RBF centers strictly to the $N$ training data points (which yields a unique, rigid interpolant), RBF-Gen places $K$ centers ($K > N$) uniformly or quasi-randomly across the design domain.

• This creates an underdetermined linear system ($\Phi w = y$) where $\Phi \in \mathbb{R}^{N \times K}$.
• Because $K > N$, there are infinitely many weight vectors $w$ that perfectly interpolate the training data.

B. Null Space Exploration

The solution space is decomposed into:
$$w = w_0 + N\alpha$$

• $w_0$: A particular solution (minimum norm) that fits the data.
• $N$: A basis for the null space of $\Phi$ (dimensions $K \times (K-N)$).
• $\alpha$: Free coefficients representing the "degrees of freedom" in the solution space.
• Significance: The null space allows the model to explore a family of functions that all fit the data but differ in their behavior in unobserved regions.

C. Generative Integration of Knowledge

A neural network generator $G(z; \theta)$ maps latent variables $z \sim \mathcal{N}(0, I)$ to the null-space coefficients $\alpha$. The generator is trained to select specific $\alpha$ values that satisfy domain priors provided by experts.

The training objective minimizes a composite loss function:
$$L_{gen} = \sum \lambda_i \text{pen}_i(\hat{f}_z) + \sum \gamma_j \text{KL}(p_{gen}(s_j) \parallel p_{target}(s_j))$$

• Penalty Terms (Structural Priors): Soft constraints enforcing properties like:
  • Monotonicity: Ensuring the function increases or decreases with specific variables.
  • Positivity/Bounds: Ensuring outputs stay within physical limits.
  • Smoothness/Convexity: Enforcing curvature constraints.
• KL Divergence Terms (Distributional Priors): Aligning the statistical distribution of generated function statistics (e.g., extrema, regional averages, gradients) with target distributions derived from expert intuition.

3. Key Contributions

1. Novel Framework: Introduction of RBF-Gen, the first framework to systematically integrate qualitative SME knowledge (monotonicity, bounds, convexity) into RBF surrogates via a generative null-space approach.
2. Mechanism for Knowledge Encoding: Unlike PINNs that require explicit governing equations, RBF-Gen encodes knowledge through penalty functions and distributional alignment, making it applicable even when physical equations are unknown or too complex.
3. Handling High-Dimensional Scarcity: By decoupling basis centers from data points and using the null space, the method creates a flexible function space that can be constrained by knowledge rather than just data density.
4. No Adversarial Training: Unlike InfoGAN, RBF-Gen does not require a discriminator, simplifying training stability in low-data regimes.

4. Numerical Results & Case Studies

The authors validated RBF-Gen on three case studies:

Case 1: 1D Cantilever Beam Optimization

• Setup: Minimizing compliance of a beam with $D$ elements using $N$ training samples.
• Results:
  • In the highly scarce regime ($N/D = 1$), RBF-Gen significantly outperformed standard RBF, achieving much higher design improvements (up to 60% better in some dimensions).
  • In the moderate data regime ($N/D = 2$), standard RBF performed slightly better, indicating that RBF-Gen's priors are most critical when data is extremely limited.
  • Training Time: RBF-Gen takes longer to train (seconds/minutes) but is negligible compared to the hours/days required for high-fidelity simulations.

Case 2: 2D Shell Thickness Optimization

• Setup: Optimizing a 2D cantilever plate using Reissner–Mindlin shell models.
• Results:
  • Standard RBF failed to improve upon the initial design as dimensionality increased ($D > 100$), often finding worse designs.
  • RBF-Gen consistently found improved designs across all dimensions, even with $N/D = 1$, demonstrating superior scalability in high-dimensional, data-scarce environments.

Case 3: Semiconductor Etching Process (Real-World Data)

• Setup: Modeling a High Aspect Ratio Contact (HARC) etching process with 34 real experiments and 17 input variables.
• Knowledge: Experts provided monotonicity relationships (e.g., "Variable 1 increases QoI 1") in a redacted table.
• Method: Used Leave-Two-Out (L2O) cross-validation.
• Results:
  • RBF-Gen reduced the Average Relative Error (ARE) and Average Absolute Error (AAE) for 3 out of 5 Quantities of Interest (QoIs) compared to standard RBF.
  • For the remaining QoIs, performance was comparable to the baseline.
  • This confirmed that expert monotonicity priors can significantly enhance surrogate reliability in real-world manufacturing settings.

5. Significance and Conclusion

• Practical Impact: RBF-Gen bridges the gap between data-driven AI and traditional engineering expertise. It allows engineers to leverage "gut feeling" or qualitative rules to build robust models when data is too expensive to generate.
• Robustness: The method is particularly effective in data-scarce, high-dimensional scenarios where traditional interpolation fails.
• Future Directions: The authors note limitations regarding hyperparameter sensitivity (tuning the number of centers and penalty weights) and the need to incorporate more complex priors (e.g., periodicity). Future work aims to develop adaptive tuning strategies and extend the framework to uncertainty quantification for optimal designs.

In summary, RBF-Gen represents a significant step forward in surrogate modeling by transforming the "underdetermined" nature of sparse data problems into an opportunity to inject domain knowledge, resulting in more physically meaningful and optimization-effective models.