🤖 machine learning

Uncertainty Quantification of Engineering Structures by Polynomial Chaos Expansion and Multivariate Active Learning

This paper proposes an adaptive sequential sampling method for constructing polynomial chaos expansion surrogate models that efficiently handles multiple quantities of interest by balancing input space exploration with aggregated variance exploitation, thereby improving surrogate accuracy and stability compared to non-sequential approaches.

Original authors: Qitian Lu, Jafar Jafari-Asl, Panagiotis Spyridis, Lukas Novak

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qitian Lu, Jafar Jafari-Asl, Panagiotis Spyridis, Lukas Novak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Core Problem: The "Too Expensive to Test" Dilemma

Imagine you are an engineer designing a massive offshore wind turbine. You want to know how it will behave under different conditions: strong winds, big waves, and varying water depths. To find out, you use a super-computer simulation.

However, running this simulation is incredibly slow and expensive. It might take hours or even days to get the answer for just one specific set of conditions. If you want to test 10,000 different scenarios to be safe, it would take years. You simply don’t have that much time or money.

So, engineers use a shortcut called a "Surrogate Model" (or a "digital twin"). Instead of running the full, heavy simulation every time, you run it a few times to gather data, and then you build a lightweight, fast mathematical formula that predicts what the heavy simulation would say. It’s like learning the taste of a soup by tasting it three times, rather than drinking the whole pot every time you want to check the flavor.

The Specific Challenge: Multiple Outputs at Once

In this paper, the authors tackle a specific headache: Multi-Output Problems.

Most simulations don’t just give you one number (like "total stress"). They give you a list of numbers simultaneously. For a wind turbine, you might get:

How much the tower bends sideways.
How much it twists.
The stress on the foundation.
The vibration levels.

The problem is that these different outputs behave differently. One output might be very sensitive to wind speed, while another is more sensitive to wave height. If you pick your test points randomly to get a good prediction for the bending, you might end up with terrible predictions for the twisting.

The old way to fix this was to do separate tests for each output. But that doubles or triples your work, which is too expensive. The authors wanted a way to pick test points that are good for all the outputs at the same time.

The Solution: The "Smart Scout" Strategy (Active Learning)

The authors propose a method they call "Multivariate Active Learning" using a tool called Polynomial Chaos Expansion (PCE). Think of PCE as the specific type of lightweight formula they are building.

Their strategy works like a smart scout exploring a dark, complex forest (the input space of all possible wind/wave conditions):

Start Small: You start with a few random test points.
Look for "Variance" (The Uncertainty): The model looks at its current predictions and asks, "Where am I most unsure?" In math terms, this is called "variance." High variance means the model is guessing wildly in that area.
Look for "Distance" (The Exploration): The model also asks, "Where are there empty gaps in my map?" It doesn’t want to cluster all its tests in one spot; it wants to cover the whole forest.
The "Theta" ( $\Theta$ ) Criterion: This is the secret sauce. It’s a formula that balances the two questions above. It calculates a score for every potential new test point.
- It rewards points that are in areas of high uncertainty (high variance).
- It rewards points that are far away from where you’ve already tested (distance).
- Crucially, it does this for all outputs combined. It aggregates the uncertainty from the bending, twisting, and stress into one single score.

The Analogy: Painting a Mural

Imagine you are painting a huge, detailed mural, but you only have a limited number of paintbrush strokes.

The Old Way (Random/LHS): You sprinkle paint dots randomly across the canvas. Some areas get too many dots, others get none. The picture looks messy and inaccurate.
The Separate Way: You paint the sky carefully, then erase it and paint the ground carefully. This wastes time and makes the colors clash.
The Authors' Way (Active Learning): You look at the canvas. You see that the sky is blurry (high uncertainty) and the corner is empty (high distance). You place your next brushstroke there. Then you look again. You see the ground is now blurry. You place the next stroke there.

By always placing your next stroke where it reduces the most "blur" across the entire painting, you get a clear, accurate image with far fewer strokes than the random method.

What Did They Prove?

The authors tested this "Smart Scout" strategy on three examples:

A mathematical function with tricky "singularities" (sharp spikes that are hard to predict).
A reinforced concrete bridge beam (checking for bending and shear forces).
An offshore wind turbine (checking for moments and displacements under wind and waves).

The Results:

Better Accuracy: Their method got closer to the true answer faster than the standard random method (Latin Hypercube Sampling).
More Stable: If you ran the test 100 times, their method gave very consistent results. The random method sometimes got lucky and sometimes got unlucky; their method was reliably good.
Better at Catching Extremes: In engineering, you care most about the worst-case scenarios (like the strongest storm). Their method was better at finding these extreme points because it focused on areas of high variance.

Summary

The paper introduces a smarter way to pick data points for computer simulations that produce multiple results. Instead of guessing where to test, the algorithm actively chooses the next test point based on where it is most uncertain and where it hasn't looked yet, balancing the needs of all the different outputs. This saves time and money while making the predictions more accurate and reliable.

Technical Summary: Uncertainty Quantification of Engineering Structures by Polynomial Chaos Expansion and Multivariate Active Learning

Problem Statement
In engineering applications, high-fidelity models (e.g., finite element simulations) often produce multiple quantities of interest (QoIs) simultaneously under the same input parameters. Uncertainty Quantification (UQ) for these multi-output systems is computationally expensive, necessitating the use of surrogate models such as Polynomial Chaos Expansion (PCE). The accuracy of non-intrusive PCE surrogates constructed via regression is heavily dependent on the quality of the Experimental Design (ED), or training data.

Standard static sampling methods (e.g., Monte Carlo, Latin Hypercube Sampling) select all points a priori and cannot exploit information from previously evaluated samples. While adaptive sequential sampling (active learning) exists for single-output PCE, extending it to multi-output scenarios is non-trivial. Separate sampling for each output increases computational cost and ignores correlations between outputs, potentially compromising data consistency. Furthermore, different outputs may exhibit varying sensitivities and complexities, making a single static design inadequate for all outputs simultaneously.

Methodology
The authors propose a generalized adaptive sequential sampling strategy for multi-output PCE surrogates, extending the single-output $\Theta$ criterion previously developed by Novák et al. The method operates within a candidate pool framework:

Initialization: A small initial ED is used to construct an initial PCE surrogate.
Candidate Pool: A large set of candidate points is generated (e.g., via LHS) to serve as a reservoir for potential new samples.
Multi-Output $\Theta$ Criterion: New samples are selected sequentially from the candidate pool by maximizing a criterion $\Theta(\xi^{(c)})$ $Θ (ξ^{(c)})$ that balances exploitation (variance contribution) and exploration (spatial distribution). The criterion is defined as:
$\Theta(\xi^{(c)}) = \sqrt{\left( \sum_{r=1}^{N_{out}} \frac{\sigma_r^2(\xi^{(c)})}{\max_{\xi_i^{(c)} \in S_c} \sigma_r^2(\xi_i^{(c)})} \right) \left( \sum_{r=1}^{N_{out}} \frac{\sigma_r^2(\xi^{(s)})}{\max_{\xi_i^{(s)} \in S_s} \sigma_r^2(\xi_i^{(s)})} \right)} \times l_{c,s}^M$
Where:
- $\xi^{(c)}$ is a candidate point and $\xi^{(s)}$ is its nearest neighbor in the current ED.
- $\sigma_r^2(\xi)$ is the local variance contribution of the $r$ -th output at point $\xi$ , derived from the PCE coefficients.
- The first term is the geometric mean of the normalized variance aggregations across all $N_{out}$ outputs for the candidate and its nearest neighbor. This normalization ensures that outputs with different variance magnitudes contribute comparably to the criterion.
- $l_{c,s}^M$ is the Euclidean distance between the candidate and its nearest neighbor, promoting space-filling (exploration).

The geometric mean is used instead of the arithmetic mean to penalize candidates located in regions with negligible variance density (e.g., far tails of Gaussian distributions), preventing the selection of points that are spatially distant but statistically insignificant.

Key Contributions

Generalization of Active Learning: The paper generalizes the variance-based $\Theta$ criterion from single-output to multi-output PCE surrogate construction.
Normalized Variance Aggregation: A novel normalization scheme is introduced to aggregate variance contributions from multiple outputs, ensuring that the sampling strategy is not dominated by outputs with inherently higher variance magnitudes.
Balanced Criterion: The proposed criterion explicitly balances the exploitation of high-variance regions (where the surrogate is uncertain) with the exploration of the input space (via distance metrics), tailored for vector-valued QoIs.

Results
The proposed method is validated against Latin Hypercube Sampling (LHS) using three numerical examples:

2D Mirror Line Singularities: A synthetic function with localized singularities. The $\Theta$ criterion rapidly reduced Mean Absolute Error (MAE) and relative variance error compared to LHS, with significantly lower variability across repeated runs. Samples clustered effectively around the singular curves.
Reinforced Concrete Beam: A three-span continuous beam with 15 outputs (bending moments and shear forces) driven by 8 random variables. The $\Theta$ criterion demonstrated superior performance in reducing maximum absolute error ( $AE_{max}$ ) and improving surrogate stability (lower Leave-One-Out error, $1-Q^2$ ). It provided more robust second-order statistics estimation, particularly for outputs with complex response surfaces.
Offshore Wind Turbine (OWT): Two cases were analyzed: (a) wind-induced moments (2 outputs) and (b) multiphysics coupled displacements (2 outputs, 5 inputs). In both cases, the $\Theta$ criterion achieved lower $AE_{max}$ and faster convergence of variance estimates compared to LHS. In the multiphysics case, despite a finite candidate pool limiting asymptotic performance, the $\Theta$ criterion showed faster improvement in early and intermediate stages.

Significance and Claims
The paper claims that the proposed multi-output $\Theta$ criterion provides a more efficient and robust sampling strategy for multi-output PCE surrogates under limited computational budgets. Key benefits include:

Improved Accuracy and Stability: The method reduces prediction errors (especially worst-case errors) and enhances the stability of the surrogate model across repeated realizations.
Efficient Variance Estimation: It provides more reliable estimates of second-order statistics (variance) with fewer samples.
Unified Sampling: It avoids the inconsistency and computational overhead of separate sampling for each output by utilizing a unified criterion that aggregates multi-output information.

The authors acknowledge limitations, including the reliance on a predefined candidate pool (which may be costly to generate in high dimensions) and the computational overhead of frequent model updates in high-dimensional sparse regression. They suggest future work could involve incorporating output correlations via techniques like Principal Component Analysis (PCA) or combining the method with multi-element PCEs.