A Streaming Sparse Cholesky Method for Derivative-Informed Gaussian Process Surrogates Within Digital Twin Applications

Technical Summary: A Streaming Sparse Cholesky Method for Derivative-Informed Gaussian Process Surrogates Within Digital Twin Applications

Problem Statement

Digital Twin (DT) frameworks aim to model the behavior of specific physical assets (Physical Twins, PT) to enable real-time forecasting and condition-based maintenance. A central challenge in DT development is balancing model fidelity with computational efficiency. While full-scale physics-based simulations offer high fidelity, they often exceed real-time constraints. Conversely, reduced-order models risk losing individualized accuracy. Furthermore, DTs must evolve alongside their physical counterparts, requiring the ingestion of in-service data to update the model.

Standard Gaussian Process (GP) surrogates are effective for uncertainty quantification but face scalability issues when enhanced with derivative information. While incorporating derivative data (e.g., velocities, accelerations, or sensitivities) significantly improves predictive accuracy and captures local sensitivities, it drastically increases the dimension of the covariance matrix. This expansion leads to prohibitive computational costs for matrix inversion and poor numerical conditioning, particularly when dealing with high-order derivatives and large datasets. Existing sparse approximation methods often struggle to accommodate these derivative-enhanced structures efficiently in a dynamic, streaming context.

Methodology

The authors propose an end-to-end DT solution that integrates three core advancements: derivative-informed GP modeling, sparse Cholesky factorization, and a dynamic streaming update algorithm.

1. Derivative-Informed Gaussian Processes:
The study extends standard GP formulations to include function values and their derivatives up to an arbitrary order $d$ (specifically up to the 4th order in this work). By utilizing a squared-exponential (SE) kernel—which ensures sufficient differentiability—the authors construct a joint covariance matrix $K_{der}$ that includes blocks for function values and all derivative orders. Theoretical lemmas establish that this derivative-informed covariance matrix remains symmetric positive-definite and that including higher-order derivatives monotonically reduces the posterior variance (uncertainty) of the GP prediction.

2. Sparse Cholesky Factorization:
To address the computational intractability of inverting the large, dense $K_{der}$ matrix, the authors employ a sparse Cholesky factorization of the precision matrix.

• Ordering: They utilize Maximum-Minimum Distance (MMD) ordering to permute training points, ensuring that points in the sequence are not too close to each other, which promotes sparsity.
• Sparsification: A sparsity set is defined based on a sparsification parameter $\rho$ and the insertion radius of points.
• Derivative Integration: The authors extend the sparse Cholesky algorithm to handle derivative observations. They introduce a "point-wise ordering algorithm" where derivative measurements are grouped with their corresponding function values in the matrix structure. This allows the sparse structure to be maintained while incorporating derivative blocks.

3. Dynamic Streaming Update Algorithms:
Recognizing that DTs require real-time adaptation, the paper introduces two dynamic update approaches (SU-Approach1 and SU-Approach2) that avoid full retraining when new data arrives:

• Fixed/Dynamic Split: The training set is partitioned into a "fixed" set (historical data) and a "dynamic" set (recent data).
• Supernode Management: The algorithm identifies "supernodes" (groups of columns with identical sparsity patterns). When new data arrives, it is added to the dynamic set. The algorithm only re-evaluates the Cholesky factors for the affected dynamic supernodes, reusing the factors for the fixed supernodes.
• Re-training Criteria: The system includes logic to detect when dynamic updates are insufficient (e.g., continuous divergence in prediction error, outlier detection, or reaching a data budget). In such cases, a full re-training is triggered with the accumulated data.

Key Contributions

1. Derivative-Enhanced Sparse Cholesky: The authors generalize sparse Cholesky factorization to accommodate derivative-informed GPs, enabling the use of high-order derivative data without the $O(N^3)$ cost of exact inversion.
2. Dynamic Update Framework: They develop a streaming algorithm that updates the sparse Cholesky factor incrementally. This allows the DT surrogate to adapt to new physical twin data in real-time with significantly reduced computational overhead compared to retraining from scratch.
3. Engineered DT System: The work presents a complete, integrated DT demonstrator for aircraft structural health monitoring, combining the mathematical advancements into a functional system that models fatigue crack growth.

Results

Numerical Verification:
Experiments on Griewank and Rosenbrock functions demonstrated that incorporating derivative information significantly reduces prediction error. For a 3D Griewank function with 27 training points, the Mean Squared Error (MSE) dropped from $10^{-2}$ (no derivatives) to $10^{-13}$ (4th-order derivatives). The sparse Cholesky approximation with appropriate sparsification parameters ($\rho$) achieved accuracy comparable to the exact GP.

Digital Twin Application (Fatigue Crack Growth):
The method was applied to predict fatigue crack growth in an aircraft structure using a semi-elliptical surface crack model.

• Accuracy: The derivative-enhanced models significantly outperformed nominal models. At the final inspection point, the relative percent difference ($\eta$) between predicted and actual crack growth rates decreased from ~1.8% (0th-order) to ~0.2% (4th-order) for the retrained exact GP, and similarly for the dynamic sparse approach.
• Efficiency: The dynamic sparse Cholesky method achieved prediction accuracies comparable to the retrained exact GP but with a computational cost of $O(Ms^2)$ (where $M$ is the dynamic set size and $s$ is the average non-zeros per column), compared to $O(N^3)$ for full retraining. In runtime tests, the dynamic sparse method completed the simulation in ~3 minutes versus ~25 minutes for the full retraining approach.
• Robustness: The system demonstrated robustness to intrinsic noise (simulated via Monte Carlo variations in material properties) and partial derivative observations. Increasing the frequency of inspections further improved prediction accuracy.

Significance and Claims

The paper claims to provide a principled and computationally efficient foundation for deploying adaptive, derivative-informed GP surrogates in digital twin applications, particularly for aerospace and mechanical systems where both accuracy and scalability are critical.

The authors emphasize that their contribution is not the invention of derivative-informed GPs or sparse Cholesky methods in isolation, but rather the engineering of a cohesive system that successfully aligns these components for the specific demands of a Digital Twin. Key practical observations highlighted include:

• The necessity of careful optimization of regularization (nugget) and length scale parameters to manage the conditioning of derivative-informed covariance matrices.
• The importance of the units and magnitude of derivative data relative to primary variables.
• The trade-off between inspection frequency and computational cost, noting that higher update frequencies yield better convergence to the physical twin's behavior.

The study concludes that the proposed streaming sparse Cholesky method effectively bridges the gap between high-fidelity derivative data and real-time DT requirements, enabling more accurate, individualized, and computationally feasible predictions for asset management.