Sequential Experimental Designs for Kriging Model

Technical Summary: Sequential Experimental Designs for Kriging Model

Problem Statement
Computer experiments often rely on Kriging (Gaussian process) models as surrogate models to approximate complex physical and engineering systems. The primary objective is to minimize the discrepancy between the surrogate and the true model across the entire experimental domain. While single-stage space-filling designs are common, they rely on a priori assumptions and may fail to cover critical regions, leading to wasted resources. Sequential designs offer a dynamic alternative by iteratively adding points based on accumulated data. However, existing sequential methods face two significant limitations:

1. Observation-based batch sequential designs are under-studied. In many practical scenarios (e.g., high-fidelity aerospace simulations), the fixed costs of setting up an experiment (calibration, initialization) are high, while the marginal cost of adding multiple points within a single round is negligible. Current methods rarely address this "batch" scenario effectively.
2. Inefficiency of one-point sequential designs. Traditional one-point approaches require numerous iterative rounds to accumulate sufficient data, leading to prolonged experimental cycles and inefficient resource utilization. Furthermore, naive batch selection often results in point clustering around a single optimal location, failing to explore the design space effectively.

Methodology
The paper proposes a comprehensive framework to address these gaps, comprising two novel selection criteria and a general batch sequential design algorithm.

1. Novel Selection Criteria:

   • Gradient-based Criterion ($\phi_{gra}$): Motivated by the observation that regions with large gradient magnitudes often require denser sampling to reduce fitting error, this criterion selects points that maximize the expected magnitude of the gradient of the true function, $\|\nabla f(x)\|$, while maintaining distance from existing points. It incorporates a term for local response variability ($|\hat{y}_X(x) - f(x^*)|$) to ensure points are selected where the function actually varies. Theoretically, this criterion approximates the upper bound of the prediction error.
   • Variance-based Criterion ($\phi_{var}$): This criterion aims to minimize the maximum fitting error by selecting points that maximize a function derived from the prediction variance and the variance of the gradient estimate. It is grounded in theoretical bounds showing that estimation error is regulated by both the prediction variance and the gradient variance.

2. Cluster-based Top-$b$ Sequential Design Framework:
   To extend any one-point sequential criterion to a batch setting (where $b > 1$ points are added per iteration), the authors propose a clustering algorithm (Algorithm 2).

   • Mechanism: Instead of simply selecting the top $b$ points that maximize the criterion $\phi(x)$ (which would cluster near a single global maximum), the algorithm discretizes the design space and sorts candidate points by their $\phi$ values.
   • Clustering Logic: Points are assigned to clusters based on spatial proximity and density rules (controlled by parameters $\alpha$ and $\beta$). The algorithm identifies distinct local maxima (clusters) and selects the representative point from the top $b$ clusters.
   • Goal: This approach prevents point clustering, ensures diverse exploration of high-value regions, and allows any existing one-point criterion to be adapted for batch scenarios.

Key Contributions

• Two New Criteria: The introduction of the gradient-based and variance-based criteria, which offer distinct theoretical perspectives on minimizing global fitting error (average vs. maximum error).
• General Batch Framework: A novel "cluster-based top-$b$" framework that solves the inherent point clustering problem in naive batch selection, enabling the efficient extension of any sequential criterion to batch scenarios.
• Theoretical Justification: The paper provides lemmas and theorems establishing that the proposed criteria relate to the upper bounds of prediction errors and gradient variances, offering a theoretical basis for their selection strategies.

Results
The authors evaluated the proposed methods against existing criteria (including Expected Improvement variants, variance minimization, and mixture discrepancy) using seven test functions (Branin, Hartman, Ackley, Zakharov, Rosenbrock, etc.) with varying dimensions.

• One-Point Performance: In terms of Root Mean Square Error (RMSE), the gradient-based criterion generally outperformed existing methods. For Maximum Absolute Error (MAE), the variance-based criterion demonstrated superior performance.
• Batch Performance: The cluster-based top-$b$ designs generated using the new criteria consistently outperformed batch designs based on existing criteria and the benchmark sequential uniform design.
• Efficiency: The batch designs achieved performance comparable to, and in some cases better than, their one-point counterparts while significantly reducing the number of experimental rounds (iterations).

Significance and Claims
The paper claims that the proposed methods address critical limitations in current sequential design literature, specifically the lack of effective observation-based batch designs and the inefficiency of one-point approaches. By demonstrating that the gradient-based criterion minimizes average error and the variance-based criterion minimizes maximum error, the authors provide practitioners with targeted tools for specific optimization goals. Furthermore, the proposed batch framework is presented as a versatile solution that enhances resource efficiency by reducing experimental rounds without sacrificing fitting accuracy, making it particularly suitable for resource-intensive computer experiments. The authors conclude that their proposed criteria and framework generally outperform existing approaches in terms of fitting accuracy.