Gaussian Process Differential Ensembles for Joint Inference on Curves, Derivatives, and Integrals

Technical Summary: Gaussian Process Differential Ensembles for Joint Inference on Curves, Derivatives, and Integrals

Problem Statement
Functional data analysis often models observations through a single latent curve (the "anchor"). However, scientific objectives frequently extend beyond this curve to include rates (derivatives), accumulated quantities (integrals), boundary values, or nonlinear functionals of these levels. Standard post-processing of a fitted smooth curve fails to provide calibrated uncertainty for these derived quantities because it ignores the posterior covariance structure required for joint inference. Furthermore, finite-difference approximations are grid-dependent and unstable for higher-order derivatives, while standard integration lacks unique solutions without explicit boundary information. Existing Gaussian process (GP) methods for derivatives or integrated processes often treat these levels separately or leave boundary uncertainty implicit, failing to propagate uncertainty coherently across the differential state.

Methodology
The authors propose anchored Gaussian process differential ensembles, a framework that embeds an anchor curve $f_0$ into a joint Gaussian state containing its mean-square derivatives ($f_{-1}, \dots, f_{-r}$) and repeated definite integrals ($f_1, \dots, f_r$).

1. Exact Construction:

   • The ensemble is defined as a $(2r+1)$-tuple of processes. Negative indices represent derivatives, and positive indices represent repeated integrals from a reference point $t_0$.
   • Crucially, the positive-index components include explicit Gaussian integration constants ($\kappa_1, \dots, \kappa_r$). These constants represent the finite-dimensional boundary values (polynomial null spaces) necessary to define general antiderivatives.
   • The construction separates the kernel-driven covariance (induced by the anchor) from the finite-dimensional covariance induced by the integration constants. This clarifies that anchor-only observations cannot identify independent integration constants; inference on integral levels requires boundary observations, virtual constraints, or prior information.
   • The framework provides a weak mean-square construction for the joint Gaussian law and, under pathwise regularity assumptions, a literal sample-path differential chain.

2. Scalable Approximation (Transformed HSGP):

   • For stationary one-dimensional kernels, the authors utilize a Transformed Hilbert Space Gaussian Process (HSGP) approximation based on Laplacian–Dirichlet eigenfunctions.
   • Instead of deriving complex closed-form expressions for derivative and integral kernels, the method applies derivative and integral operators directly to the precomputable basis functions.
   • The covariance contribution from integration constants is retained exactly, while the kernel-driven part is approximated via spectral truncation. This allows the computation of any finite-order ensemble without deriving new covariance formulas for each specific kernel family.

3. Target-Aware Calibration (TARTARE):

   • The authors introduce TARTARE (Target-Aware Range and Truncation for Approximate Representations of Ensembles) to address the issue that finite-rank bases calibrated for the anchor often under-resolve derivative targets (which emphasize high frequencies).
   • TARTARE calibrates the computational range ($L$) and truncation level ($K$) against a user-specified monitoring set $\mathcal{M}$ of ensemble levels (e.g., the second derivative or the full ensemble).
   • The procedure involves an offline phase to determine calibration constants based on spectral tail tolerances and an online phase that iteratively refines the basis size until posterior summaries for the monitored levels are stable.

Key Contributions

1. Joint Inference Framework: Definition of the anchored Gaussian differential ensemble, establishing the joint Gaussian law for derivatives, the anchor, and integrals with explicit integration constants.
2. Explicit Boundary Handling: Separation of anchor-induced covariance from finite-dimensional boundary uncertainty, demonstrating why anchor-only data cannot identify independent integration constants.
3. Computational Implementation: A transformed Laplacian eigenspectrum implementation for stationary kernels that applies operators to basis functions rather than dense matrices, accompanied by operator-level approximation bounds and finite-grid posterior convergence results.
4. Calibration Procedure: The introduction of TARTARE, a diagnostic and calibration workflow ensuring that finite-rank approximations are adequate for specific derivative or ensemble targets, preventing the under-resolution of high-frequency components.

Results

• Simulation Study: In simulations involving second-order differential ensembles with Squared Exponential and Matérn 7/2 kernels, the authors compared TARTARE-calibrated ensembles against standard anchor-level HSGP calibration and smoothing splines.
  • Anchor-level calibration resulted in significant under-resolution of derivative targets (e.g., the second derivative $f_{-2}$), leading to high Integrated Root Mean Squared Error (IRMSE) and poor coverage probabilities.
  • TARTARE calibration, particularly when targeting derivatives (T-D2) or the full ensemble (T-all), significantly improved derivative recovery and coverage, bringing them close to the exact analytic reference while preserving accuracy for the anchor and integral levels.
  • Smoothing splines provided point estimates but failed to provide calibrated posterior coverage for derivatives.
• Application (Motorcycle Crash Data): The method was applied to infer a kinematic state from noisy head-acceleration data. The model treated acceleration as the anchor, inferring jerk, snap, velocity, and position simultaneously.
  • The framework enabled coherent inference on a short-horizon turning-point functional (predicting velocity reversal and displacement within 5 ms).
  • The results demonstrated how joint inference propagates uncertainty from the noisy acceleration measurements through the differential chain to provide probabilistic statements about future kinematic states and threshold exceedances.

Significance and Claims
The paper claims that its primary contribution is the organization of differential operations around a chosen anchor with explicit propagation of covariance and boundary information, rather than the mere ability to differentiate GPs.

• The framework ensures that inferential targets (rates, integrals, turning points) remain visible from model specification to posterior reporting.
• The separation of kernel-driven uncertainty from integration constant uncertainty is highlighted as statistically critical for coherent inference.
• The authors modestly position TARTARE as a necessary computational safeguard for finite-rank implementations when derivative or ensemble summaries are required, noting that standard anchor-only calibration is insufficient for these targets.
• The work does not claim broad empirical superiority over all functional data workflows but demonstrates the value of joint state sampling for coherent uncertainty quantification in applied models.
• Limitations acknowledged include the current restriction to one-dimensional stationary kernels, the reliance on mean-square smoothness for high-order derivatives, and the fact that integration constant inference remains prior-driven without external boundary data.