Technical Summary: Solver and Surrogate Choices for PDE Inference and Forecasting

Problem Statement

Forecasting physical systems governed by Partial Differential Equations (PDEs) from sparse, noisy observations requires two distinct stages: parameter inference (estimating unknown coefficients from data) and future propagation (predicting the system's evolution). While learned neural surrogates (e.g., Neural Operators) offer speed, they introduce approximation errors that propagate through both inference and forecasting. Conversely, numerical solvers are accurate but computationally expensive.

The core problem addressed is determining the optimal pipeline configuration that balances accuracy and computational cost. Specifically, the paper investigates how to combine surrogate models and numerical solvers across the inference and forecasting stages. It asks whether using a numerical solver for the final forecast (after surrogate inference) or for the inference stage itself (using only observed prefixes) yields better results than relying entirely on surrogates.

Methodology

1. Benchmark Pipelines

The study compares three distinct inference-to-forecast pipelines:

• A0 (Surrogate-Surrogate): Parameter inference and future forecasting are both performed entirely using a learned surrogate model. The forecast is a weighted average of candidate surrogate trajectories.
• A1 (Surrogate-Numerical): Parameter inference is performed using the surrogate model (evaluating candidates against observed prefixes). The final forecast is generated by running a single numerical trajectory using the inferred parameter mean ($\bar{\theta}$).
• P (Numerical-Numerical): Parameter inference is performed using a numerical solver (evaluating candidate prefixes against observations). The final forecast is a single numerical trajectory using the inferred numerical mean ($\bar{\theta}_{phys}$).

2. Experimental Setup

• PDE Families: Three one-dimensional periodic PDEs: Allen–Cahn (AC), Burgers (BU), and Advection–Diffusion (AD).
• Architectures: Fourier Neural Operators (FNO) and U-Nets.
• Data: 200 test initial conditions per model, with noisy sensor observations at varying prefix lengths (2 to 12 time steps).
• Inference Mechanism: A discrete candidate grid over the parameter space. Weights are assigned via Gaussian likelihoods based on the mismatch between observed data and model predictions.
• Evaluation Metrics:
  • Accuracy: Max-time spatial RMS error over a common future window (outputs 13–20).
  • Cost: Complete CPU query time (including candidate propagation, weighting, and final trajectory generation).
  • Uncertainty: Split-conformal prediction sets to provide calibrated trajectory radii and coverage.

3. Comparative Experiments

The paper conducts five specific experimental blocks:

1. Main Model Comparison: A0 vs. A1 across 12 configurations (3 PDEs $\times$ 2 architectures $\times$ 3 seeds).
2. Forecast Decomposition: Isolating the error reduction from "posterior averaging" (A0) to "plug-in prediction" (A0p) and finally to "numerical propagation" (A1).
3. Horizon Experiment: Varying observation prefix lengths to measure cost/accuracy trade-offs and the impact of restricting candidate rollout to the observed prefix.
4. Refined Reference: Evaluating against a high-resolution numerical reference ($N=512$) to ensure gains are not artifacts of coarse discretization.
5. Training Scale & Temperature: Assessing how increasing training data (24 vs. 96 initial conditions) and tuning likelihood temperatures affect the relative performance of the pipelines.

Key Results

1. Numerical Propagation Improves Accuracy (AC and BU)

• A1 vs. A0: For Allen–Cahn and Burgers equations, pipeline A1 (surrogate inference + numerical forecast) consistently outperforms A0 (fully surrogate) across all 12 configurations. The mean error reduction (A1 minus A0) is negative and statistically significant (95% intervals below zero).
• Advection-Diffusion (AD): Results are mixed and architecture-dependent. While some U-Net configurations show improvement, FNO configurations often show no significant gain or slight degradation, suggesting the benefit of numerical replacement is not universal across all PDE dynamics.

2. Cost Efficiency and Rollout Length

• Prefix Limitation: Restricting candidate rollout to the observed prefix (rather than full future fields) for pipeline A1 preserves forecast accuracy while significantly reducing computational cost.
  • For Burgers/FNO, query time decreased by ~27%.
  • For Burgers/U-Net, query time decreased by ~48%.
• Solver Choice: For the Advection-Diffusion equation, using direct Fourier propagation (analytical solution) for the numerical inference stage makes the P pipeline computationally cheaper than the surrogate-assisted A1 pipeline across the tested horizon, reversing the cost ranking seen with RK4 solvers.

3. Impact of Surrogate Quality

• Training Scale: Increasing FNO training data from 24 to 96 initial conditions significantly improved the surrogate's forward approximation accuracy.
• Diminishing Returns: As the surrogate became more accurate, the marginal gain from switching to a numerical forecast (A1 over A0) diminished. For example, in Allen–Cahn, the error reduction dropped from $-0.03295$ (24 ICs) to $-0.00687$ (96 ICs). This indicates that the value of numerical replacement is highest when the surrogate is less accurate.

4. Likelihood Temperature Sensitivity

• The choice of "likelihood temperature" (a calibration parameter for the candidate grid weights) significantly impacts the relative performance of P vs. A1, particularly for the Advection-Diffusion equation. Development-selected temperatures altered the short-prefix accuracy ranking, demonstrating that inference calibration is a critical factor in pipeline selection.

Significance and Claims

The paper claims to provide a reproducible, paired benchmark that quantifies the specific value of numerical computation at different stages of the PDE inference-to-forecast pipeline. Its primary contributions are:

1. Quantifying the "Hybrid" Benefit: It demonstrates that a hybrid approach (A1)—using a fast surrogate for inference but a precise numerical solver for the final forecast—is a robust strategy for improving accuracy in Allen–Cahn and Burgers systems, provided the surrogate is not already highly accurate.
2. Operational Cost Analysis: It highlights that computational savings can be achieved without accuracy loss by restricting candidate propagation to the observed prefix, a detail often overlooked in theoretical comparisons.
3. Context-Dependent Validity: The study modestly concludes that there is no single "best" pipeline. The optimal choice depends on:
   • The specific PDE family (AC/BU vs. AD).
   • The quality of the trained surrogate (which depends on training data volume).
   • The available propagator (RK4 vs. analytical Fourier).
   • The calibration of inference parameters (likelihood temperature).

The authors emphasize that their results are derived from synthetic, one-dimensional periodic systems. While the patterns (e.g., gains persisting against refined references) are encouraging, the specific magnitudes may vary in higher-dimensional or non-periodic physical systems. The work serves as a methodological framework for evaluating multi-fidelity pipelines rather than a prescriptive solution for all PDE forecasting tasks.