🤖 machine learning

Debiased neural operators for estimating functionals

This paper introduces DOPE, a semiparametric estimator that corrects the first-order bias of naive plug-in methods for estimating scalar functionals of solution trajectories by employing a Neyman-orthogonal, one-step approach with Riesz regression to handle neural operators as high-dimensional nuisance mappings under partial and irregular observations.

Original authors: Konstantin Hess, Dennis Frauen, Niki Kilbertus, Stefan Feuerriegel

Published 2026-04-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Konstantin Hess, Dennis Frauen, Niki Kilbertus, Stefan Feuerriegel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a chef trying to perfect a new soup recipe. You have a super-smart AI assistant (a Neural Operator) that can predict exactly how the soup will taste at every single second of the cooking process. It gives you a perfect, minute-by-minute flavor profile.

However, you don't actually care about the flavor at every second. You only care about one specific number to decide if the soup is ready:

The "Total Flavor Score" (How much flavor was in the pot over the whole hour?)
The "Burn Time" (How many minutes did the soup stay too hot?)
The "Peak Heat" (What was the hottest temperature it reached?)

These specific numbers are called functionals. They are summaries of the whole story.

The Problem: The "Naïve" Mistake

Usually, scientists do this the "naïve" way:

They ask the AI to predict the whole soup flavor curve.
They take that prediction and calculate the "Total Flavor Score" from it.

Here is the catch: Even if the AI is 99% accurate, that tiny 1% error gets magnified when you try to calculate the summary score. It's like trying to measure the total weight of a pile of sand by weighing a slightly inaccurate bucket. If your bucket is slightly off, the total weight calculation is wrong. In math terms, this is called Plug-in Bias. The error doesn't cancel out; it accumulates, giving you a systematically wrong answer.

The Solution: DOPE (Debiased Neural Operator)

The authors of this paper invented a new tool called DOPE. Think of DOPE not just as a calculator, but as a smart detective that knows how to fix the AI's mistakes.

Here is how DOPE works, using a simple analogy:

1. The "Shadow" of the Mistake

Imagine the AI makes a mistake in its prediction. DOPE doesn't just ignore that mistake. It asks: "If the AI was wrong here, how much would that specific error change my final 'Total Flavor Score'?"

Sometimes, a small error in the soup's temperature at minute 5 matters a lot for the final score. Other times, an error at minute 50 doesn't matter at all. DOPE calculates a "Sensitivity Map" (mathematically called a Riesz Representer). This map tells the system: "Pay extra attention to errors in these specific spots; ignore errors in those other spots."

2. The "Weighted Correction"

In the real world, we often can't taste the soup at every second. We only get a few random spoonfuls (irregular observations).

The Naïve approach treats every spoonful equally.
DOPE is smarter. It realizes that if you only tasted the soup when it was boiling, you missed the cooling phase. So, DOPE gives extra weight to the data points that are rare or hard to get, and less weight to the ones that are easy to get.

It combines the AI's prediction with these weighted "correction factors" to cancel out the bias. It's like having a GPS that knows the map is slightly distorted, so it automatically adjusts your route to ensure you still arrive at the right destination.

3. The "Magic Trick" (Automatic De-biasing)

The hardest part of this is figuring out that "Sensitivity Map" without doing years of complex math by hand. The authors developed a way to teach the computer to learn this map automatically using a technique called Riesz Regression.

Think of it like this: Instead of the chef trying to write a manual on how to fix the soup, they just say to the AI, "Here is the recipe, here is the goal. You figure out how to adjust your own predictions to hit the target perfectly." The AI learns to correct its own bias on the fly.

Why This Matters in the Real World

This isn't just about soup. This paper solves a problem in many critical fields:

Medicine: Doctors don't need to know a patient's blood sugar every second. They need to know the "Time in Range" (how long the sugar stayed at a healthy level). If a model is slightly off, the naïve method might say the patient is safe when they are actually in danger. DOPE fixes this, making medical decisions safer.
Climate Science: Scientists want to know the "Total Energy" of a storm. If the model is slightly off, the total energy calculation could be wildly wrong, leading to bad predictions for hurricane damage. DOPE ensures the total energy number is accurate.
Engineering: Engineers need to know the "Total Wear and Tear" on a bridge over 10 years. DOPE helps them get that number right even if the simulation of the bridge isn't perfect.

The Bottom Line

DOPE is a new method that takes a powerful but slightly imperfect AI prediction and "de-biases" it. It uses a clever weighting system to ensure that when you summarize a complex, messy process into a single, important number (like total cost, total time, or total energy), that number is honest and accurate, even if the underlying AI model isn't perfect.

It turns a "good enough" prediction into a "trustworthy" decision.

1. Problem Formulation

Context: Neural operators (e.g., Fourier Neural Operators, DeepONet) are powerful tools for learning solution maps of Partial Differential Equations (PDEs), mapping input functions (boundary conditions, coefficients) to solution trajectories.
The Gap: In many scientific applications (medicine, climate, engineering), the goal is not to recover the full solution trajectory but to estimate a scalar functional $g$ of that trajectory (e.g., Area Under the Curve (AUC), time spent above a threshold, total energy dissipation).
The Challenge:

Naïve Plug-in Bias: The standard approach is to train a neural operator $\hat{S}$ to approximate the true solution operator $S_0$ , and then compute the functional $g(\hat{S}(A))$ . The authors prove that this "plug-in" estimator suffers from first-order bias. Because $g$ is typically non-linear, approximation errors in the neural operator do not average out; instead, they propagate linearly through the Fréchet derivative of $g$ , leading to systematic errors that prevent valid statistical inference (e.g., confidence intervals).
Partial/Irregular Observations: In real-world scenarios, solution trajectories are rarely observed fully. Instead, they are observed at sparse, irregular, and often input-dependent locations (e.g., adaptive sampling in experiments). This complicates the estimation of the underlying function and the correction of bias.

2. Methodology: DOPE (Debiased Neural Operator)

The authors propose DOPE, a semiparametric framework to estimate the population average of a functional $\theta = \mathbb{E}[g(S_0(A))]$ while correcting for the bias introduced by the neural operator approximation.

Core Concept: Neyman Orthogonality

DOPE constructs a one-step, Neyman-orthogonal estimator. This ensures that the estimator is first-order insensitive to errors in the "nuisance" components (the neural operator $\hat{S}$ and the correction weights $\hat{\beta}$ ). Consequently, the bias in the final estimate depends only on the second-order errors of these components, allowing for valid inference even if the neural operator is not perfectly accurate.

Key Technical Components:

Debiasing Score Construction:
The authors derive a score function $\psi$ that corrects the plug-in estimate. The correction term relies on the Riesz representer of the functional derivative.
- Let $\Delta = S_0 - \hat{S}$ be the operator error.
- The bias is approximately $\mathbb{E}[Dg_{S_0(A)}(\Delta(A))]$ .
- Using the Riesz representation theorem, the functional derivative $Dg$ can be expressed as an inner product with a unique function $w_g$ .
- To handle irregular sampling, they introduce a design weight $\xi_0$ (inverse of the sampling density).
- The debiasing weight $\beta_0$ is defined as $\beta_0(a)(x) = \xi_0(a)(x) w_g(S_0(a))(x)$ .
Automatic Debiased Machine Learning (DML) for Operators:
A major innovation is extending automatic DML (which typically handles function-valued nuisances) to operator-valued nuisances.
- Instead of analytically deriving $\beta_0$ or inverting the sampling density (which is unstable), DOPE learns $\beta_0$ directly via Riesz regression.
- The method minimizes a primal variational objective:
  $\min_{\beta} \mathbb{E} \left[ \mathbb{E}[\beta(A)(X)^2 | A] - 2 Dg_{S_0(A)}(\beta(A)) \right]$
- This is implemented using automatic differentiation (Jacobian-vector products) to compute the functional derivative term without explicit analytical derivation.
Estimation Procedure:
- Cross-fitting: The data is split into folds. A neural operator $\hat{S}$ is trained on one fold and used to generate predictions on another.
- Riesz Regression: A second neural operator $\hat{\beta}$ is trained to approximate the debiasing weight using the Riesz regression objective.
- One-Step Estimator: The final estimate $\hat{\theta}$ is the sample average of pseudo-outcomes:
  $\tilde{\psi}_i = g(\hat{S}(A_i)) + \frac{1}{K_i} \sum_{k=1}^{K_i} \hat{\beta}(A_i)(X_{ik}) (Y_{ik} - \hat{S}(A_i)(X_{ik}))$
  The second term acts as a residual correction that cancels the first-order bias.
Prediction-Powered Inference (PPI):
The framework naturally integrates with PPI, allowing the use of large amounts of unlabeled input data to improve efficiency by leveraging the plug-in component while maintaining bias correction via the labeled subset.

3. Key Contributions

Theoretical Identification of Plug-in Bias: The authors formally prove that naïve plug-in estimation for functionals of neural operators induces first-order bias, which is non-negligible for standard convergence rates.
DOPE Framework: They introduce a novel one-step, Neyman-orthogonal estimator that removes this bias. It is model-agnostic (works with any neural operator architecture) and handles partial/irregular observations.
Extension of Automatic DML: They extend automatic debiasing from function-valued to operator-valued nuisance functions. This is achieved via Riesz regression and automatic differentiation, avoiding the need for closed-form derivations of complex functional derivatives or unstable density inversions.
Theoretical Guarantees: They establish asymptotic normality and root- $n$ convergence rates for DOPE under standard regularity conditions, provided the nuisance estimators converge at a rate of $o(n^{-1/4})$ (significantly weaker than the $o(n^{-1/2})$ required for plug-in estimators).

4. Experimental Results

The authors evaluated DOPE on two datasets:

Pharmacokinetics (PK): Modeling drug concentration curves to estimate AUC and Time-Above-Threshold (TAT).
Darcy Flow: A 2D fluid dynamics benchmark to estimate smooth excess-above-threshold functional.

Key Findings:

Superior Accuracy: DOPE significantly outperforms the naïve plug-in estimator across all settings, reducing RMSE by 10% to 43% depending on the functional and sampling irregularity.
Robustness: DOPE is robust to errors in the neural operator ( $\hat{S}$ ) and the correction weight ( $\hat{\beta}$ ). Even when nuisance estimation errors are artificially increased, the RMSE of DOPE grows much slower than that of the plug-in estimator.
Coverage: Confidence intervals constructed using DOPE achieve near-nominal coverage (approx. 95%), whereas plug-in estimators fail to provide valid coverage (often <50%).
PPI Benefits: DOPE effectively leverages additional unlabeled data to further reduce variance, outperforming plug-in methods in this regime as well.
Architecture Agnostic: Results hold for both Fourier Neural Operators (FNO) and DeepONet backbones.

5. Significance

This work bridges a critical gap between scientific machine learning (learning PDE solutions) and statistical inference (estimating quantities of interest with uncertainty quantification).

Practical Impact: It enables scientists to trust scalar summaries derived from neural operators, which is essential for decision-making in fields like clinical dosing (AUC), climate risk assessment (threshold exceedance), and engineering control.
Methodological Advancement: By solving the problem of operator-valued nuisances, it opens the door for rigorous statistical inference in complex, high-dimensional function spaces where traditional plug-in methods fail. It demonstrates that "debiased machine learning" principles can be successfully adapted to the domain of neural operators.