Technical Summary: Nonparametric Deconvolution and Denoising Using Simulation Based Inference

1. Problem Formulation

The paper addresses the dual challenge of nonparametric density deconvolution and empirical Bayes denoising in the presence of additive measurement error. The setting involves an unobserved latent random variable $X \in \mathbb{R}^d$ with an unknown density $p$, and observed noisy proxies $\tilde{X} = X + U$, where $U$ is measurement noise with a known distribution $m$.

The authors identify two coupled, ill-posed tasks:

1. Density Deconvolution: Recovering the latent distribution $p$ from the noisy sample $\tilde{X}$.
2. Denoising: Estimating individual latent realizations $X_i$ given specific noisy observations $\tilde{X}_i$.

The difficulty of these tasks depends heavily on the noise distribution. The paper distinguishes between ordinary-smooth noise (where the characteristic function decays polynomially, e.g., Laplace) and super-smooth noise (where it decays exponentially, e.g., Gaussian). Classical methods, such as Fourier inversion or kernel estimators, often struggle in high dimensions, require delicate regularization, and fail to leverage modern expressive generative models. Furthermore, likelihood-based training for generative models under noise is often intractable due to the need for high-dimensional integration.

2. Methodology: The convMMD Framework

The authors propose a likelihood-free, simulation-based framework centered on the Convolutional Maximum Mean Discrepancy (convMMD). The core insight is to avoid direct inversion of the deconvolution operator. Instead, the method matches the empirical distribution of the noisy data with the distribution obtained by convolving a candidate latent model with the known noise law.

2.1 The Objective Function

Let $q_\theta$ be a flexible generative model for the latent density. The goal is to find parameters $\theta$ that minimize the MMD between the empirical noisy distribution $\widehat{(p \ast m)}_N$ and the model-implied noisy distribution $q_\theta \ast m$:
$$\hat{\theta}_N = \arg \min_{\theta \in \Theta_J} \text{convMMD}_k^2(p, q_\theta, m) \equiv \arg \min_{\theta \in \Theta_J} \text{MMD}_k^2\left(\widehat{(p \ast m)}_N, q_\theta \ast m\right)$$
where $k$ is a bounded positive definite kernel.

2.2 Simulation-Based Optimization

The objective is likelihood-free. It requires only the ability to:

1. Sample from the latent model $Y \sim q_\theta$.
2. Simulate noise $U_Y \sim m$.
3. Form noisy samples $\tilde{Y} = Y + U_Y$.

This allows the use of stochastic gradient optimization (e.g., Adam) via Monte Carlo approximation of the expectations in the MMD objective. The framework supports sieve classes $\mathcal{Q}_J$ (e.g., Gaussian Mixture Models or Neural Generative Models like Normalizing Flows and GANs) where the complexity $J$ grows with the sample size.

2.3 Empirical Bayes Denoising

Once the latent density $\hat{q}_{\theta_N}$ is learned, it serves as an empirical prior for denoising. For a specific observation $\tilde{x}_i$, the posterior density is:
$$\hat{\pi}_{\theta_N}(x | \tilde{x}_i) \propto m(\tilde{x}_i - x) \hat{q}_{\theta_N}(x)$$
The denoised estimate is the posterior mean $\hat{x}_i = \mathbb{E}_{\hat{q}_{\theta_N}}[X | \tilde{X} = \tilde{x}_i]$. For implicit models (like GANs) where the density is intractable, the posterior mean is approximated via self-normalized importance sampling in the latent space.

3. Theoretical Contributions

The paper extends previous parametric work on convMMD to the nonparametric regime, providing rigorous finite-sample and asymptotic guarantees.

3.1 Finite-Sample Oracle Inequalities

The authors prove that the learned model performs nearly as well as the best element in the sieve class in terms of convMMD loss. Specifically, with high probability:
$$\text{convMMD}_k(p, \hat{q}_{\theta_N}, m) \leq \inf_{\theta \in \Theta_J} \text{convMMD}_k(p, q_\theta, m) + O\left(\sqrt{\frac{K}{N}}\right)$$
This establishes a parametric convergence rate ($O(N^{-1/2})$) in the latent Reproducing Kernel Hilbert Space (RKHS) geometry induced by the noise-smoothed kernel.

3.2 Nonparametric $L^2$ Convergence Rates

By imposing stronger smoothness assumptions (Sobolev smoothness of the latent density) and approximation properties of the sieve, the authors derive explicit $L^2$ convergence rates for the latent density estimator $\hat{q}_{\theta_N}$:

• Ordinary-Smooth Regime: The rate is polynomial: $O(N^{-\beta/(2\beta + 2\gamma + \nu)})$, where $\beta$ is the Sobolev smoothness, $\gamma$ is the noise decay exponent, and $\nu$ is the kernel decay exponent.
• Super-Smooth Regime: The rate is logarithmic: $O((\log N)^{-\beta/\gamma^*})$, reflecting the severe ill-posedness of recovering high-frequency information from exponentially decaying noise.

3.3 Denoising Guarantees

The paper links density estimation error to denoising performance. It proves that the excess empirical Bayes risk (the error in the posterior mean) inherits the squared $L^2$ rate of the latent density estimator. Thus, accurate deconvolution implies accurate denoising.

4. Experimental Results

The authors validate their theory and method on synthetic and image data:

• Convergence Rates: Experiments on 1D heteroscedastic deconvolution confirm the theoretical rates. The $L^2$ error follows the predicted polynomial decay for ordinary-smooth (Laplace) noise and logarithmic decay for super-smooth (Gaussian) noise. The latent MMD error consistently exhibits the parametric $N^{-1}$ rate.
• Denoising with Outliers: On 2D structured datasets (Two Moons, Circles, Checkerboard) contaminated with 3% outliers, the convMMD framework (using both GMM and Normalizing Flow sieves) outperforms baselines like NPEB (Nonparametric Empirical Bayes) and XDGMM (Extreme Deconvolution). The method recovers complex topologies and sharp boundaries better than likelihood-based EM approaches, which tend to oversmooth or fail on outliers.
• Scalability and Robustness: In high-dimensional settings (up to $D=15$ on a "Tangled Ribbon" manifold), convMMD with Normalizing Flows significantly outperforms baselines. It demonstrates robustness to noise misspecification (e.g., using a constant noise estimate instead of per-sample heteroscedastic estimates).
• High-Dimensional Image Denoising: Applied to MNIST ($D=784$), the framework remains stable under spatially correlated noise where methods relying on pixel independence (like Noise2Self) degrade. While not optimized for image-specific architectures, the results demonstrate the framework's ability to scale deconvolution to high-dimensional, correlated noise regimes.

5. Significance and Claims

The paper claims to bridge the gap between classical deconvolution theory and high-capacity generative modeling. Its significance lies in:

1. Unification: Casting nonparametric deconvolution and denoising as a single generative learning problem.
2. Likelihood-Free Efficiency: Enabling the training of expressive models (flows, GANs) directly on noisy data without intractable likelihood evaluations.
3. Theoretical Grounding: Providing the first nonparametric convergence rates for simulation-based deconvolution, recovering classical inverse-problem dependencies (polynomial vs. logarithmic rates).
4. Practical Robustness: Demonstrating that simulation-based matching is more robust to outliers and noise misspecification than traditional likelihood-maximization approaches.

The authors acknowledge limitations, noting that convergence guarantees assume global minima on non-convex objectives and specific regularity conditions (known noise, Sobolev smoothness). They suggest future work on data-driven kernel selection and extending theory to the dynamics of stochastic gradient methods in modern neural architectures.