Technical Summary: Outlier-Robust Diffusion Posterior Sampling for Bayesian Inverse Problems

1. Problem Statement

Bayesian Inverse Problems (BIPs) aim to recover an unknown variable $x$ from noisy measurements $y = F(x) + \epsilon$. While diffusion models (DMs) have emerged as powerful data-driven priors for solving BIPs, existing diffusion-based solvers rely on a presumed likelihood function $p(y|x)$ to guide the generation process. In practical scenarios, this likelihood is often misspecified due to:

1. Incorrect noise assumptions: Real-world noise may be heavy-tailed rather than Gaussian.
2. Outlier contamination: Measurements may contain sparse, large-magnitude errors (e.g., speckle noise in optical tasks or scattering artifacts).

Standard diffusion posterior sampling (DPS) methods, which typically approximate the likelihood using Tweedie's formula and assume Gaussian noise, are stable under moderate Gaussian noise but lack robustness. When faced with outlier contamination or likelihood misspecification, these solvers can drive the posterior distribution far from the target, resulting in significant recovery degradation.

2. Methodology: Robust Diffusion Posterior (RDP)

The authors propose Robust Diffusion Posterior Sampling (RDP), a framework designed to maintain stability under measurement perturbations while explicitly mitigating the effects of outliers.

Theoretical Foundation

The paper first establishes the stability of standard DPS solvers for linear BIPs under Gaussian noise, proving that the Kullback-Leibler (KL) divergence between the posterior induced by noisy measurements and the noiseless posterior is bounded quadratically by the noise magnitude. However, this bound grows unbounded under outlier contamination, revealing a vulnerability.

To address this, RDP adopts a Generalized Bayesian Inference (GBI) approach. Instead of using the standard negative log-likelihood, RDP replaces it with a robust, weighted loss function $\ell_y(x)$. The generalized posterior is defined as:
$$p^{(\ell)}(x|y) \propto p(x) \exp\{-\tau \ell_y(x)\}$$
where $\tau$ is a temperature parameter.

The Robust Guidance Mechanism

RDP transforms the standard DPS guidance term into a component-wise weighted objective. At each diffusion time step $t$, the method:

1. Estimates the clean data $\hat{x}_{0|t}$ from the noisy state $x_t$ using the pre-trained score network.
2. Computes the fitting residuals $r_i = y_i - [F(\hat{x}_{0|t})]_i$ for each measurement component.
3. Applies an adaptive weighting function $w(r_i)$ to the log-likelihood. The loss function becomes:
   $$\ell_y(x_t) = -\sum_{i=1}^{d_y} w(r_i) \log \tilde{p}(y_i | \hat{x}_{0|t})$$
4. Weighting Function: The paper utilizes the Inverse Multi-Quadratic (IMQ) function: $w(r) = (1 + r^2/c^2)^{-1/2}$.
   • For small residuals, $w(r) \approx 1$, recovering the standard Gaussian guidance.
   • For large residuals (outliers), $w(r)$ decreases, down-weighting their influence on the posterior update.
5. Adaptive Threshold: To account for the dynamic nature of residuals during the denoising process, the threshold parameter $c$ is adaptively calibrated at each step $t$ based on the $q$-th quantile of the absolute residuals.

Theoretical Guarantees

The authors prove that if the weighting function $w$ satisfies specific conditions (boundedness of $|r w(r)|$ and $|r^2 w'(r)|$), the resulting posterior is outlier-robust. Specifically, the Posterior Influence Function (PIF)—measuring the KL divergence between the posterior under noiseless and contaminated measurements—is uniformly bounded. This contrasts with standard DPS, where the PIF is unbounded under outlier contamination.

3. Key Contributions

1. Stability Analysis: The paper provides a theoretical characterization of the stability of diffusion posterior sampling for linear BIPs under measurement noise, quantifying the posterior deviation.
2. Robustness Vulnerability Identification: The analysis reveals that while standard DPS is stable under Gaussian noise, it lacks robustness against outlier contamination, leading to significant performance degradation.
3. RDP Framework: The authors introduce RDP, a plug-and-play module compatible with existing gradient-based diffusion samplers (e.g., DPS, LGD, $\Pi$GDM, PnPDM). It provides theoretical guarantees of outlier robustness for linear BIPs.
4. Empirical Validation: Extensive experiments demonstrate that RDP effectively mitigates the impact of outliers and likelihood misspecification in both linear (Inverse Scattering) and nonlinear (Phase Retrieval, Image Inpainting, Deblurring) tasks, while maintaining competitive performance in well-specified settings.

4. Experimental Results

The method was evaluated on the InverseBench benchmark across four tasks: Inverse Scattering, Image Inpainting, Deblurring, and Phase Retrieval.

• Outlier Contamination: Under sparse impulsive noise (e.g., 1%–5% of pixels corrupted with high magnitude), standard methods (DPS, LGD, etc.) suffered significant performance drops (e.g., 15%–50% degradation in PSNR/SSIM). RDP variants consistently outperformed baselines, recovering performance levels close to the well-specified setting.
• Likelihood Misspecification: Under heavy-tailed noise (Student-t distribution), standard methods degraded by 10%–30%. RDP reduced this degradation, showing gains of 5%–29% over baselines.
• Well-Specified Settings: In scenarios with standard Gaussian noise, RDP performed competitively with standard methods, demonstrating that the robustness mechanism does not compromise accuracy when the likelihood is correctly specified.
• Generality: The approach was successfully integrated into four different diffusion solvers, confirming its sampler-agnostic nature.
• Efficiency: The computational overhead of RDP is negligible, as it only adds a component-wise weighting step to the existing guidance calculation.

5. Significance and Claims

The paper claims that RDP offers a systematic solution to a critical limitation in current diffusion-based inverse problem solvers: their sensitivity to likelihood misspecification and outliers. By grounding the method in Generalized Bayesian Inference and providing theoretical bounds on posterior influence, the authors establish a rigorous foundation for robust recovery.

The significance lies in the ability to apply powerful diffusion priors in real-world scientific and imaging scenarios where noise models are imperfect or data is contaminated, without requiring retraining of the diffusion model or complex architectural changes. The method is presented as a "plug-and-play" enhancement that preserves the stability of standard solvers while adding provable robustness.

Limitations Acknowledged:
The authors note that their theoretical analysis primarily focuses on linear BIPs, and extending these guarantees to nonlinear settings remains an open direction. Additionally, the experiments primarily consider isotropic noise, whereas structured or correlated noise is also common in practice.