📊 statistics

Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers

This paper addresses the limitation of existing Diffusion Inverse Solvers benchmarks that focus solely on reconstruction accuracy by introducing a systematic study of posterior fidelity and proposing a ground-truth-free metric, score-based Kernel Stein Discrepancy (score-KSD), to evaluate how well generated samples capture the target posterior distribution in both controlled simulations and real-world inverse problems.

Original authors: Xiaoyu Qiu, Taewon Yang, Zhanhao Liu, Guanyang Wang, Liyue Shen

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiaoyu Qiu, Taewon Yang, Zhanhao Liu, Guanyang Wang, Liyue Shen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Accuracy Trap"

Imagine you are trying to solve a mystery, but the clues you have are blurry and incomplete. In science and engineering, this is called an inverse problem. You see the result (like a blurry photo or a noisy signal) and you have to guess what caused it.

Recently, scientists started using a new type of AI called a Diffusion Inverse Solver (DIS). Think of these solvers as a team of detectives. When they look at the blurry clues, they don't just guess one single answer; they generate a whole cloud of possible answers to show how uncertain they are.

The Trap:
Until now, we've only been judging these detectives by how close their single best guess is to the real answer. We use a score called "Accuracy" (like PSNR).

The Paper's Claim: This is a trap. A detective might get a high accuracy score just by getting lucky and guessing a point very close to the truth, even if their entire cloud of guesses is wrong. They might miss the real solution entirely or only guess one tiny possibility, ignoring other valid ones.
The Analogy: Imagine a weather forecaster.
- Forecaster A predicts "It will rain exactly at 2:00 PM." It rains at 2:00 PM. High Accuracy. But they missed that it might also rain at 3:00 PM or 4:00 PM.
- Forecaster B predicts "It will rain anytime between 2:00 PM and 5:00 PM." It rains at 2:00 PM. Lower Accuracy (because their single point guess was less precise), but their Uncertainty was perfect. They captured the whole truth.
- The paper argues we need to stop only praising Forecaster A and start checking if Forecaster B's "cloud of guesses" actually matches the real weather patterns.

The Solution: A New "Truth Detector" (Score-KSD)

The problem is: How do you check if a detective's cloud of guesses is right if you don't know the real answer? In real-world science (like medical scans), we often don't have the "ground truth" to compare against.

The authors created a new tool called Score-KSD.

How it works (The Metaphor):
Imagine the "Truth" is a hidden landscape with hills and valleys. The "Score" is like a compass that always points uphill toward the most likely places.

The Compass: The paper shows that even if we don't know the exact map (the true answer), we can build a compass using the physics of the problem and the AI's training. This compass points toward the "right" areas.
The Test: We take the cloud of guesses generated by the AI and check: "Do these guesses follow the compass?"
- If the guesses are scattered randomly or stuck in the wrong valley, the compass will spin wildly. The Score-KSD number will be high (bad).
- If the guesses cluster perfectly where the compass points, the Score-KSD number will be low (good).

The Key Innovation: This tool doesn't need to see the "Ground Truth" (the real answer). It only needs to check if the AI's guesses are consistent with the rules of the universe (the physics) and the AI's own training.

What They Found

The authors tested this new tool on many different problems, from fixing blurry photos to reconstructing MRI scans and CT scans.

Accuracy $\neq$ Truth: They found that the AI methods with the "sharpest" single images (highest accuracy) often had terrible "clouds of guesses." They were confident but wrong about the uncertainty.
The "Trap" Confirmed: Some methods looked great on standard charts but failed the Score-KSD test. They were producing "off-posterior" samples—guesses that looked good but were statistically impossible given the physics of the problem.
A New Standard: The Score-KSD metric successfully identified which AI methods were actually capturing the full range of possibilities (the true uncertainty) and which ones were collapsing into a single, potentially misleading guess.

Summary

Old Way: "Look how close this single guess is to the real thing!" (Misses the bigger picture of uncertainty).
New Way: "Does this whole group of guesses follow the rules of physics and probability, even if we don't know the real answer?"
Result: The paper proves that being "accurate" doesn't mean you understand the uncertainty. Their new tool, Score-KSD, is a way to check if an AI is being honest about what it knows and doesn't know, without needing a cheat sheet (ground truth) to compare against.

Technical Summary: Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers

Problem Statement

Inverse problems in scientific and engineering domains (e.g., medical imaging, geophysics, astronomy) are inherently ill-posed, often yielding multiple physically plausible solutions due to measurement noise and incomplete data. While Diffusion Inverse Solvers (DIS) have emerged as a powerful paradigm for solving these problems by leveraging pretrained diffusion priors, current evaluation benchmarks suffer from a critical limitation: they focus almost exclusively on reconstruction accuracy (e.g., PSNR, SSIM).

This focus creates an "Accuracy Trap," where methods are judged solely on point-wise reconstruction quality. However, stochastic DIS methods are designed to sample from a posterior distribution $p(x|y)$ to quantify uncertainty. A method may achieve high accuracy by producing a single deterministic-like output that happens to be close to the ground truth, yet fail to capture the true distributional behavior (e.g., missing modes, collapsing variance, or generating off-posterior samples). Existing metrics for distributional fidelity, such as Wasserstein distance or FID, require access to ground-truth posterior samples or densities, which are unavailable in real-world scientific applications. Consequently, there is a lack of robust, ground-truth-free metrics to evaluate whether DIS methods genuinely recover the target posterior distribution.

Methodology

1. Systematic Study of Posterior Fidelity

The authors conduct a controlled study using synthetic inverse problems with known analytical true posteriors. By visualizing posterior samples and comparing them against ground-truth distributions, they demonstrate that different DIS methods exhibit qualitatively distinct behaviors (e.g., mode collapse, poor coverage of weak modes) even when their reconstruction accuracy is comparable. This establishes the necessity of evaluating distributional behavior beyond accuracy.

2. Score-Based Kernel Stein Discrepancy (Score-KSD)

To address the lack of ground-truth samples in real-world settings, the paper proposes Score-KSD, a theoretically grounded metric that evaluates posterior consistency without requiring access to the true posterior density or samples.

Posterior Score Approximation: The method relies on the decomposition of the posterior score $\nabla_x \log p(x|y)$ via Bayes' rule:
$\nabla_x \log p(x|y) = \nabla_x \log p(y|x) + \nabla_x \log p(x)$
- Likelihood Score: For Gaussian noise, the likelihood score $\nabla_x \log p(y|x)$ is analytically computable using the forward operator $A$ and the measurement $y$ .
- Prior Score: The prior score $\nabla_x \log p(x)$ is approximated using a pretrained diffusion model. By perturbing a clean image $x$ with small noise and querying the diffusion score network $s_\theta$ , the authors approximate the prior score.
- Combined Score: The sum of the likelihood score and the approximated prior score yields an estimated posterior score field $\hat{s}_p(x; y)$ .
Kernel Stein Discrepancy (KSD): Using the estimated score field, the authors apply KSD to measure the discrepancy between the distribution of samples generated by a DIS method ( $\hat{q}_N$ ) and the target posterior $p(x|y)$ . KSD measures the maximum violation of the Stein identity $\mathbb{E}[T_p f(X)] = 0$ over a Reproducing Kernel Hilbert Space (RKHS).
- The squared KSD admits a closed-form empirical estimator based on the generated samples and the kernel function.
- A normalized version, score-KSD, is proposed to account for data dimensionality, allowing for intra-task comparison. A lower score-KSD indicates higher consistency with the target posterior score field.

Key Contributions

Identification of the "Accuracy Trap": The paper systematically demonstrates that high reconstruction accuracy does not guarantee high posterior fidelity. Different DIS methods can produce similar PSNR scores while exhibiting vastly different uncertainty behaviors (e.g., deterministic collapse vs. well-dispersed sampling).
Proposal of Score-KSD: The authors introduce a ground-truth-free metric for evaluating posterior consistency. Unlike previous distributional metrics, Score-KSD only requires the forward model, the noise level, and a pretrained diffusion prior, making it applicable to real-world inverse problems where the true posterior is unknown.
Comprehensive Evaluation: The study validates Score-KSD through:
- Controlled Simulations: Comparing Score-KSD rankings against visual posterior fidelity and ground-truth analytical scores.
- Real-World Tasks: Applying the metric to linear inverse scattering, under-sampled MRI reconstruction, and sparse-view CT reconstruction (both in-distribution and out-of-distribution).

Results

Simulation Studies: In synthetic settings with known posteriors, Score-KSD successfully distinguishes between methods that recover the full posterior geometry (including weak modes) and those that suffer from mode collapse. Methods with high accuracy but poor distributional coverage (e.g., RED-Diff, FPS in single-run settings) exhibited significantly higher Score-KSD values than methods with better posterior consistency (e.g., MCG-Diff, DiffPIR).
Real-World Experiments:
- Score-KSD effectively discriminates between DIS methods and trivial baselines (unconditional priors, pure noise).
- The metric reveals that methods with similar PSNR scores often have divergent Score-KSD values, confirming that accuracy alone is insufficient for evaluating stochastic solvers.
- In Out-of-Distribution (OOD) settings (e.g., reconstructing lung CTs using a prior trained on LIDC-IDRI), Score-KSD values increased, correctly indicating degraded posterior consistency due to prior mismatch.
- The metric showed stability across different test images within the same task, supporting its use as a robust diagnostic tool.

Significance and Claims

The paper argues that robust uncertainty quantification is a prerequisite for deploying DIS methods in risk-sensitive scientific applications. The primary significance of this work lies in shifting the evaluation paradigm from point-wise accuracy to distributional fidelity.

The authors claim that Score-KSD provides a meaningful, theoretically grounded diagnostic that:

Reveals Hidden Behaviors: It uncovers distributional failures (like mode collapse) that accuracy metrics miss.
Enables Ground-Truth-Free Evaluation: It allows for the assessment of posterior consistency in real-world scenarios where analytical posteriors are unavailable.
Highlights the "Accuracy Trap": It demonstrates that optimizing for PSNR does not necessarily lead to better uncertainty quantification, urging the community to adopt distribution-aware evaluation standards.

The authors modestly note that Score-KSD is a within-task diagnostic (values are not directly comparable across different inverse problems due to varying posterior sharpness and dimensions) and that its accuracy depends on the correct estimation of the noise scale $\sigma_y$ . They identify the sensitivity of this noise estimation as a direction for future work.