Technical Summary: Conditional Diffusion Under Linear Constraints

Problem Statement

The paper addresses the problem of zero-shot conditional sampling for linear inverse problems (e.g., image inpainting, super-resolution) using pretrained diffusion models. In these scenarios, an observation $y$ is related to a clean signal $Z$ via a linear operator $A$ such that $y = AZ$. The goal is to sample from the conditional distribution $\text{Law}(Z \mid AZ = y)$.

Existing projection-based methods (e.g., DDRM, DDNM) enforce measurement consistency by analytically correcting the "normal" component of the signal (the part determined by $A$) during the reverse diffusion process. However, the authors argue that measurement consistency alone is insufficient. While the normal component is fixed by the observation, the "tangent" component (the part in the null space of $A$) remains underdetermined. Standard projection-based methods leave the tangent dynamics governed entirely by the pretrained unconditional score. This leads to a tangent-space bias, where the probability mass is distributed incorrectly along the feasible affine set, often accumulating errors from dominant modes of the unconditional prior that are irrelevant to the specific observation.

Methodology

The authors propose a two-stage sampling framework grounded in a normal–tangent decomposition of the conditional score function under Gaussian noising.

1. Normal–Tangent Decomposition

For a variance-exploding (VE) diffusion process $X_t = Z + W_t$, the conditional score $s^*_t(x)$ can be decomposed into normal ($P_\perp$) and tangent ($P_\parallel$) components relative to the row space of $A$:

• Normal Component: The normal score $P_\perp s^*_t(x)$ is available in closed form via Tweedie's formula and the observation $b = P_\perp Z$. It is given by $\frac{1}{t}P_\perp(b - x)$.
• Tangent Component: The tangent score $P_\parallel s^*_t(x)$ is unknown. Projection-based methods approximate this by the unconditional tangent score $P_\parallel s_t(x)$.

The authors identify that the error in standard zero-shot sampling arises from replacing the unknown conditional tangent score with the unconditional one. This approximation is most problematic at high noise levels where the unconditional score aggregates information from all data modes, potentially pulling the sample toward irrelevant clusters.

2. Proposed Sampler: Projected-Langevin Initialization + Guided Denoising

To mitigate this bias, the authors propose a three-step algorithm (Algorithm 1):

1. Initialization at Intermediate Noise ($t^*$): Instead of starting the reverse process from the highest noise level ($T$), the sampler starts at an intermediate "safe" noise level $t^*$. At this level, the normal component is sampled exactly from the conditional law $\text{Law}(P_\perp X_{t^*} \mid P_\perp Z = b)$.
2. Projected Underdamped Langevin Mixing: Before beginning the reverse denoising, the sampler runs a short phase of projected underdamped Langevin dynamics (using the BAOAB integrator) restricted to the affine set defined by the noisy normal component. This step uses the projected unconditional score $P_\parallel s_{t^*}$ to mix the tangent component. The goal is to produce an initialization that is already well-mixed along the feasible manifold and consistent with the constraint, reducing the accumulation of tangent-score mismatch errors.
3. Guided Reverse Denoising: From the initialized state at $t^*$, the sampler performs guided reverse denoising up to $t=0$. The dynamics use the exact normal drift (to enforce the constraint) and the pretrained unconditional tangent score (to guide the tangent directions).

Key Contributions

The paper makes four primary contributions:

1. Theoretical Decomposition: It derives a rigorous normal–tangent decomposition for affine conditional diffusion, showing that the normal score is exactly determined by the measurement, isolating the tangent score as the sole source of approximation error in zero-shot settings.
2. Algorithmic Proposal: It introduces a sampler that combines exact normal correction with a projected Langevin initialization phase. This phase explicitly mixes the tangent directions at an intermediate noise level before guided denoising, addressing the accumulation of bias seen in naive projection methods.
3. Information-Theoretic Error Bounds: The authors prove a pathwise error bound (Theorem 4) showing that the Kullback–Leibler (KL) divergence between the ideal conditional path and the surrogate path is upper-bounded by the conditional mutual information $I(Z_\parallel; Z_\perp \mid X_{t^*})$. This provides an information-theoretic criterion: the method is accurate when the residual statistical dependence between the tangent and normal components is small at the chosen noise level.
4. Terminal Guarantees: Under a latent Gaussian-mixture model assumption, the paper provides a terminal KL bound (Theorem 9) that separates the error into an initialization term and a pathwise term. Furthermore, under a separation condition on the latent codebook (Theorem 11), the authors show that both error terms decay exponentially with the separation-to-noise ratio, establishing regimes where the method is highly accurate.

Experimental Results

The proposed method, termed LCDM-BAOAB, was evaluated on standard linear imaging inverse problems (inpainting and 8× super-resolution) across CelebA-HQ, LSUN Church, and ImageNet datasets.

• Baselines: The method was compared against DDNM, a strong zero-shot projection-based baseline.
• Metrics: Performance was measured using Fréchet Inception Distance (FID) and Learned Perceptual Image Patch Similarity (LPIPS) under a matched compute budget (100 network function evaluations).
• Findings:
  • LCDM-BAOAB consistently outperformed DDNM across all datasets and tasks.
  • Improvements were most significant on ImageNet and for 8× super-resolution, settings characterized by high tangent-space ambiguity (many high-resolution images consistent with the same low-resolution observation).
  • Visual results showed that LCDM-BAOAB produced cleaner, more coherent reconstructions with fewer texture artifacts compared to DDNM, particularly in high-ambiguity scenarios.

Significance and Claims

The paper claims that enforcing measurement consistency in the normal directions is not sufficient for solving highly ill-posed linear inverse problems with diffusion priors. The central obstruction is the tangent-space ambiguity, where the unconditional score fails to correctly distribute probability mass along the feasible set.

The significance of this work lies in:

• Identifying the Source of Bias: It isolates the tangent-score approximation as the specific mechanism causing bias in existing zero-shot methods.
• Dimension-Free Guarantees: The theoretical bounds are dimension-free, depending instead on the conditional mutual information and the sensitivity of the tangent conditional to the normal code.
• Practical Efficacy without Fine-Tuning: The method achieves state-of-the-art zero-shot performance by leveraging inference-time conditioning and Langevin mixing, without requiring task-specific fine-tuning or learning additional models.
• Theoretical Justification for Tangent Mixing: It provides a theoretical justification for why mixing in the tangent directions at intermediate noise levels improves sampling quality, linking this improvement to the reduction of residual statistical dependence between observed and unobserved components.