Technical Summary: Diffusion Model Attribution via Spectral Coupling of Denoiser Responses

1. Problem Statement

The paper addresses the challenge of diffusion model attribution: determining which specific diffusion model generated a given image. This is critical for provenance verification, intellectual property protection, and misuse detection.

The core difficulty lies in the fact that diffusion models trained on different datasets often converge to similar score functions, producing visually indistinguishable outputs. Consequently, attribution methods relying solely on generated images (output-space signals) face inherent ambiguity. Existing non-invasive approaches suffer from fundamental limitations:

• Classifier-based methods struggle to distinguish architecturally similar variants that produce visually similar outputs.
• Inversion-based methods (e.g., RONAN, LATENTTRACER) rely on reconstruction losses. When candidate models share the same autoencoder (VAE), these losses become uninformative, causing these methods to collapse to random chance.
• Invasive methods (e.g., watermarking) require model modification or retraining, which degrades generation quality and is inapplicable to existing open-source models.

The paper posits that the discriminative signal does not reside in the generated output but in the denoising function itself, specifically its spectral geometry.

2. Methodology: Spectral Denoising Signatures (SDS)

The authors propose Spectral Denoising Signatures (SDS), a non-invasive attribution method that fingerprints a model by probing its denoising behavior rather than analyzing its outputs.

Core Insight

A model's denoising score function exhibits a distinctive spectral geometry, reflected in how it redistributes energy across spatial frequency bands during the denoising process. This behavior is intrinsic to the model's learned score function and is independent of the specific prompt or noise seed used for generation.

The SDS Pipeline

1. Standardization: To ensure signatures reflect model identity rather than artifacts, all candidate models use a shared VAE for encoding input images into a consistent latent space ($z_0$) and a shared scheduler. Probing is done with empty text embeddings to remove prompt-dependent signals.
2. Spectral Partitioning: The frequency plane of the latent space is partitioned into $K$ concentric radial rings. Ring 0 covers low frequencies (global structure), while Ring $K-1$ covers high frequencies (fine texture).
3. Frequency-Controlled Probing:
   • Injection: Band-limited noise ($\eta^{(k)}$) is injected into a specific frequency ring $k$ at a chosen timestep $t$.
   • Denoiser Query: The denoiser $\hat{\epsilon}_\theta(z_t, t, \emptyset)$ is queried to predict the noise.
   • Residual Analysis: The residual $r = \hat{\eta} - \eta^{(k)}$ is computed. The energy of this residual is measured across all output frequency rings $j$.
   • Coupling Matrix: This yields a coupling matrix $C \in \mathbb{R}^{K \times K}$, where $C[k, j]$ quantifies how energy injected in ring $k$ is routed to ring $j$ by the denoiser. This matrix is a projection of the denoiser's Jacobian.
4. Signature Aggregation:
   • Measurements are taken across multiple perturbation amplitudes ($S$) and multiple timesteps ($T_n$) spanning the denoising trajectory.
   • The resulting coupling matrices are concatenated and flattened to form a high-dimensional signature vector $\Phi(M, z_0)$.
   • Prototype Construction: For each model, a prototype signature $\Psi(M)$ is created by averaging signatures from $N$ reference images. Row normalization is applied to remove content-dependent scale variations.
5. Identification: A query image is encoded, probed to generate its signature, and compared against stored prototypes using a nearest-prototype match (argmin) or a trained linear classifier (LinearSVC).

3. Key Contributions

1. SDS Framework: A non-invasive attribution method that probes the denoiser's spectral geometry via standard forward passes alone, requiring no inversion, optimization, or model modification.
2. Intrinsic Model Signatures: Demonstration that spectral coupling signatures are model-intrinsic, distortion-robust, and discriminative across variations in training data, architecture, and training procedures.
3. Resolution of Baseline Failure Modes: Proof that probing the denoiser directly resolves the fundamental failure of existing inversion-based baselines when models share the same autoencoder.

4. Experimental Results

The method was evaluated on a closed-set task involving eight diverse diffusion models (including Stable Diffusion v1.4/v1.5, SDXL, PixArt-α, and fine-tuned variants) with varying training data, architectures, and training procedures.

• Attribution Accuracy: SDS achieved ≈99.9% accuracy across the eight models using a LinearSVC classifier. It successfully distinguished between SD v1.4 and SD v1.5 (which share nearly all weights and the same VAE), a case where output-based features failed completely.
• Cross-Domain Generalization: When prototypes built from SD-Prompts were tested on images generated from MS-COCO prompts, LinearSVC maintained 96.2% accuracy, a drop of only 3.8 percentage points. This indicates the signatures are largely model-intrinsic rather than prompt-dependent.
• Robustness: SDS remained highly robust against strong post-processing distortions (rotation, JPEG compression, cropping, brightness scaling, noise, and blur). While pixel-space distortions degraded image quality significantly, the spectral routing signature remained intact, with LinearSVC achieving >90% accuracy even under severe noise and blur.
• Baseline Comparison:
  • RONAN and LATENTTRACER (inversion-based) collapsed to ~20–27% accuracy (near random chance) for the five models sharing the same VAE.
  • SDS achieved 100% accuracy on these same models, outperforming the best baseline (LATENTTRACER) by 55 percentage points and RONAN by 70 percentage points on the most challenging model pairs.

5. Significance and Claims

The paper claims that model identity is encoded in the spectral geometry of the denoiser's Jacobian, not in the generated outputs. This finding establishes a principled basis for attribution that is independent of the specific autoencoder used.

• Practicality: SDS requires only standard forward passes, making it applicable to the vast ecosystem of existing open-source diffusion models without the need for retraining or watermarking.
• Fundamental Insight: The work demonstrates that even when models converge to similar score functions visually, they retain distinct "spectral fingerprints" in how they route energy across frequency bands.
• Limitations: The authors acknowledge that SDS currently operates in a closed-set setting (the source model must be in the registry) and requires white-box access to candidate model weights (though this is satisfied by open-source models). It does not detect post-generation tampering.

In summary, the paper argues that by shifting the focus from what a model generates to how it denoises, one can achieve reliable, non-invasive attribution even in the most challenging scenarios where models share architectures and autoencoders.