Spectral Forensics of Diffusion Attention Graphs for Copy-Move Forgery Detection

1. Problem Statement

Copy-Move Forgery (CMF) is a pervasive form of image manipulation where a region of an image is duplicated and pasted elsewhere within the same image to conceal or fabricate content. Detecting CMF is challenging because the duplicated regions often undergo geometric transformations, compression, and blending, making them visually indistinguishable from the original.

Traditional detection methods rely on:

• Hand-crafted features: Block matching, keypoint descriptors (SIFT/SURF), or dense field methods.
• Supervised Deep Learning: Models trained on specific forgery datasets, which often fail to generalize to unseen manipulation types or require extensive retraining.

The paper addresses the need for a training-free, generalizable forensic framework that does not rely on forgery-specific labels or retraining.

2. Core Hypothesis & Theoretical Framework

The authors propose a novel theoretical insight: Copy-move forgery induces approximate subgraph duplication in the self-attention affinity graphs of pretrained diffusion models.

• The Mechanism: When a region is copied, the self-attention mechanism of a diffusion model (specifically Stable Diffusion) creates a "duplicated subgraph" in its token affinity matrix.
• Spectral Consequence: According to perturbation theory and the Weyl interlacing theorem, duplicating a subgraph compresses local eigenvalue spacings and redistributes spectral mass in the Normalized Graph Laplacian.
• Why Normalized Laplacian? The raw attention matrix conflates content magnitude with structural connectivity. The authors argue that degree normalization (constructing $L = I - D^{-1/2}AD^{-1/2}$) isolates the structural connectivity pattern, making the spectral redistribution caused by duplication detectable, whereas raw spectra are often masked by magnitude variations.

3. Methodology: GraphSpecForge

GraphSpecForge is a fully training-free forensic pipeline that operates as a one-class anomaly detector. It requires no forgery labels for calibration.

Pipeline Steps:

1. Attention Extraction:
   • A single forward pass is performed through a pretrained Stable Diffusion v1.5 U-Net.
   • Self-attention affinity matrices ($A^{(\ell)}$) are extracted from all 16 transformer layers (spanning encoder, bottleneck, and decoder).
2. Graph Construction:
   • Matrices are symmetrized and non-negativity clipped to form undirected weighted graphs.
   • Normalized Graph Laplacians are computed for each layer.
3. Spectral Feature Extraction:
   • Eigenvalue spectra are computed for both the raw attention and the normalized Laplacian.
   • Feature Groups:
     • Transport Features: Wasserstein-1 distance ($W_1$) between the test image's spectrum and a pre-computed authentic reference distribution (cached from authentic training images only).
     • Filter-Bank Features: Heat trace, spectral entropy, and band-pass summaries.
     • Duplication-Sensitive Features: Mass near $\lambda=1$, local spacing compression.
     • Non-Spectral Controls: Degree statistics, spectral radius, etc., to rule out trivial graph properties.
4. Layer Selection (FSEL):
   • A Forgery Spectral Expert Layer (FSEL) score ranks layers based on distributional separation, causal perturbation sensitivity (ablation of top eigencomponents), and stability.
   • The system consistently identifies decoder layers (specifically up_blocks.2 and up_blocks.3) as the most informative.
5. Decision & Fusion:
   • Anomaly scores from top-$k$ layers are fused using reliability-weighted aggregation (softmax).
   • A threshold is set based on authentic validation quantiles (e.g., 1% False Positive Rate).

4. Key Contributions

1. Theoretical Framework: Establishes a link between copy-move forgery and spectral perturbation in diffusion attention graphs, grounded in graph duplication and eigenvalue interlacing theory.
2. Training-Free Detector: Achieves detection without any forgery-specific training data or fine-tuning, relying solely on the structural priors of a pretrained diffusion model.
3. Laplacian Superiority: Demonstrates that the Normalized Graph Laplacian consistently outperforms raw attention spectra by approximately +0.057 AUROC, proving that structural connectivity is a more robust forensic signal than raw attention magnitude.
4. Multi-Benchmark Validation: Validates the method across four diverse datasets (RecodAI-LUC, MICC-F220, CoMoFoD, COVERAGE) without retraining, showing the mechanism is portable across different manipulation styles (SIFT-style, geometric transforms, occlusion).
5. Rigorous Falsification: Includes extensive ablation studies, null-graph controls, and specificity tests to confirm the signal is not a statistical fluke or a response to generic image anomalies.

5. Experimental Results

The method was evaluated on four benchmarks. Key findings include:

• RecodAI-LUC (Large-scale, $n=5,128$):

  • AUROC: 0.606 (95% CI: 0.580–0.638).
  • Significance: Permutation $p = 0.005$.
  • Low-FPR Performance: TPR@1%FPR = 12.1%.
  • Note: While moderate, this is statistically significant for a zero-shot method.

• Cross-Dataset Performance (Generalization):

  • MICC-F220: AUROC = 0.752.
  • CoMoFoD: AUROC = 0.774, AUPRC = 0.833, MCC = 0.499, TPR@1%FPR = 32.5%.
  • COVERAGE: AUROC = 0.673.
  • Observation: The method performs significantly better on smaller, specific benchmarks than the large-scale, noisy RecodAI-LUC, suggesting the signal is robust but sensitive to dataset noise floors.

• Ablation Insights:

  • Laplacian vs. Raw: Laplacian consistently wins (+0.056 to +0.113 AUROC gain).
  • Feature Importance: Wasserstein transport distance carries the majority of the signal; duplication-mass features provide marginal gains.
  • Layer Depth: Decoder layers (mid-resolution) are critical; fusing top-$k$ layers improves performance over single-layer detection.
  • Specificity: The detector distinguishes copy-move from natural self-similarity (e.g., tiled patterns) and generic corruptions (blur, noise), though it is not perfectly specific to copy-move alone.

6. Significance and Limitations

Significance:

• Paradigm Shift: Moves away from "learning to detect forgeries" toward "analyzing structural perturbations in foundation models."
• Portability: Proves that the spectral anomaly induced by subgraph duplication is a fundamental property of copy-move forgery, detectable across different datasets and manipulation types without retraining.
• Interpretability: The method offers a theoretical explanation (spectral redistribution) rather than a "black box" deep learning decision.

Limitations:

• Accuracy Ceiling: The AUROC (0.606 on the main benchmark) is lower than supervised state-of-the-art detectors (which often exceed 0.9). This is expected as the method is zero-shot.
• No Localization: The method provides an image-level anomaly score but cannot localize the forged region (pixel-level masks) because it operates on global graph eigenvalues.
• Computational Cost: Full eigendecomposition of high-resolution attention matrices (e.g., 64x64 tokens) is computationally expensive.
• Specificity Confounds: The detector can be triggered by natural high-repetition structures (e.g., symmetric objects), though less strongly than by actual forgeries.

Conclusion

GraphSpecForge introduces a principled, training-free approach to copy-move forgery detection by leveraging the spectral properties of diffusion model attention graphs. While not yet a replacement for high-accuracy supervised detectors, it establishes a new forensic paradigm where approximate subgraph duplication creates a detectable spectral redistribution in the normalized Laplacian. The method's ability to generalize across four distinct benchmarks without retraining highlights the potential of using foundation models as structural forensic tools.