Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking

Technical Summary: Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking

Problem Statement
The proliferation of AI-generated synthetic images, ranging from GANs to diffusion models, poses a significant challenge for digital forensics. While early deepfake detection focused on specific generator architectures, the field has shifted toward "universal" detection, which requires robust generalization to unseen models. However, existing state-of-the-art (SOTA) approaches often rely heavily on large-scale pretrained models and complex feature extraction, leading to high computational costs and energy consumption. This conflicts with the principles of "Green AI," which advocates for sustainable, resource-efficient machine learning. Furthermore, specialized domains (e.g., aquaculture) increasingly rely on synthetic data for training, where poor-quality or manipulated synthetic samples can propagate errors in critical applications. The core challenge is to develop a detection strategy that achieves strong generalization across diverse and unseen generators while remaining computationally efficient and resilient to model compression.

Methodology
The authors propose a training strategy centered on Frequency-Domain Masking as a form of supervised data augmentation. Unlike traditional masked image modeling (MIM) used for self-supervised pre-training, this method applies masking during supervised classification training to force the detector to learn robust, generalizable features rather than relying on superficial generator-specific artifacts.

The methodology involves three primary components:

1. Frequency-Domain Masking: Input images are transformed into the frequency domain using the Fast Fourier Transform (FFT). Specific frequency bands (Low, Mid, High, or All) are randomly selected and masked (zeroed out) based on a masking ratio, while phase information is preserved to maintain spatial consistency. The image is then inverse-transformed back to the spatial domain for training.
2. Comparative Augmentations: The study systematically compares frequency masking against:
   • Spatial Masking: Pixel-level and patch-level occlusion in the spatial domain.
   • Geometric Transformations: Random rotation and translation.
3. Structured Pruning Analysis: To evaluate sustainability and efficiency, the authors apply structured layer-wise pruning (removing output channels based on L1-norm importance) to the trained models. They assess whether the robustness gained through frequency masking persists under severe parameter reduction (up to 80% pruning).

Key Contributions

1. Supervised Frequency Masking: The paper introduces a novel training augmentation that applies random masking in the frequency domain within a supervised framework. This approach is distinct from prior MIM works that focus on reconstruction objectives.
2. Empirical Comparison of Augmentations: The authors provide a comprehensive analysis comparing spatial masking, geometric transformations, and frequency masking. They demonstrate that frequency masking yields superior generalization capabilities compared to spatial or geometric methods.
3. Green AI Alignment: The study validates that frequency masking enhances model robustness even under significant model compression (structured pruning), offering a pathway to lightweight, sustainable deepfake detectors.
4. Cross-Domain Validation: The method is evaluated not only on standard GAN and diffusion benchmarks but also on a specialized "FakeFish" dataset for aquaculture, demonstrating applicability in domains where synthetic data quality is critical.

Results

• Generalization Performance: On standard benchmarks (Wang et al., Ojha et al.), frequency masking achieved the highest Mean Average Precision (mAP) and Area Under the ROC Curve (AUROC). Specifically, a 15% masking ratio applied across all frequency bands yielded optimal results (88.10% mAP, 88.72% AUROC), outperforming spatial masking and geometric transformations.
• Pruning Resilience: When models were subjected to structured pruning (up to 80% parameter reduction), those trained with frequency masking retained significantly higher performance compared to baselines and those trained with translation or rotation alone. For instance, on MobileNetv2 with 50% pruning, frequency masking maintained an mAP of 73.15%, whereas the baseline dropped to 53.95%.
• State-of-the-Art Improvement: Integrating frequency masking with existing SOTA backbones (e.g., Wang et al., Gragnaniello et al.) consistently improved their performance across all tested datasets, including unseen diffusion models.
• Specialized Domain: In the aquaculture "FakeFish" dataset, the proposed method outperformed prior works by a large margin (e.g., 79.84% mAP vs. 73.38% for Wang et al. on ControlNet-generated images), proving effective at detecting low-quality or manipulated synthetic data in niche applications.
• Ablation Studies: The study found that masking the Green channel yielded slightly higher performance than Red or Blue, but masking all channels (RGB) remained the most robust choice. While Discrete Cosine Transform (DCT) showed marginal gains over FFT, FFT was retained for its established efficacy in forensic literature.

Significance and Claims
The paper claims that frequency-domain masking serves as a practical and sustainable step toward universal deepfake detection. By suppressing reliance on generator-specific spectral artifacts during training, the method encourages the learning of intrinsic structural features that generalize across diverse generative architectures. The authors emphasize that this approach aligns with Green AI principles by enabling high-performance detection on compressed, resource-constrained models without sacrificing generalization. The work suggests that frequency-based representations are particularly effective for identifying synthetic media because they capture the subtle, periodic artifacts introduced by upsampling and generation pipelines that are often invisible in the spatial domain. The authors conclude that while the method shows promise, it requires careful tuning of masking ratios and may need further adaptation for video (temporal cues) and extreme compression scenarios.