Technical Summary: FakeI2V-Bench

Problem Statement

Recent advancements in Video Generation Models (VGMs), such as Sora and Gen2, have intensified the threat of deepfake videos, creating an urgent need for robust detection mechanisms. While significant research has focused on video-level deepfake detectors, these approaches face fundamental limitations, including poor generalization across diverse content and high computational overhead. Concurrently, image-level deepfake detectors have achieved remarkable progress but remain underexplored in the video domain. Existing benchmarks often lack systematic evaluation of image-level detectors for video tasks, frequently relying on frame-level aggregation without addressing the distribution shift between static images and video frames. Furthermore, current benchmarks often suffer from outdated detector coverage and limited content diversity, failing to reflect the capabilities of the latest generation models.

Methodology

The authors propose FakeI2V-Bench, a comprehensive benchmarking framework, and IV-Bridge, a general enhancement framework to adapt image-level detectors for video scenarios.

1. FakeI2V-Bench Framework

The benchmark is constructed around four core modules:

• Evaluation Datasets: A collection of 97,548 videos divided into facial and general categories. It integrates four datasets:
  • Celeb-DF v2 (CDFV2) and DeepFake Detection (DFD) for facial content.
  • GenVideo (GV) and GenVidBench (GVB) for general content generated by diverse VGMs (including diffusion and autoregressive models like Sora, Gen2, and SVD).
• Detectors Evaluated:
  • Video-level: Eight state-of-the-art detectors, including face-focused (e.g., FTCN, LAA-Net) and general-purpose models (e.g., DeMamba, D3).
  • Image-level: Twelve representative detectors trained on GAN or diffusion data with various backbones (e.g., ResNet, Vision Transformer).
• Evaluation Metrics: Area Under Curve (AUC) and Average Precision (AP).
• Aggregation Strategies: The benchmark evaluates image-level detectors using six frame-to-video aggregation modes: Sampling (SMP), Averaging (AVG), Maximum (MAX), Minimum (MIN), Median (MED), and Variance (VAR).

2. IV-Bridge Enhancement Framework

To address the performance gap when applying image-level detectors to videos, the authors introduce IV-Bridge, a two-stage pipeline:

• Video-Frame Fine-Tuning (VFT): Image-level detectors are fine-tuned on a dataset of video frames (derived from FF++ and GenVideo) to bridge the distribution gap between static images and dynamic video content. The fine-tuning process adapts the loss function based on the detector's output type (two-logit vs. single-logit).
• Multi-Mode Aggregation (MMA): Instead of relying on a single fixed aggregation strategy, IV-Bridge extracts features from six different frame-to-video aggregation modes. These features are fed into a lightweight Random Forest classifier to learn the optimal combination of modes for the final video-level decision.

Key Contributions

1. FakeI2V-Bench: The first benchmark to systematically evaluate both video-level and image-level detectors across a wide range of datasets, including the latest 2025-generation models (GVB). It covers 97,548 videos and 20 detectors.
2. IV-Bridge Framework: A novel method to enhance image-level detectors for video detection via VFT and MMA, enabling them to outperform dedicated video-level models.
3. Systematic Evaluation: A rigorous analysis demonstrating that image-level detectors suffer performance degradation on video frames but can be significantly improved through the proposed framework.

Experimental Results

• Video-Level Detectors (RQ1): Current state-of-the-art video-level detectors show limited generalization. While some achieve high AUC (>90%) on specific datasets (e.g., FTCN on DFD), their performance drops significantly (often to ~50% AUC) on other datasets, particularly when moving between facial and general content.
• Image-Level Detectors (RQ2): When applied directly to video frames, image-level detectors experience severe performance degradation compared to their performance on static images (e.g., NPR drops from 92.17% to 55.38% AUC). However, simple aggregation strategies can partially recover performance; notably, the best image-level detector (RINE with MIN aggregation) achieved 80.16% AUC, slightly outperforming the best video-level detector (FTCN at 79.99% AUC).
• IV-Bridge Performance (RQ3): The IV-Bridge framework significantly boosts performance.
  • Superiority: Eleven out of twelve IV-Bridge-enhanced detectors surpassed the strongest video-level detector (FTCN). The best variant, RINE-IV, achieved 93.80% AUC and 97.63% AP.
  • Generalization: Enhanced detectors demonstrated stronger generalization across different VGMs (e.g., Sora, Gen2, SVD) compared to video-level detectors.
  • Efficiency: Several enhanced image-level detectors (e.g., Patch-IV, CNNDet-IV) achieved higher performance with significantly lower model parameters and inference times compared to video-level detectors. For instance, Patch-IV (4.34M parameters) outperformed FTCN (14.77M parameters) in both speed and accuracy.
  • Sparse Forgery: In short-term forgery scenarios (20% forged frames), IV-Bridge-enhanced detectors maintained competitive performance, with Patch-IV achieving 77.15% AUC.

Significance and Claims

The paper claims that FakeI2V-Bench establishes a rigorous benchmark for deepfake video detection, highlighting the underutilized potential of image-level detectors. The authors assert that:

1. Current video-level detectors lack sufficient cross-domain generalization.
2. Application of image-level detectors to video is insufficient without systematic adaptation, but such adaptation is highly effective.
3. The IV-Bridge framework provides a practical, efficient, and high-performing pathway to extend mature image-level detectors to the video domain, offering a superior alternative to training complex video-level models from scratch.

The work suggests that future research should focus on leveraging and adapting image-level detectors rather than solely developing new video-specific architectures, given the latter's computational costs and generalization challenges.