SurFITR: A Dataset for Surveillance Image Forgery Detection and Localisation

1. Problem Statement

Current image forgery detection and localisation models struggle to generalise to surveillance-style imagery. Existing benchmarks (e.g., FaceForensics++, GenImage) primarily focus on object-centric or face-centric natural images with large manipulated regions or full-image synthesis. In contrast, real-world surveillance forgery presents unique challenges:

• Subtlety and Localisation: Tampering is often highly localised (e.g., removing a specific person or bag) rather than global.
• Complex Context: Scenes involve varied viewpoints, occluded subjects, lower visual quality, and complex backgrounds.
• Domain Gap: Models trained on standard datasets fail to detect these subtle manipulations in surveillance settings, leading to significant performance degradation.
• Data Scarcity: There is a lack of large-scale, fine-grained, semantically grounded datasets specifically designed for surveillance forgery.

2. Methodology: The SurFITR Dataset and Pipeline

The authors introduce SurFITR (Surveillance Forgery Image Test Range), a dataset containing over 137,000 tampered images derived from six diverse surveillance corpora (UCF-Crime, NTU-Fight, ShanghaiTech, CUHK-Avenue, UCSD-Ped1/2).

A. Dataset Composition

• Scale & Diversity: 137k+ tampered images + 137k+ original images.
• Editing Operations: Four types of manipulations:
  1. Removal (RM): Deleting an entity.
  2. Targeted Replacement (RE(T)): Replacing an entity with a specific, context-consistent alternative.
  3. Open-ended Replacement (RE(O)): Replacing with a semantically related but non-identical object.
  4. Addition (ADD): Inserting a new entity.
• Targets: Both Human and Object entities.
• Generation Models: Images are generated using five State-of-the-Art (SOTA) models (e.g., FLUX.1, Qwen-Image, SD3.5) to facilitate cross-model generalisation studies.
• Splits:
  • SurFITR-Base: Primary training and evaluation set (FLUX.1-generated).
  • SurFITR-Transfer: Evaluation set for cross-domain shifts (generated by 4 other models).

B. Automated Generation Pipeline

The dataset is constructed via a Multimodal LLM (MLLM)-powered pipeline to ensure semantic coherence and realism:

1. Frame Selection: An open-vocabulary detector (YOLO-World) and segmentation model (SAM 2.1) identify objects. An MLLM (Qwen2.5-VL) scores frames based on forensic value.
2. Instruction Generation:
   • Removal/RE(O): Direct prompts based on masks.
   • Targeted Replacement: A two-stage process where an MLLM describes the scene, and a text-based LLM proposes context-consistent substitutions.
   • Addition: The MLLM proposes objects and locations, enforcing physical plausibility (depth, perspective). Depth maps (Depth Anything V2) ensure correct scaling.
3. Localised Tampering: Manipulation is restricted to pixel-level masks and boundary regions using patch-based generation. The rest of the image remains untouched.
4. Verification & Quality Control:
   • Automated Checks: Re-running detectors to confirm removal/addition; checking pixel-level change thresholds.
   • Quality Scoring: MLLMs rate images on Realism (visual plausibility) and Detectability (artifact visibility). Low-quality samples are discarded.
   • Human Verification: A subset (~5%) is manually verified to ensure the automated pipeline matches human perception.

3. Key Contributions

1. SurFITR Dataset: The first large-scale dataset specifically for surveillance-style forgery detection and localisation, featuring fine-grained, spatially localised tampering in complex real-world scenes.
2. MLLM-Driven Pipeline: A novel, fully automated pipeline leveraging Multimodal LLMs and open-world grounding models to generate semantically grounded, scene-specific edits at scale.
3. Comprehensive Benchmarking: Extensive evaluation of 13 baseline models (CNNs, Transformers, and MLLMs) demonstrating that existing methods fail in surveillance settings, while training on SurFITR yields significant improvements.
4. Cross-Domain Analysis: The dataset enables the study of generalisation across different scenes, generation models, and editing types, highlighting specific failure modes in current forensic research.

4. Experimental Results

The authors evaluated 13 baseline models (including PSCC-Net, TruFor, HiFi-Net, and various Qwen/Gemini MLLMs) on SurFITR.

• Zero-Shot Performance (Baseline Failure):
  • Existing detectors trained on natural images perform poorly on SurFITR.
  • Detection: While some models achieved moderate AUROC (>0.5), F1 scores were extremely low (often <0.1), indicating poor score calibration and an inability to distinguish subtle tampering.
  • Localisation: Pixel-level Intersection over Union (P-IoU) and F1 scores were near zero for most models, confirming they cannot localise subtle edits in surveillance footage.
• Training on SurFITR (Significant Gains):
  • Fine-tuning models on SurFITR-Base resulted in substantial improvements in both in-domain and cross-domain settings.
  • In-Domain: Models like PSCC-Net and Qwen3-VL-8B achieved AUROC >0.9 and F1 >0.8 on the Base test set.
  • Cross-Domain: Models trained on SurFITR maintained robust performance when tested on the Transfer set (different generation models), though some degradation occurred due to scene shifts.
• Key Findings:
  • Scene Variation: The primary cause of detection instability is variation in the scene environment (e.g., indoor vs. outdoor, camera quality).
  • Manipulation Size: Localisation performance degrades significantly as the manipulated region becomes smaller.
  • MLLM Potential: Fine-tuned MLLMs (Qwen3-VL-8B) achieved performance comparable to dedicated forensic detectors, suggesting MLLMs can be effective forensic tools when properly trained.

5. Significance and Impact

• Bridging the Gap: SurFITR addresses the critical lack of data for surveillance forensics, moving beyond the "object-centric" bias of current research.
• Benchmark for Robustness: It serves as a rigorous testbed to expose the limitations of current forensic models, particularly their sensitivity to scene context and subtle edits.
• Foundation for Future Research: The dataset and pipeline provide a foundation for developing scene-invariant and fine-grained forensic models.
• Ethical Safeguards: The authors release the dataset under a research-only license and omit specific implementation details of the generation pipeline to prevent misuse for creating real-world forgeries. The goal is to advance detection capabilities to counter generative misinformation in surveillance contexts.

In conclusion, SurFITR demonstrates that while current forensic tools are ill-equipped for surveillance imagery, the dataset itself provides the necessary supervision to train models that can effectively detect and localise sophisticated, subtle forgeries in real-world monitoring scenarios.