Technical Summary: LocateEdit-Bench

Problem Statement

The rapid advancement of AI-powered image editing, particularly the shift from mask-based inpainting to instruction-based editing (driven by models like Multi-Modal Diffusion Transformers), has created a critical gap in digital forensics. While traditional forgery localization methods are effective against manipulations defined by explicit masks (which provide low-level boundary signals), they struggle with instruction-driven edits. These modern edits are semantically coherent and visually seamless, lacking predefined boundaries, which renders existing localization techniques ineffective. There is a lack of large-scale, high-quality datasets and benchmarks specifically designed to evaluate localization methods against this new, instruction-driven paradigm.

Methodology

1. Dataset Construction (LocateEdit-Bench)

The authors introduce LocateEdit-Bench, a large-scale dataset comprising 231,000 edited images with corresponding pixel-level segmentation masks. The construction pipeline involves three main stages:

• Data Collection: Starting from the OmniEdit dataset (1.2M samples), the authors filtered raw images and GPT-4o generated instructions. They utilized Vision-Language Models (Qwen3-VL) to ensure semantic plausibility, discarding instructions where the target object was contextually incongruous or absent. This resulted in ~400K high-quality image-instruction pairs.
• Image Editing: The filtered pairs were processed using four state-of-the-art instruction-based editing models: Qwen-Image-Edit, Flux-Kontext, Step1X-Edit, and BAGEL. These models represent both specialized editing architectures and unified generation-understanding frameworks. Images were resized to ~1024×1024 pixels to match model preferences.
• Mask Generation: Since instruction-based editing does not provide ground-truth masks, the authors employed a multi-model segmentation framework. ChatGPT analyzed instructions to identify target objects, which were then segmented using SAM3 (Segment Anything Model 3). A confidence threshold (>0.5) was applied to ensure mask quality.

The dataset covers three fundamental manipulation types: Object Addition, Object Swap, and Attribute Modification.

2. Evaluation Protocols

To rigorously assess localization methods, the authors designed two evaluation protocols:

• Protocol 1 (Full-Set Evaluation): Models are trained on a composite dataset containing all editing models and tested on each individual category to evaluate general localization capabilities.
• Protocol 2 (Cross-Editor Generalization): Models are trained exclusively on data from a single editing model and tested on all other categories. This simulates real-world scenarios where forensic tools encounter unseen editing algorithms.

Additionally, robustness was tested against common post-processing operations, specifically JPEG compression and Gaussian blurring.

3. Metrics

Evaluation focuses on pixel-level performance using:

• Classification Metrics: Accuracy, AUC, and F1-Score.
• Segmentation Metrics: Dice Similarity Coefficient (Dice Score) and Intersection over Union (IoU).
• Thresholding: A standard 0.5 threshold is used for most models, while ObjectFormer uses an Equal Error Rate (EER) threshold.

Key Results

Performance of Existing Methods

Extensive benchmarking of state-of-the-art localization methods (e.g., ObjectFormer, PSCC-Net, IML-ViT, PIM-Net, Mesorch) and vanilla segmentation models (CLIP, SegFormer, DINOv3) revealed significant limitations:

• General Feasibility but High Difficulty: While localization is feasible, current methods show substantial performance gaps. Traditional pixel-wise metrics (Accuracy, AUC) are often inflated by large, untouched background regions, masking poor performance on the actual edited areas.
• Semantic Reasoning is Crucial: SegFormer (a vanilla segmentation model) achieved the best overall performance, followed closely by Mesorch (which uses SegFormer as a backbone). This suggests that deep integration of semantic features is more effective than low-level anomaly detection for this task. Methods relying primarily on low-level signals (e.g., ObjectFormer) performed poorly on segmentation metrics.
• Editor Dependency: Performance varies significantly based on the editing model. BAGEL-edited images were the most challenging to localize, while Flux-Kontext images were the easiest. This indicates that forgery traces are highly specific to the editing architecture.

Generalization and Robustness

• Cross-Editor Failure: In the Cross-Editor Generalization protocol, all models suffered substantial performance degradation when tested on unseen editors. This highlights that current methods fail to capture cross-model commonalities, likely because different editors introduce distinct visual artifacts and semantic inconsistencies.
• Sensitivity to Degradation: Robustness analysis showed a consistent decline in performance under JPEG compression and Gaussian blurring. Segmentation-based metrics (F1, Dice) were particularly sensitive, suggesting that image quality degradation obscures fine-grained manipulation traces.

Significance and Claims

The paper positions LocateEdit-Bench as a foundational resource for the next generation of image forensics. Its primary contributions are:

1. Bridging the Gap: It is the first large-scale benchmark dedicated to the localization of instruction-based image edits, addressing the obsolescence of datasets focused solely on inpainting.
2. Revealing Limitations: The benchmark exposes the critical limitations of current forensic tools, demonstrating that they are not robust to the semantic coherence and visual seamlessness of modern instruction-driven edits.
3. Guiding Future Research: By establishing that semantic feature integration is superior to low-level signal analysis and highlighting the generalization gap across different editors, the work provides a clear direction for developing editor-agnostic and semantically-aware localization methods.

The authors conclude that while the task is challenging, the dataset provides the necessary infrastructure to keep pace with the evolving landscape of AI-powered image manipulation, ultimately supporting the development of more reliable detection tools for digital media integrity.