MIRAGE: Benchmarking and Aligning Multi-Instance Image Editing

1. Problem Statement

Instruction-guided image editing has advanced significantly with models like FLUX.2 and Qwen-Image-Edit. However, these State-of-the-Art (SOTA) models struggle in complex multi-instance scenarios where a single image contains multiple similar objects (e.g., three identical sculptures or five similar dogs) that require distinct, independent modifications based on a composite instruction.

Current models suffer from two primary failure modes in these settings:

• Over-editing: Applying an edit intended for one specific instance to all similar instances in the image (e.g., changing the material of all sculptures instead of just the leftmost one).
• Spatial Misalignment: Failing to correctly identify the target instance, leading to edits applied to the wrong object or unintended modifications to the background.

Existing benchmarks lack the density and complexity to diagnose these fine-grained errors, as they primarily focus on single-object edits or simple referring expressions.

2. Methodology

The paper proposes a two-pronged approach: a new benchmark (MIRA-Bench) and a training-free framework (MIRAGE).

A. MIRA-Bench: A New Benchmark

To rigorously evaluate multi-instance editing, the authors constructed MIRA-Bench, the first benchmark specifically designed for this task.

• Composition: 100 high-quality image-instruction-mask triplets.
• Complexity: Each image contains 3 to 5 similar instances of the same object.
• Instructions: Each sample includes 5 simultaneous, compositional editing instructions (e.g., "Change the leftmost to X, remove the middle one, add Y to the rightmost...").
• Construction Pipeline:
  1. Image Generation: Uses FLUX.2 to synthesize images based on prompts ensuring multiple distinct instances with unique local attributes.
  2. Instruction Generation: Uses a Vision-Language Model (VLM, Qwen3-VL) to generate 5 atomic edits per image, ensuring one-to-one correspondence with specific instances.
  3. Ground Truth: Uses the VLM to predict bounding boxes, which are refined by SAM2 to generate precise segmentation masks for evaluation.

B. MIRAGE Framework: Multi-Instance Regional Alignment via Guided Editing

MIRAGE is a training-free inference framework designed to plug into existing diffusion models (like FLUX.2 or Qwen-Image-Edit) to solve the over-editing and misalignment issues. It operates in two stages:

1. Instruction Parsing & Target Localization (VLM-based)

• A VLM (e.g., Qwen3-VL) acts as a parser to decompose a global, complex instruction into $K$ atomic sub-tasks.
• For each sub-task, the VLM extracts a referring expression (e.g., "the leftmost turtle") and predicts a bounding box for that specific region.
• This creates a set of region-level pairs: $(x^k, I^k)$, where $x^k$ is the cropped region and $I^k$ is the specific edit instruction.

2. Multi-Branch Parallel Denoising Strategy
MIRAGE employs a hybrid inference strategy that runs a global branch and $K$ regional branches in parallel:

• Early Stage ($t > \rho T$):
  • The global branch is inactive.
  • Each regional branch independently performs denoising on its specific cropped region ($x^k$) conditioned on its specific instruction ($I^k$).
  • The resulting latent representations are spatially mapped back to the full image grid and overwritten onto the background latents (derived from the forward diffusion of the reference image).
  • Goal: Inject precise semantic edits into specific regions early in the generation process.
• Late Stage ($t \le \rho T$):
  • The regional branches terminate.
  • The global branch takes over, performing conditional denoising on the full image using the global instruction.
  • Crucially, the background regions (outside the target masks) are locked to the reference trajectory (forward diffusion of the original image) to prevent drift. Only the target regions are updated by the global branch.
  • Goal: Refine details and ensure global coherence while strictly preserving the background integrity.

3. Key Contributions

1. MIRA-Bench: A novel benchmark filling a critical gap in image editing evaluation, featuring multi-instance scenes with 3–5 similar objects and 5 compositional instructions per image.
2. MIRAGE Framework: A training-free, plug-and-play method that significantly improves the precision of mainstream diffusion models without requiring additional model training or fine-tuning.
3. Novel Inference Strategy: The introduction of a multi-branch parallel denoising approach combined with a reference trajectory for background preservation, effectively decoupling target edits from background stability.

4. Experimental Results

The authors evaluated MIRAGE on MIRA-Bench and the existing RefEdit-Bench using SOTA models (FLUX.2, Qwen-Image-Edit-2511) and metrics like EditScore (Prompt Following, Consistency, Perceptual Quality).

• Quantitative Performance:
  • Integrating MIRAGE with FLUX.2 [Dev] improved Prompt Following (PF) from 6.98 to 8.09 and Consistency (Cons) from 8.38 to 9.01 on MIRA-Bench.
  • MIRAGE-enabled FLUX.2 outperformed the proprietary GPT-Image-1.5 model in both PF and Consistency on complex multi-instance tasks.
  • Significant improvements were observed across all base models, with Consistency gains up to +0.7 points.
• Qualitative Performance:
  • MIRAGE successfully resolved over-editing (e.g., changing only the specified dog's eyes without affecting others) and spatial misalignment (e.g., correctly removing the 5th goat in a row of 5).
  • Background preservation was significantly enhanced, preventing the "drift" often seen in standard diffusion editing.
• Efficiency:
  • MIRAGE is computationally efficient. While sequential editing (applying instructions one by one) takes ~5x longer and accumulates errors, MIRAGE performs joint inference in a single trajectory.
  • It is actually ~15% faster than the base model alone because regional edits are performed on smaller, cropped latent spaces during the early stages.

5. Significance and Impact

• Solving a Critical Bottleneck: MIRAGE addresses the "multi-instance" bottleneck that has hindered the deployment of instruction-guided editing in real-world scenarios where scenes are rarely single-object.
• Training-Free Scalability: By avoiding fine-tuning, MIRAGE can be immediately applied to any new SOTA diffusion model, making it a highly scalable solution.
• Benchmarking Standard: MIRA-Bench sets a new standard for evaluating fine-grained control, pushing the community to develop models capable of handling complex, compositional reasoning rather than just global style transfers.
• Robustness: The framework demonstrates that separating the "localization" (via VLM) and "generation" (via diffusion) tasks, while synchronizing them via a reference trajectory, is a superior strategy for maintaining spatial consistency in complex edits.

In conclusion, MIRAGE represents a significant step forward in making AI image editing reliable for complex, multi-object scenes, offering a robust, training-free solution that outperforms even proprietary models in specific high-difficulty tasks.