🤖 AI

OSOR: One-Step Diffusion Inpainting for Effect-Aware Object Removal

OSOR is a one-step diffusion model that achieves efficient, effect-aware, and mask-robust object removal by introducing an occupancy-guided discriminator, an alpha head for handling imperfect masks, and a semantic-anchored verification pipeline to curate high-quality training data, resulting in superior perceptual quality with significantly faster inference than multi-step baselines.

Original authors: Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang

Published 2026-06-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Qinming Zhou, Chenxi Sun, Deyang Kong, Junhao He, Xiangheng Tang, Peike Yu, Haotian Wu, Leilei Cao, Linfeng Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a photo of a beautiful beach, but someone accidentally left their old, rusty fishing boat in the middle of the shot. You want to remove it.

The Problem:
If you just use a standard "eraser" tool, it might delete the boat but leave behind a weird, dark shadow or a reflection on the water that doesn't belong there anymore. Also, if you try to draw the eraser circle yourself, you might miss a tiny bit of the shadow or accidentally cover some of the sand.

Most current AI tools that do this job are like slow, over-thinking chefs. They take a long time to "cook" the image, going through dozens of steps to make sure the background looks perfect. They are powerful but too slow for real-time use on your phone or for quick edits.

The Solution: OSOR (One-Step Object Removal)
The paper introduces a new method called OSOR. Think of OSOR as a magical, instant photo editor that can remove an object and its messy side effects (like shadows and reflections) in a single, lightning-fast step.

Here is how it works, broken down into three simple tricks:

1. The "Smart Boundary" Teacher (Occupancy-Guided Discriminator)

When you try to erase something in one single step, the edges often look blurry or fuzzy, like a bad photocopy.

The Analogy: Imagine a teacher grading a student's drawing. A normal teacher just says, "This part is inside the box, this part is outside." But OSOR's teacher is smarter. It looks at the edge of the box and says, "This pixel is 50% inside the box, so it should be half-painted."
The Result: This "fractional" teaching helps the AI learn exactly where the object ends and the background begins, creating a sharp, clean edge without the blur.

2. The "Guess-Work" Helper (Alpha Head)

Users are often bad at drawing perfect circles around objects. You might draw a circle that is too small, missing the shadow, or too big, covering too much.

The Analogy: Think of the AI as a painter who is given a rough sketch of what to erase. Instead of just erasing exactly what you drew, the AI has a "sixth sense" (the Alpha Head). If you draw a small circle around a boat, the AI looks at the water and thinks, "Ah, there's a shadow here that belongs to the boat, even though you didn't draw it." It then gently paints over that shadow too.
The Result: Even if your selection is messy or incomplete, the AI figures out the rest of the "mess" (shadows, reflections) and cleans it up automatically.

3. The "Fact-Checker" Pipeline (SAVP)

To teach this AI to be so good, the researchers needed a massive library of "Before" and "After" photos. But finding perfect examples is hard because many are fake or messy.

The Analogy: Imagine trying to teach a student by showing them thousands of homework examples. Some examples are wrong. The researchers built a robot fact-checker (SAVP). This robot looks at a "Before" and "After" photo, checks if the changes match the instructions (e.g., "Remove the boat"), and verifies that the shadow really disappeared. If the example is good, it keeps it; if it's bad, it throws it away.
The Result: They used this robot to build a huge, high-quality textbook (called CORNE) with 280,000 perfect examples, which the AI studied to learn how to remove objects and their effects perfectly.

The Big Win

The paper claims that while other methods might take 6 to 25 seconds to edit a photo, OSOR does it in under one second (specifically, about 0.4 to 0.9 seconds on a powerful computer).

Speed: It is 4 to 30 times faster than the next best methods.
Quality: It removes not just the object, but the "ghosts" (shadows/reflections) left behind, which other fast methods often miss.
Robustness: It works well even if the user draws a messy or incomplete mask.

In short, OSOR is like hiring a professional photo editor who works at the speed of a camera shutter, fixing your photos instantly while making sure no shadows or reflections are left behind.

Technical Summary: OSOR – One-Step Diffusion Inpainting for Effect-Aware Object Removal

Problem Statement

Real-world object removal is a fundamental image editing task that faces three primary challenges:

Effect-Awareness: Simply erasing pixels within a mask is insufficient. Objects often leave persistent visual effects (e.g., cast shadows, reflections, environmental interactions) that require strong semantic understanding of scene geometry and lighting to remove correctly.
Mask-Robustness: In practical scenarios, user-provided masks are frequently imperfect—either too large (covering unrelated content) or too small (failing to encompass the object and its effects). Existing methods often struggle to produce reasonable results with such careless inputs.
Efficiency: Interactive applications, particularly on mobile devices, require extremely low latency. Current state-of-the-art diffusion-based methods, while offering superior perceptual quality, rely on hundreds of billions of parameters and tens of iterative denoising steps, resulting in high computational costs and inference times that are impractical for real-time use.

Methodology

The authors propose OSOR (One-Step Object Removal), a framework designed to achieve efficient, effect-aware, and mask-robust object removal in a single inference step. The methodology is built upon three core technical components:

1. Semantic-Anchored Verification Pipeline (SAVP) and the CORNE Dataset

To address the lack of high-quality, effect-aware training data, the authors introduce SAVP. This pipeline processes noisy instruction-based triplets (e.g., "remove object X") to verify and extract reliable supervision.

Verification: SAVP computes multi-feature difference heatmaps (log-luminance, chromaticity, gradient magnitude) and aligns them with semantic bounding boxes generated by Grounding DINO. It filters out pairs with dispersed artifacts or misaligned semantics.
Mask Synthesis: Validated difference regions are fused with object-core masks generated by SAM2 to create effect-aware target masks.
Dataset: Using SAVP, the authors curated CORNE, a dataset containing 280,000 verified object removal pairs with effect-aware annotations. They also constructed AnimeEraseBench and TextEraseBench to evaluate performance on specific domains (anime images and text overlays).

2. Two-Phase Training Curriculum

OSOR employs a two-stage training strategy to balance boundary consistency with robustness to imperfect masks.

Phase I: Boundary-Consistent One-Step Removal
- Goal: Adapt a pretrained diffusion backbone (SDXL-Inpainting or FLUX Fill) for reliable single-step restoration under well-localized, effect-aware masks.
- Occupancy-Guided Discriminator: To prevent blurry boundaries common in single-step generation, the authors introduce a multi-scale discriminator. Instead of using hard binary masks or Gaussian smoothing for patch targets, it computes fractional occupancy values via area pooling. This provides precise, fractional supervision for boundary patches, stabilizing the adversarial training of the one-step model.
- Latent Restoration: The model performs hard blending in the latent space, preserving unmasked regions exactly while restricting gradients to the edited region.
Phase II: Alpha-Aware Robust Removal
- Goal: Improve robustness to conservative or misaligned user masks that fail to cover soft effects (shadows/reflections).
- Lightweight Alpha Head: A small projection layer is appended to the backbone to predict a soft alpha map ( $\hat{\alpha}$ ) alongside the denoising output. This allows the model to infer the effective editing region beyond the provided mask.
- Incomplete-Mask Conditioning: Training uses "conservative" masks (e.g., tight object cores or geometrically perturbed masks) as input conditions, while the ground truth remains the full effect-aware mask.
- Adaptive Blending: Instead of hard latent blending, the model uses alpha compositing ( $z_{out} = \hat{\alpha} \odot \hat{z}_0 + (1 - \hat{\alpha}) \odot \bar{z}$ ) to seamlessly blend the generated content with the original image, effectively removing residual effects.

Key Contributions

OSOR Model: A single-step diffusion model that simultaneously achieves high efficiency, effect-aware removal (shadows/reflections), and robustness to imperfect masks.
Technical Innovations:
- An occupancy-guided discriminator using fractional occupancy targets for stable single-step training.
- A lightweight alpha head that leverages pretrained diffusion priors to predict appropriate removal regions with minimal overhead.
- SAVP, a pipeline for generating large-scale, verified effect-aware supervision data.
Resources:
- CORNE: A high-quality dataset of 280K verified removal pairs.
- Benchmarks: AnimeEraseBench and TextEraseBench for evaluating removal in complex domains.

Experimental Results

Experiments were conducted on six benchmarks (CORNE-Val, RORD-Val, AnimeEraseBench, TextEraseBench, OmniPaint-Bench, RemovalBench) comparing OSOR against strong multi-step diffusion baselines (e.g., ObjectClear, OmniEraser, FLUX Fill, SDXL-Inpainting).

Performance: OSOR surpasses multi-step baselines in perceptual quality (FID, LPIPS, PSNR, SSIM) while being significantly faster.
- On AnimeEraseBench, OSOR-FLUX achieved a 2.24 dB higher PSNR and was 27× faster than the second-best method.
- On CORNE-Val, OSOR-FLUX achieved the best scores across all metrics under both object-only and effect-aware mask settings.
Efficiency: OSOR processes a 1024 × 1024 image in under one second on a single NVIDIA A100 GPU, representing a 4× to 30× speedup over existing methods.
Qualitative Analysis: OSOR consistently removes both objects and associated effects (shadows, reflections) while preserving cleaner background structures, whereas other methods often leave residues, hallucinate content, or fail to handle imperfect masks.

Significance and Claims

The paper claims that simply applying pretrained diffusion models is insufficient for object removal; targeted training strategies are necessary to unlock their potential for this specific task. OSOR demonstrates that one-step diffusion is viable for complex editing tasks when combined with:

Precise boundary supervision (occupancy-guided discrimination).
Mechanisms to handle mask uncertainty (alpha heads and adaptive blending).
High-quality, effect-aware supervision derived from noisy data (SAVP).

By reducing inference time by an order of magnitude while maintaining or improving perceptual quality, OSOR bridges the gap between high-fidelity generative editing and the latency requirements of interactive and edge applications.