OSInsert: Towards High-authenticity and High-fidelity Image Composition

This paper proposes OSInsert, a two-stage framework that combines high-authenticity pose adjustment with high-fidelity detail preservation to overcome the limitations of existing methods in generating realistic image compositions.

The Gist

Imagine you are a digital chef trying to add a delicious, complex dish (like a steaming bowl of ramen with intricate toppings) to a photo of a cozy dining table.

The Problem:
Until now, digital chefs had to choose between two bad options:

1. The "Realistic but Ruined" Chef (High-Authenticity): This chef is great at making the bowl look like it belongs on the table. They adjust the angle so the bowl sits flat, add the right shadows, and make the lighting match the room. But, in doing so, they accidentally turn the beautiful, detailed ramen into a blurry, colorless blob. The details are lost.
2. The "Detailed but Floating" Chef (High-Fidelity): This chef is a perfectionist. They take the exact photo of the ramen and paste it onto the table. Every noodle and drop of sauce is perfect. But, they didn't adjust the angle. The bowl looks like it's floating in mid-air or sticking out of the table at a weird angle. It looks fake, like a sticker.

For years, technology couldn't do both at the same time. You had to choose between a realistic-looking but blurry object, or a sharp object that looked like it didn't belong.

The Solution: OSInsert (The "Two-Step" Master Chef)
The authors of this paper, Jingyuan Wang and Li Niu, came up with a clever two-step strategy called OSInsert. Instead of trying to do everything in one messy step, they split the job into two specialized tasks.

Think of it like building a custom suit:

Step 1: The Tailor (Getting the Shape Right)

First, they use a tool called ObjectStitch. Imagine a master tailor who doesn't care about the fabric pattern yet; they only care about the shape.

• They take the background (the table) and the object (the ramen).
• They cut out the space where the ramen should go.
• They use AI to "grow" a new ramen bowl that fits perfectly into that space. It leans the right way, casts the right shadow, and matches the room's lighting.
• The Catch: This new ramen bowl looks a bit like a blurry sketch. It has the right shape and pose, but it's missing the fine details.

Step 2: The Embroiderer (Adding the Details)

Now, they bring in a second tool called InsertAnything. Imagine a master embroiderer who is obsessed with details.

• They look at the "blurry sketch" from Step 1.
• They use a high-tech scanner (called SAM, the "Segment Anything Model") to trace the exact outline of the blurry bowl. This is crucial—it tells the embroiderer exactly where the bowl is and where the table is, so they don't accidentally paint the table!
• The embroiderer then takes the original, perfect photo of the ramen and carefully "paints" those sharp, crisp details only inside the outline of the blurry bowl.
• The Result: The bowl keeps the perfect pose and lighting from Step 1, but now it has the sharp, beautiful details from the original photo.

Why This is a Big Deal

Before this, trying to do both steps at once was like asking one person to be a sculptor and a painter simultaneously. They would get confused, and the result would be a compromise that wasn't great at either.

OSInsert separates the jobs:

1. Sculptor (Step 1): "Make it fit the room."
2. Painter (Step 2): "Make it look like the real thing."

By using a "bridge" (the precise mask from the SAM tool) to pass the work from the sculptor to the painter, the final image looks incredibly real. The object fits the scene perfectly, but it also looks exactly like the original object you wanted to insert.

In a Nutshell:
OSInsert is like hiring two specialists instead of one generalist. One specialist makes sure the object fits the room's geometry, and the second specialist ensures the object looks exactly like the original. The result is a photo that looks so real, you'd swear the object was actually there.

Technical Summary

1. Problem Statement

Generative image composition (object insertion) aims to seamlessly integrate a foreground object into a background image. Current methods face a fundamental trade-off between two conflicting objectives:

• High Authenticity: Ensuring the foreground matches the background in terms of pose, viewpoint, lighting, and perspective. Existing high-authenticity methods (e.g., ObjectStitch) achieve this but often sacrifice fine-grained details, causing texture blurring, color distortion, and loss of semantic identity.
• High Fidelity: Preserving the original appearance, texture, and details of the foreground object. Existing high-fidelity methods (e.g., InsertAnything) excel at detail retention but lack the ability to adapt the foreground's pose or viewpoint, resulting in an obvious "copy-and-paste" effect when the reference pose differs from the background context.

The core challenge is that single-stage generative models struggle to simultaneously optimize for spatial compatibility (authenticity) and detail preservation (fidelity) without compromising one for the other.

2. Methodology: OSInsert Framework

The authors propose OSInsert, a two-stage decoupled framework that separates the optimization of authenticity and fidelity into sequential, independent stages. This design leverages the strengths of existing state-of-the-art models without modifying their internal structures.

Stage 1: Authenticity Generation

• Goal: Generate a foreground object that is spatially and visually compatible with the background (correct pose, viewpoint, scale, and lighting).
• Model: Uses ObjectStitch (a diffusion-based generative inpainting model).
• Process:
  1. A masked background image is created by erasing the target region (defined by a bounding box) in the original background.
  2. ObjectStitch takes the masked background, the bounding box, and the foreground reference image as input.
  3. It generates an intermediate composite image where the foreground is perfectly aligned with the background's geometry and lighting.
• Trade-off: While the pose is correct, the generated foreground suffers from detail loss (blurred textures, distorted colors).

Stage 2: Fidelity Filling

• Goal: Restore the fine-grained visual details of the original reference foreground while strictly adhering to the shape and pose generated in Stage 1.
• Bridge Component: The Segment Anything Model (SAM) is used to extract a high-precision pixel-level mask from the intermediate composite image generated in Stage 1. This mask precisely defines the foreground contour, excluding background pixels within the bounding box.
• Model: Uses InsertAnything (a high-fidelity in-context editing method based on Diffusion Transformers).
• Process:
  1. A new masked background is created using the original background image and the precise SAM mask.
  2. InsertAnything takes this masked background, the SAM mask, and the original high-quality reference image as input.
  3. It fills the masked region with the original details (color, texture, shape) from the reference image, constrained by the contour defined in Stage 1.
• Outcome: The final image retains the realistic pose of Stage 1 and the high-fidelity details of the reference image.

3. Key Contributions

1. Decoupled Two-Stage Strategy: The paper introduces a novel paradigm that breaks the authenticity-fidelity trade-off by separating the tasks into two sequential stages rather than attempting to solve them in a single monolithic model.
2. Modular Integration: The framework effectively combines existing SOTA models (ObjectStitch for authenticity and InsertAnything for fidelity) and the Segment Anything Model (SAM) for precise mask extraction, creating a scalable solution without retraining base models.
3. Precision Masking: The use of SAM to generate a pixel-level mask from the intermediate result is a critical innovation. It ensures the second stage only modifies the foreground region, preventing "bleeding" of details into the background and preserving the integrity of the original scene.
4. Open Source: The authors have released the code and models to facilitate further research.

4. Experimental Results

• Dataset: Experiments were conducted on the MureCOM dataset, which features diverse scenes, complex foreground objects (e.g., patterned ceramics, textured metals), and significant pose/viewpoint differences between reference and target.
• Baselines: Compared against:
  • Academic: ObjectStitch (High-authenticity), InsertAnything (High-fidelity).
  • Commercial: Banana pro and Seedream 5.0.
• Findings:
  • vs. Academic Baselines: OSInsert outperforms both single-stage baselines. It avoids the detail loss of ObjectStitch and the copy-and-paste artifacts of InsertAnything.
  • vs. Commercial Models: While commercial models show good general performance, they often fail in spatial precision (objects slightly offset from the bounding box, incorrect scaling) and background integrity (altering background lighting or color tones). OSInsert strictly adheres to the bounding box constraints and preserves the original background perfectly.
  • Metrics: OSInsert achieves significant improvements in both authenticity (spatial alignment) and fidelity (detail preservation) metrics.

5. Significance

• Solving a Core Bottleneck: OSInsert addresses the primary limitation of current generative composition technology, enabling the creation of images that are both geometrically realistic and semantically accurate.
• Practical Applicability: The method is highly relevant for industries requiring precise object insertion, such as e-commerce (product display in various scenes), film/TV post-production (VFX integration), and augmented reality, where both realism and brand identity (detail preservation) are critical.
• Research Direction: It establishes a new baseline for generative image composition, suggesting that modular, multi-stage pipelines may be more effective than single-stage optimization for complex, multi-objective generative tasks.