💻 computer science

RoPEMover: Depth-Aware Object Relocation via Positional Embeddings

RoPEMover introduces a depth-aware object relocation method that manipulates rotary positional embeddings within diffusion transformers to achieve geometry-consistent spatial rearrangement, including realistic occlusion handling and shadow updates, with state-of-the-art performance despite minimal real-world supervision.

Original authors: Ipek Oztas, Duygu Ceylan, Aybars Bugra Aksoy, Aysegul Dundar

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ipek Oztas, Duygu Ceylan, Aybars Bugra Aksoy, Aysegul Dundar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a photograph of a living room with a coffee table in the middle. You want to move that table to the corner of the room. If you just cut the table out of the picture and paste it into the corner, it looks fake. It might be floating in mid-air, it won't cast a shadow on the new floor, and it won't look like it's sitting behind a lamp that was previously in the corner.

RoPEMover is a new tool that solves this problem. Instead of just cutting and pasting pixels, it acts like a "digital architect" that understands the 3D shape of the room, even though it's only looking at a flat 2D picture.

Here is how it works, broken down into simple concepts:

1. The "GPS" Inside the Brain (RoPE)

Modern AI image generators (like the ones that make art from text) have a built-in "GPS" system inside their brain. In technical terms, this is called Rotary Positional Embeddings (RoPE). Think of this GPS as a map that tells the AI exactly where every single pixel is located relative to everything else.

Usually, this map is fixed. But the authors of this paper realized: What if we could physically move the coordinates on this map?

If you tell the AI, "Move the coffee table 2 feet to the left," RoPEMover doesn't just drag the pixels. It warps the GPS coordinates of the table. It tells the AI's brain, "Pretend the table is now in this new spot." Because the AI is following its own internal map, it automatically redraws the table in the new spot, making it look like it was always there.

2. The "Depth Goggles" (Depth-Awareness)

The tricky part of moving things is knowing what is in front and what is behind. If you move a chair in front of a painting, the chair should block the painting. If you move it behind, the painting should cover the chair.

Old methods often get this wrong, making objects look like they are floating or ignoring other items. RoPEMover puts on a pair of "Depth Goggles."

It looks at the original photo and guesses how deep every part of the scene is (how far away it is from the camera).
When you move an object, it calculates the new depth.
It then tells the AI: "This object is now closer than the wall, so draw the wall behind it," or "This object is further away, so draw the wall in front of it."

This allows the AI to handle occlusions (hiding things) perfectly. It can even "invent" the parts of the background that were previously hidden by the object, filling them in realistically.

3. The "Shadow and Light" Magic

When you move a physical object, the shadows move with it. If you move a lamp, the light it casts changes.

RoPEMover doesn't just move the object; it moves the relationship the object has with the light. Because it understands the 3D geometry, it knows where the light is coming from and redraws the shadows and reflections in the new location. It ensures the object looks "grounded" and part of the scene, not like a sticker.

4. How It Learned (The Training)

You might think an AI needs to watch millions of hours of video to learn how to move things. Surprisingly, RoPEMover learned with very little data.

The Synthetic Playground: First, the researchers taught it using simple, computer-generated blocks (like a digital Lego set). This taught the AI the rules of moving things (how shadows work, how depth changes).
The Real-World Polish: Then, they showed it a tiny number of real photos (about 165 pairs) where objects were actually moved. This was just enough to teach the AI how to handle the messy, complex details of real life, like textures and specific lighting.

What Can It Do?

According to the paper, this method allows for:

Moving Objects: Dragging an object to a new spot while keeping it looking real.
Removing Objects: Taking an object away and filling in the background perfectly (including revealing what was behind it).
Adding Objects: Pasting a new object into a scene so it casts the right shadow and fits the depth.
Fixing Depth: Placing an object behind a glass vase or in front of a tree, handling the complex layering correctly.

The Bottom Line

RoPEMover is like giving the AI a "3D understanding" of a flat photo. Instead of treating the image as a flat sheet of paper to be cut and pasted, it treats the image as a 3D world where objects have depth, shadows, and relationships with their surroundings. By tweaking the AI's internal "GPS" and "Depth Goggles," it can move things around with a level of realism that previous tools struggled to achieve.

Technical Summary: RoPEMover

Problem Statement

Moving an object within a single image is a complex image editing task that requires more than simple spatial displacement. A successful edit must preserve the object's identity, adapt to perspective and scale changes, plausibly complete previously occluded regions, and maintain coherent scene-dependent effects such as shadows, reflections, and illumination. Existing approaches often fail to achieve this level of consistency. Two-step methods (removal followed by insertion) frequently struggle with shadow handling and object identity preservation. Copy-and-paste techniques with harmonization often fail to account for perspective changes, resulting in physically inconsistent edits (e.g., floating objects). While recent generative approaches leverage large-scale synthetic data or video diffusion models to learn motion priors, they often require expensive data construction, complex pipelines, or temporal modeling that is unnecessary for single-image editing.

Methodology

RoPEMover introduces a geometry-aware object motion method that operates directly on the internal representations of diffusion transformers, specifically manipulating Rotary Positional Embeddings (RoPE). The approach avoids the need for explicit 3D reconstruction or large-scale motion datasets by leveraging the strong spatial priors already encoded in pretrained models.

Core Mechanism: RoPE Manipulation

The method is built on the insight that RoPE defines a structured spatial field within the attention layers of diffusion transformers. By warping these positional embeddings, the model can be induced to synthesize an object at a new location with controlled motion.

Drag Signal Encoding: The user provides a drag signal defined as a triplet $(dx, dy, s)$, representing pixel-space displacement and a scale ratio. These parameters are converted into token-grid offsets and a scaling factor.
RoPE Warping: For tokens within the source object mask, the RoPE encodings are replaced with those computed for the target position. Specifically, the row and column indices $(r, c)$ are transformed to $(r', c')$ based on the displacement and scale. This warping is applied only during the initial denoising steps (e.g., the first 20 steps) to establish the structural layout early in the reverse diffusion process.

Depth-Aware 3D RoPE

To handle occlusions and depth ordering (placing objects in front of or behind scene elements), the method extends 2D RoPE into a depth-aware formulation:

Repurposing the Temporal Axis: In factorized 3D RoPE (used in models like Qwen-Image), the temporal axis is typically unused for single-image targets. RoPEMover repurposes this axis to inject metric depth information.
Depth Estimation and Synthesis: A metric depth map is estimated for the source image. At inference, a target depth map is synthesized by:
- Inpainting the background depth (solving the Laplace equation) to fill the region vacated by the object.
- Warping the object's depth map according to the drag signal and applying a user-specified depth offset ( $\Delta z$ ).
- Compositing the warped object depth onto the background using a Z-buffer test to determine occlusion ordering.
- Blending depth discontinuities at the mask boundary.
Injection: The synthesized depth values are normalized and injected into the first complex component of the temporal axis of the RoPE frequency tensor. This provides the model with explicit 3D geometric context, enabling consistent handling of occlusions and scene-dependent effects.

Training Strategy

The model is trained using a data-efficient, two-stage curriculum:

Stage 1 (Synthetic Data): The model is trained on a large dataset of synthetic scenes generated from CLEVR. These scenes provide pixel-perfect ground truth for object motion, disocclusion, and depth, allowing the model to learn precise and controllable edit behavior without realistic rendering overhead.
Stage 2 (Real Data Fine-tuning): A lightweight fine-tuning stage is performed on a small dataset of 165 real-world paired photographs. This step bridges the domain gap, improving visual fidelity, identity preservation, and the handling of complex real-world lighting and materials.

Key Contributions

The paper outlines four primary contributions:

Geometry-Aware Object Motion via Representation Manipulation: A novel method enabling object movement by directly manipulating RoPE within diffusion transformers, providing explicit spatial control without requiring 3D reconstruction or video modeling.
Depth-Guided Spatial Transformation: An extension of RoPE manipulation with depth-aware conditioning. This enables geometry-consistent displacement, correct occlusion handling (placing objects behind or in front of other elements), and the generation of coherent content in newly revealed regions.
Data-Efficient Learning: The approach learns effective motion behavior from simple synthetic data and a minimal set of real images, avoiding the need for massive motion datasets or complex video-based training pipelines.
State-of-the-Art Performance: The method achieves superior results in object motion tasks, producing high-fidelity edits with physically consistent propagation of shadows and illumination, outperforming prior approaches in both visual quality and geometric consistency.

Results and Evaluation

The method was evaluated on the ObjMove-A and ObjMove-B benchmarks.

Quantitative Performance: On ObjMove-A, RoPEMover achieved state-of-the-art results across all metrics, including CLIP Score, DINO Score, DreamSim, and PSNR. It significantly outperformed baselines such as ObjectMover, FreeFine, MagicFixup, and GeoDiffuser, particularly in preserving object identity (high DINO/CLIP scores) and background consistency.
Qualitative Performance: Visual comparisons demonstrated that RoPEMover correctly handles challenging scenarios, such as moving objects behind transparent surfaces, relocating small structured objects, and maintaining consistent shadows and reflections. Unlike two-step methods that often produce "floating" objects or copy-paste artifacts, RoPEMover generates edits that are physically grounded and scene-aware.
Ablation Studies: Experiments confirmed that both RoPE warping and depth conditioning are critical. Removing warping resulted in poor spatial control, while removing depth conditioning led to inaccuracies in depth ordering and occlusion. Furthermore, the second stage of training on real data was shown to be essential for improving identity preservation and visual fidelity compared to training on synthetic data alone.
User Study: In a study with 20 participants comparing against three strong baselines, RoPEMover was preferred in 83.4% of votes across metrics of edit faithfulness, realism, and background consistency.

Significance

RoPEMover demonstrates that precise, geometry-aware object relocation can be achieved by directly manipulating the internal positional representations of diffusion transformers. By interpreting RoPE as a manipulable spatial field and augmenting it with depth information, the method enables controlled object movement that respects scene geometry, occlusions, and lighting. This approach offers a more efficient and effective alternative to methods relying on video priors or large-scale motion datasets, providing a robust solution for content creation, visual effects, and interactive design where physical consistency is paramount. The authors note that while the method improves controllability, it inherits the computational costs of diffusion sampling and the potential biases of the underlying generative models.