🤖 AI

Reproducing DragDiffusion: Interactive Point-Based Editing with Diffusion Models

This paper presents a reproducibility study of DragDiffusion using the DragBench benchmark, confirming the method's core claims regarding interactive point-based editing while identifying critical sensitivity to specific hyperparameters like the optimized timestep and feature level, and demonstrating that multi-timestep optimization offers no accuracy gains despite higher computational costs.

Original authors: Ali Subhan, Ashir Raza

Published 2026-02-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Subhan, Ashir Raza

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a digital photo of a cat sitting on a windowsill. You want to move the cat so it's looking out the window instead of at the camera. In the past, doing this required complex Photoshop skills or hoping a magic AI prompt would guess exactly what you wanted.

DragDiffusion is a new "magic wand" that lets you do this by simply clicking on the cat's nose and dragging it to the window. It's like telling the AI, "Move this point here," and the image magically rearranges itself to make it happen.

This paper is a reproducibility study. Think of it as a group of chefs (the researchers) trying to recreate a famous, award-winning recipe (DragDiffusion) that was published by another chef. They want to know: "Is this recipe actually as good as they say? Does it work if we follow the instructions exactly? Are there any hidden tricks we missed?"

Here is the breakdown of their findings using simple analogies:

1. The Core Recipe: How DragDiffusion Works

The original method is like a time-traveling sculptor.

The Process: The AI takes your photo and turns it into "noise" (static), then slowly rebuilds it.
The Trick: Instead of rebuilding the whole photo from scratch, the AI pauses at one specific moment in the middle of the process (the "intermediate timestep"). At this exact moment, it listens to your instruction ("Drag the nose here") and tweaks the image just enough to make it happen, then finishes the job.
The Safety Net: To make sure the cat still looks like your cat and not a different animal, the AI uses a special "identity guard" (LoRA) that remembers the original features.

2. The Taste Test: What the Researchers Found

The researchers tried to recreate the original results and found that the recipe works, but it's very sensitive to the temperature.

A. Timing is Everything (The "Timestep" Analogy)

Imagine you are trying to mold clay.

Too Early (Wet Clay): If you try to move the cat when the clay is too wet (early in the process), the shape is too fluid. You drag the nose, but the whole head melts. The result is blurry and inaccurate.
Too Late (Hardened Clay): If you wait until the clay is bone dry (late in the process), you can't move the nose at all without cracking the whole statue.
Just Right (Leather-hard): The researchers confirmed that the original authors were right: you must stop at that perfect "leather-hard" moment in the middle. This is the sweet spot where the cat is solid enough to hold its shape but soft enough to be moved.

B. The "Identity Guard" (LoRA)

Without the "identity guard" (LoRA), the AI gets confused. If you drag the cat's nose, the AI might accidentally turn the cat into a dog or a blob.

The Finding: The researchers proved that this guard is essential. It's like a bouncer at a club; without it, the wrong people (wrong features) get in, and the party (the image) falls apart. With the guard, the cat stays a cat, just in a new pose.

C. The "Mask" (Regularization)

Imagine you are painting a wall, but you only want to paint the window frame, not the whole room.

The Finding: The AI needs a "mask" (a stencil) to know exactly where you want to drag the point. If you don't use a strong enough mask, the AI gets excited and starts dragging the background, the sky, and the floor along with the cat. The researchers found that a "medium-strength" mask works best—it keeps the background still while letting the cat move.

D. The "Eyes" of the AI (Feature Supervision)

The AI has to look at the image to know where to move things. It has different "layers of vision":

Superficial Vision: Sees only colors and textures (too blurry to know where a nose is).
Deep Vision: Sees the whole picture but misses the tiny details (too vague).
Mid-Level Vision: This is the Goldilocks zone. The researchers confirmed that the AI needs to look at the "middle layer" of its brain to get the geometry right. If it looks too deep or too shallow, the cat ends up in the wrong spot.

3. The "What If" Experiment: Doing More Work

The researchers asked: "What if, instead of pausing once in the middle, we pause three times and adjust the cat at every single step?"

The Result: It was like trying to steer a car by checking the map three times for every inch you drive. It didn't make the drive any smoother or more accurate. In fact, it just took 2.7 times longer to compute.
The Lesson: The original "single pause" strategy was the smartest choice. It's efficient and effective.

The Final Verdict

The researchers concluded that DragDiffusion is real and works. It's not a fluke.

What's Easy: The code is well-written, and the general steps are easy to follow.
What's Tricky: You have to be very precise with your settings. If you pick the wrong "time" to stop the process, or if you forget the "identity guard," the results will be messy.

In short: DragDiffusion is a powerful tool for moving things in photos, but like a high-performance race car, it needs to be tuned exactly right to win the race. The researchers successfully drove the car and confirmed it goes fast, provided you don't mess with the engine settings.

1. Problem Statement

Interactive image editing requires precise geometric control, which text-based prompts often fail to provide. DragDiffusion (Shi et al., 2024) was proposed as a solution, allowing users to manipulate images by dragging handle points to target locations. The method relies on optimizing a single diffusion latent at an intermediate timestep, guided by motion supervision on UNet features, identity-preserving fine-tuning (LoRA), and spatial regularization.

While the original paper claimed high efficacy, the complex interplay of diffusion inversion, latent optimization, and feature-level supervision introduces significant sensitivity to hyperparameters. This study aims to:

Independently verify the empirical claims of DragDiffusion.
Assess the robustness of the method against implementation details and hyperparameter choices.
Determine if the single-timestep optimization strategy is strictly necessary or if multi-timestep optimization offers benefits.

2. Methodology

The study strictly adhered to the authors' released implementation and the DragBench benchmark, using Stable Diffusion v1.5 as the backbone.

Core Pipeline:

DDIM Inversion: The input image is inverted to obtain a latent representation.
Latent Optimization: A single latent at a specific timestep $t$ is iteratively optimized. The objective minimizes a motion supervision loss (matching UNet feature activations between handle and target points) and a spatial mask regularization term (to prevent global distortions).
Identity Preservation: Low-Rank Adaptation (LoRA) is applied to the UNet attention layers to fine-tune the model, preventing identity drift during editing.
Generation: The optimized latent is used in a guided DDIM denoising process to generate the final image.

Experimental Design:
The authors conducted controlled ablation studies on five key claims and introduced one extension:

Timestep Selection: Varying the optimization timestep ( $t \in \{20, 35, 50\}$ ).
LoRA Necessity: Comparing with and without LoRA fine-tuning.
LoRA Duration: Varying training steps ($0, 20, 40, 80, 100, 120$).
Mask Regularization: Varying the regularization weight $\lambda$ ($0.0, 0.1, 0.5, 1.0$).
Feature Supervision: Testing different UNet decoder blocks ($1$ to $4$) for motion supervision.
Extension (Multi-Timestep): A novel experiment optimizing latents at multiple timesteps ( $\{30, 35, 40\}$ ) simultaneously to test if single-timestep optimization is a sufficient approximation.

Evaluation Metrics:

Mean Distance (MD): Euclidean distance between target points and final semantic locations (lower is better).
Image Fidelity (IF): $1 - \text{LPIPS}$ , measuring preservation of original appearance (higher is better).

3. Key Results

The study successfully reproduced the qualitative and quantitative trends of the original paper, validating the core design choices while highlighting specific sensitivities.

Claim 1: Timestep Selection (Validated)
- Result: Intermediate timesteps ( $t=35$ ) yielded the best trade-off.
- Data: $t=35$ achieved the lowest MD (34.90). Early timesteps ( $t=20$ ) preserved fidelity but lacked spatial accuracy; late timesteps ( $t=50$ ) degraded both metrics due to high noise.
Claim 2: LoRA Necessity (Validated)
- Result: LoRA is critical. Without it, spatial accuracy dropped significantly (MD increased from 34.90 to 55.68), and identity preservation failed.
Claim 3: LoRA Duration (Validated)
- Result: Performance improved rapidly with initial steps, peaking at 100 steps (MD 34.55). Beyond 80–100 steps, returns diminished, and performance slightly degraded at 120 steps.
Claim 4: Mask Regularization (Validated)
- Result: Moderate regularization ( $\lambda=0.1$ ) was optimal. $\lambda=0$ caused uncontrolled edits; $\lambda \ge 0.5$ improved fidelity but reduced spatial precision.
Claim 5: Feature Supervision (Validated)
- Result: Mid-level features (Decoder Block 3) provided the best balance. Block 1 was too coarse (poor spatial accuracy), while Block 4 was too texture-focused (poor structural motion).
Extension: Multi-Timestep Optimization
- Result: Jointly optimizing multiple timesteps did not improve spatial accuracy (MD increased slightly to 36.28) and increased computational cost by 2.7x. This confirms that neighboring timesteps provide redundant information, making the single-timestep strategy sufficient and efficient.

4. Key Contributions

Validation of Core Claims: The study confirms that DragDiffusion's design—specifically intermediate-timestep optimization, LoRA-based identity preservation, and mid-level feature supervision—is robust and reproducible.
Sensitivity Analysis: It identifies that the method is highly sensitive to specific hyperparameters:
- Optimized timestep.
- Feature level for motion supervision.
- Mask regularization strength.
- Conversely, LoRA duration and mask strength have broader operating ranges once reasonable defaults are set.
Efficiency Verification: By disproving the utility of multi-timestep optimization, the study reinforces the computational efficiency of the original single-timestep approach.
Reproducibility Benchmark: The authors provide a clean, verified codebase and highlight implementation pitfalls (e.g., library versioning, seed control, and numerical precision) that are critical for future researchers.

5. Significance

This work serves as a crucial "sanity check" for the interactive image editing community. It demonstrates that DragDiffusion is not a "black box" with fragile results but a well-founded method whose performance relies on specific, identifiable design choices.

For Practitioners: It provides a clear "recipe" for reliable reproduction, emphasizing that small deviations in timestep or feature selection can lead to significant failure.
For Researchers: It closes the loop on the single-timestep hypothesis, suggesting that future work should focus on other architectural improvements rather than extending optimization across timesteps.
For the Field: It sets a standard for reproducibility in diffusion-based editing, showing that while complex pipelines are sensitive, their core mechanisms can be systematically validated and understood.