💻 computer science

Velocity-Space 3D Asset Editing

The paper introduces VS3D, a training-free and mask-free framework for local 3D asset editing that resolves identity leakage and signal interference in rectified-flow generators by implementing targeted velocity-space interventions—specifically Reconstruction-Anchored Source Injection, Partial-Mean Guidance, and Twin-Agreement Residual injection—directly within the ODE sampler.

Original authors: Hao Liu, Yuxuan Lin, Jingfeng Guo, Ruihang Chu, Junjie Wang, Ruotong Li, Yujiu Yang

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hao Liu, Yuxuan Lin, Jingfeng Guo, Ruihang Chu, Junjie Wang, Ruotong Li, Yujiu Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a digital 3D statue, like a clay model of a moose. You want to give it a new hat without changing its ears, nose, or the texture of its fur. This is the goal of local 3D editing: changing one specific part while keeping everything else exactly the same.

The paper introduces a new tool called VS3D that does this perfectly, without needing you to manually paint a "mask" (a digital stencil) to tell the computer what to touch and what to ignore. It also doesn't require retraining the AI.

Here is how the authors explain the problem and their solution, using simple analogies:

The Problem: The "Leaky Hose"

Current AI tools for 3D generation work like a hose spraying water (velocity) to build a shape. To edit the shape, they try to change the direction of the water in one spot.

However, the authors found three major problems with how existing tools try to do this:

The "Spillover" (Identity Leakage): When you try to push water in one direction to add a hat, the pressure leaks out and accidentally sprays the rest of the moose too. The moose's ears might wiggle or its fur might change color because the "edit signal" wasn't contained.
The "Weak Signal" (No Amplification): To stop the spillover, people usually turn down the water pressure. But if you turn it down too much, the hat you are trying to add becomes faint and weak. You can't have a strong hat and a quiet moose at the same time with current methods.
The "Global Pull" (Identity Drag): In the final steps of building the statue, the AI looks at a picture of the new moose (with the hat) and tries to make the whole statue look like that picture. This "pulls" the untouched parts (like the moose's legs) away from their original look, distorting them to match the new hat.

Existing tools try to fix this by using external "masks" (telling the AI "don't touch here") or by stitching pieces together after the fact. The authors say this is like trying to fix a leaky hose by putting tape on the outside, rather than fixing the pressure valve inside.

The Solution: VS3D (The "Internal Fix")

VS3D fixes the problem from the inside of the AI's engine (the "ODE sampler") by adjusting the water pressure and direction at three specific stages. They use three clever tricks:

1. RASI: The "Custom Anchor"

The Analogy: Imagine the AI is a boat drifting in the ocean. When you want to steer it to a new destination (the hat), the current usually pulls the whole boat off course.
The Fix: RASI acts like a custom anchor that is dropped specifically for this boat. It calibrates the engine so that when you steer toward the hat, the engine knows exactly how to cancel out the drift on the rest of the boat. It ensures the "water" only moves where you want it to, stopping the spillover without needing a mask.

2. PMG: The "Signal Booster"

The Analogy: Imagine you are trying to hear a whisper (the edit) in a noisy room (the random noise of the AI). If you just listen, you might miss the whisper or hear too much noise.
The Fix: PMG acts like a smart sound engineer. It listens to the whisper twice: once with a lot of background noise and once with less. By comparing the two, it can figure out exactly what the "whisper" (the hat) is and turn up the volume only on that specific sound. If the boat is already stable (no edit needed), the booster stays silent. This makes the edit strong without messing up the rest.

3. TAR: The "Double-Check"

The Analogy: After the boat is steered, the crew starts painting the details. They look at a photo of the new boat and accidentally paint the old parts wrong because they are too focused on the new hat.
The Fix: TAR runs a "twin" simulation in parallel. It asks the AI: "If we built this exact same shape but used the old photo, would the legs look different?"
- If the legs look the same in both simulations, the AI knows: "Ah, these legs don't need the new hat's influence." It then gently pulls the texture back to the original look.
- If the legs look different, it knows: "This is part of the edit," and leaves it alone.
  This ensures the fur and colors on the untouched parts stay true to the original.

The Result

By combining these three internal fixes, VS3D can take a 3D asset, add a hat, remove a tail, or change a color, and the rest of the object remains perfectly frozen in time.

No Masks: You don't need to draw a circle around the part you want to change.
No Retraining: It works with the AI exactly as it was originally built.
High Quality: The authors tested it on many objects and found it keeps the original look much better than other methods, even when doing complex changes like adding and removing things at the same time.

In short, VS3D is like a master sculptor who knows exactly how to chisel a new feature into a statue without accidentally chipping away the parts they wanted to keep, all by adjusting their internal technique rather than using external tape or glue.

Technical Summary: VS3D (Velocity-Space 3D Asset Editing)

1. Problem Statement

Precise local editing of 3D assets—modifying a specific region while strictly preserving the rest—is a fundamental challenge for native 3D generators. Existing approaches typically rely on mechanisms external to the generative model, such as manual 3D masks, post-hoc voxel merging, or 2D multi-view lifting. These methods fail to address the root cause of identity leakage: the behavior of the velocity field inside the Rectified-Flow Ordinary Differential Equation (ODE) sampler.

When using a frozen Rectified-Flow DiT (Diffusion Transformer) for editing, the velocity field must simultaneously carry a strong edit signal on the target region and vanish on preserved content. However, a single velocity field struggles to satisfy these conflicting requirements, leading to three specific failure modes:

Identity Leakage: Even when conditions agree locally, the velocity difference ( $v_\Delta$ ) remains non-zero on preserved regions due to Classifier-Free Guidance (CFG) asymmetry and attention-mediated residuals from global image embeddings.
Lack of Edit Amplification: Strengthening the edit signal often inadvertently perturbs the identity of preserved regions because both tasks compete for the same degrees of freedom within the velocity field. Furthermore, Monte-Carlo estimation of the velocity difference in 3D is dominated by variance rather than consistent signal.
Global Condition Override: In the geometry and material refinement stages, a single global image-feature token pulls every token toward the target appearance, overwriting source identity and causing texture drift or geometric distortion.

Current solutions outsource identity preservation to external tools rather than intervening directly within the ODE sampler where the corruption originates.

2. Methodology: VS3D

The authors propose VS3D, an inversion-free, training-free, and mask-free framework that addresses these issues through targeted interventions inside the ODE sampler. The method operates on a native 3D generator (specifically TRELLIS 2.0) which unfolds in three stages: sparse-structure generation, geometry generation, and material generation.

VS3D integrates three complementary modules corresponding to specific stages of the pipeline:

A. Reconstruction-Anchored Source Injection (RASI)

Target: Stage 1 (Dense Occupancy Latent)
RASI addresses the two-channel identity leakage (CFG asymmetry and global condition residuals).

Mechanism: Instead of using a fixed unconditional embedding ( $\phi_0$ ), RASI optimizes a per-step, asset-specific unconditional embedding ( $\phi_t$ ) via a source-reconstruction probe.
Process: At each timestep, the method solves an optimization problem to find a $\phi_t$ that, when used in the velocity calculation, forces the Euler step to reconstruct the source latent ( $x_{src}$ ) exactly when the target condition is replaced with the source condition.
Effect: This "anchors" the source trajectory. During actual editing, the cached $\phi_t$ absorbs the leakage channels, causing $v_\Delta$ to collapse toward zero on non-edited voxels without requiring an external mask.

B. Partial-Mean Guidance (PMG)

Target: Stage 1 (Edit Signal Amplification)
PMG resolves the conflict between preserving identity and amplifying the edit signal.

Mechanism: It exploits the Monte-Carlo structure of the velocity difference estimation. The method computes a high-quality mean estimate ( $\hat{\mu}_S$ ) and a noisier partial-sample mean ( $\hat{\mu}_L$ ) of the velocity difference.
Process: It extrapolates away from the noisier estimate: $\hat{\mu}_{final} = \hat{\mu}_S + w(\hat{\mu}_S - \hat{\mu}_L)$ .
Effect: This amplification is automatically gated by the velocity field itself. On non-edited regions where RASI has driven the signal to zero, the extrapolation is negligible. On edited regions where a consistent signal exists, the amplification strengthens the edit direction without requiring additional forward passes or masks.

C. Twin-Agreement Residual Injection (TAR)

Target: Stages 2 & 3 (Sparse Geometry and Material)
TAR prevents the global condition from overwriting source identity in the sparse refinement stages, where direct ODE coupling is impossible due to mismatched voxel supports.

Mechanism: The sampler runs a "twin" forward pass on the same coordinate scaffold and noise, but conditioned on the source image ( $c_{src}$ ) instead of the target.
Process: It computes a per-token disagreement map ( $d_i$ ) between the target output ( $z_{tgt}$ ) and the twin source output ( $z_{twin}$ ). Tokens with low disagreement (high agreement) are identified as non-edited.
Effect: For agreeing tokens, the method injects a residual from the source encoding ( $z_{src}$ ) back into the target latent, effectively pulling the geometry and material back toward the source. Disagreeing tokens (the edit region) remain untouched. This allows the sampler to decide token-by-token what to preserve.

3. Key Contributions

RASI: A method to eliminate two-channel identity leakage (CFG asymmetry and attention residuals) by optimizing a per-step unconditional embedding, ensuring $v_\Delta$ vanishes on non-edited regions without external masks.
PMG: A technique to decouple edit amplification from identity preservation by extrapolating between subsample means of the velocity difference, automatically gating amplification to regions with consistent edit signals.
TAR: A mechanism to suppress global identity drag in sparse stages by using a twin-forward disagreement map to inject source residuals only where the sampler agrees that content should be preserved.
VS3D Framework: A unified, purely velocity-space editing paradigm that achieves high-fidelity local edits with strong identity preservation without inversion, training, or manual masking.

4. Experimental Results

The authors evaluated VS3D on a benchmark derived from Objaverse, comparing it against state-of-the-art native-3D editors (TRELLIS 2.0, VoxHammer, Nano3D) and 2D-lifted editors (Instant3dit, Edit360, VecSet-Edit).

Quantitative Performance: VS3D achieved the best results across most metrics, including the highest PSNR (22.51), SSIM (0.867), and lowest LPIPS (0.145), indicating superior identity preservation. It also achieved the highest DINO-I (0.841) and CLIP-T (0.294) scores, demonstrating high edit fidelity.
Qualitative Performance: The method successfully handled elementary operations (add/remove) and compound edits (simultaneous add/remove) where baselines failed. It maintained structural coherence and texture consistency in non-edited regions, avoiding the seam artifacts and topology inconsistencies common in 2D-lifted approaches.
Ablation Studies: The study confirmed that each module addresses a distinct failure mode: RASI corrected occupancy drift, PMG restored edit strength, and TAR recovered fine-grained material and texture details in later stages.

5. Significance and Claims

The paper claims that VS3D demonstrates that a purely velocity-space approach can serve as a general-purpose editing paradigm for pretrained 3D generators. By intervening directly inside the ODE sampler, the method achieves high-fidelity local editing without the need for:

External 3D masks (which are tedious and brittle).
Model inversion (which is computationally expensive and unstable).
Additional training or fine-tuning (LoRA).

The authors acknowledge a limitation: because no hard mask isolates the edit boundary, pixel-level texture consistency in non-edit regions is "close to, rather than identical to" the original. However, they argue this is an inherent characteristic of mask-free methods and that the perceptual difference is minor and acceptable for practical scenarios. The work suggests that controlling the velocity field from within is the most direct path to simultaneous identity preservation and edit amplification, offering a new perspective for future mask-free 3D editing research.