💻 computer science

EditFlow3D: Automated Local Editing of 3D Assets with Trajectory Preservation

EditFlow3D is a training-free framework that enables precise, automated local editing of 3D assets by leveraging a VLM-driven workflow to generate masks and visual guidance, while employing mask-guided differential flow and step-wise trajectory preservation to edit target regions without compromising the structure and appearance of non-target areas.

Original authors: Rui Nie, Chuang Wang, Haitao Zhou, Jiahe Song, Buyu Li, Sheng Wang, Qian Yu

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rui Nie, Chuang Wang, Haitao Zhou, Jiahe Song, Buyu Li, Sheng Wang, Qian Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical clay sculpture that you can reshape just by talking to it. This is the dream behind 3D generative models, a branch of artificial intelligence that builds three-dimensional objects from text descriptions or pictures. Think of these models as incredibly skilled, but slightly clumsy, digital sculptors. They are great at making a whole new statue from scratch, but if you ask them to just "change the hat" on an existing statue without messing up the face or the arms, they often struggle. They might melt the whole thing, or accidentally paint the statue's nose blue while trying to fix the hat.

The core challenge here is local editing: making precise changes to a specific part of a 3D object while keeping everything else exactly the same. It's like trying to swap the engine of a car without taking the whole car apart or accidentally changing the color of the tires. To do this well, a computer needs two things: a perfect map of where to change things (a mask) and a way to make the change without the "magic" of the new instruction accidentally leaking into the parts that shouldn't change. Until now, getting that perfect map automatically has been tricky, and keeping the rest of the object stable has been even harder.

Enter EditFlow3D, a new method that acts like a super-precise, training-free digital sculptor. Instead of needing to relearn how to sculpt every time (which is what "training" usually means), it uses a smart assistant to figure out exactly what you want and where to do it. The system works in two main stages. First, a "Vision-Language Model" (a type of AI that understands both pictures and words) looks at your 3D object and your instruction (like "replace the red cape with black bat wings"). It picks the best angle to look at the object, draws a picture of what the result should look like, and—crucially—draws a precise 3D map (a mask) of exactly which pixels belong to the cape and which belong to the rest of the statue.

Once the map is ready, the real magic happens. The system uses a technique called Differential Flow Guidance. Imagine the 3D object is a river flowing toward a final shape. The AI calculates the difference between how the river would flow if it were just the original statue versus how it would flow if it were the statue with the new wings. It then pushes the water only in the direction of the wings, but only inside the area marked by the cape mask. This ensures the change happens exactly where you want it.

However, there's a catch: even if you push the water only in the cape area, the ripples can sometimes spread and shake the rest of the statue. To stop this, the authors introduced Trajectory Preservation Guidance. Think of this as a gentle, invisible hand that holds the rest of the statue steady. As the AI builds the new wings step-by-step, this "hand" constantly checks the parts of the statue that shouldn't change (like the face or the legs) and gently nudges them back to their original shape if they start to drift. Unlike older methods that might just freeze those parts in place (which can cause jagged, broken edges), this method softly guides them, keeping the transition smooth and the whole object looking natural.

The researchers tested this new method on a wide variety of tasks, from adding smoke to a train's chimney to replacing a chicken with a panda. They compared it against other top methods and found that EditFlow3D was much better at hitting the target area without ruining the rest of the model. In fact, in a user study with 40 people, nearly half of the participants preferred EditFlow3D's results over all other methods, praising its ability to follow instructions accurately while keeping the original object's look intact. The paper suggests that by combining a smart "map maker" with a "steady hand" that guides the changes, we can finally edit 3D objects with the precision of a surgeon and the ease of a conversation.

Technical Summary: EditFlow3D

Problem Statement

Controllable local editing of 3D assets requires precise target localization and appropriate visual guidance. Existing methods face two primary challenges:

Mask Acquisition: There is a lack of a simple yet accurate method to obtain 3D masks for specific regions.
Region Preservation: Current approaches struggle to achieve desired edits while faithfully preserving the structure and appearance of non-target regions. Methods often suffer from unintended degradation, cross-view inconsistencies, or boundary artifacts when modifying specific parts of an asset.

While recent advances in 3D generative models allow for asset creation from text or images, modifying existing assets (e.g., replacing components or removing geometry) requires balancing target modification with source preservation. Spatial masks alone cannot prevent target conditioning from affecting non-target regions, and direct feature replacement can disrupt cross-region consistency.

Methodology

EditFlow3D is a training-free framework designed for accurate local editing of 3D assets. It operates in the native representation space of a pretrained 3D generative model (specifically the TRELLIS family) and consists of two main stages:

1. VLM-Driven Automated Editing Control Construction

This stage converts flexible user inputs (text instructions, optional reference images, and source 3D assets) into precise visual and spatial controls.

View Selection & Guidance Generation: A Vision-Language Model (VLM) analyzes multi-view renderings of the source asset to select the optimal view ( $I_{src}$ ) that exposes the target region. The VLM then directs an image editing model to generate a visual guidance image ( $I_{edit}$ ) depicting the desired modification.
3D Mask Construction:
- Initial Decomposition: P3-SAM is used to decompose the source asset into parts in native 3D space.
- Refinement: To address region leakage at part junctions, the system utilizes hierarchical partitions from PartField and multi-granularity partitions from S2AM3D. It maps these outputs to surface points and selects candidate parts based on "overlap purity" against the initial mask.
- Fusion: Candidates are filtered and fused based on geometric connectivity and surface-normal consistency to produce a refined surface mask ( $M_{ref}$ ).
- Final Mapping: $M_{ref}$ is mapped to the native sparse coordinates of the generative model to create the final editing mask ( $M_{edit}$ ) and its complement ( $M_{keep}$ ).

2. Local 3D Flow Editing

This stage performs the edit using a two-pronged guidance strategy within the generative model's flow:

Differential Flow Guidance (DFG):
- Operates during sparse voxel generation.
- Calculates the editing direction by comparing velocity fields predicted under the source visual condition ( $c_{src}$ ) and the target visual condition ( $c_{edit}$ ).
- The difference ( $\Delta v_t$ ) serves as the editing vector.
- The 3D mask ( $S_{edit}$ ) spatially constrains this update, restricting flow changes strictly to the target region: $z_{edit} \leftarrow z_{edit} + \gamma \Delta t S_{edit} \odot \Delta v_t$ .
Trajectory Preservation Guidance (TPG):
- Operates during subsequent feature generation (geometry and texture).
- Addresses the issue where global visual conditioning causes non-target features to drift from the source state.
- Instead of hard-replacing intermediate features (which causes artifacts), TPG enforces a soft constraint. It calculates a preservation loss ( $L_{pres}$ ) by comparing the predicted clean state in the non-target region ( $S_{keep}$ ) against the encoded source state ( $x_{src}$ ).
- The editable state is softly corrected via the gradient of this loss: $z_{edit} \leftarrow z_{edit} - \lambda (\nabla_{z_{edit}} L_{pres})$ . This maintains boundary coherence and reduces unintended degradation without disrupting the generation trajectory.

Key Contributions

EditFlow3D Framework: A training-free, inversion-free framework that automatically converts user inputs into accurate visual and spatial controls for local 3D editing.
Novel Editing Algorithm: A local 3D flow-editing algorithm combining Mask-Guided Differential Flow (for targeted modification) and Step-Wise Trajectory Preservation (for non-target consistency). This approach restricts flow updates to the target region while suppressing drift in non-target regions without hard feature replacement.
EditFlow-Bench: A new benchmark comprising 100 source assets and 200 editing cases across five types (addition, deletion, replacement, geometry modification, appearance modification). This complements the existing Edit3D-Bench by covering a broader variety of structural and texture changes.

Results

The framework was evaluated on both EditFlow-Bench and the public Edit3D-Bench using TRELLIS and TRELLIS.2 backbones.

Quantitative Performance: EditFlow3D achieved the best overall results across all metrics (Similarity, CD, PSNR, SSIM, LPIPS, DINO-I, FID) on both benchmarks. It demonstrated superior target-region alignment compared to baselines like Nano3D, PartFlow, VoxHammer, and Vinedresser3D, while significantly outperforming them in preserving non-target regions.
Qualitative Comparison: Visual results showed that while baselines often introduced global darkening, incomplete modifications, or boundary discontinuities, EditFlow3D successfully localized changes while preserving the geometry and appearance of unedited regions.
Ablation Studies:
- Removing DFG significantly reduced editing accuracy (Sim. dropped from 0.318 to 0.287).
- Removing TPG maintained editing accuracy but severely degraded preservation metrics (PSNR, SSIM, LPIPS, FID), confirming its role in preventing non-target drift.
- Adaptive view selection and mask refinement were shown to be critical for both accuracy and preservation.
User Study: In a study with 40 participants, EditFlow3D was preferred over all baselines in 43% of cases for instruction alignment, 57% for visual quality, and 49% for source preservation.

Significance

The paper claims that EditFlow3D addresses the critical trade-off between target modification and source preservation in 3D editing. By introducing Trajectory Preservation Guidance, the method solves the problem of non-target region degradation that persists even when spatial masks are used, as target conditioning can still influence the generation process globally. The framework demonstrates that precise local editing can be achieved without training new models or inverting source assets, relying instead on the native representation space of pretrained generative models coupled with automated, VLM-driven control construction. The introduction of EditFlow-Bench provides a necessary standard for evaluating diverse structural and texture edits, filling a gap in existing evaluation protocols.