KGEdit: Ambiguity-Aware Knowledge Graphs for Training-Free Precise Video Generation and Editing

Technical Summary of KGEdit: Ambiguity-Aware Knowledge Graphs for Training-Free Precise Video Generation and Editing

Problem Statement
Recent advancements in training-free video generation have expanded the capabilities of text-to-video (T2V) diffusion models. However, existing methods struggle when processing complex textual instructions due to inherent ambiguities in natural language. Current approaches often rely on holistic semantic encoding, cross-attention manipulation, or inversion-based guidance, which fail to explicitly capture key attributes and semantic relations. Consequently, models are prone to semantic ambiguity, incorrect concept binding (e.g., misinterpreting "bank" as a financial institution rather than a riverbank), and cross-frame inconsistency (temporal flickering and drift). Achieving precise, temporally stable video generation without additional model training remains a significant challenge, particularly when instructions involve identity, appearance, motion, or multi-attribute control.

Methodology
The authors propose KGEdit, a unified, training-free framework designed to enhance interpretability and controllability by structuring the textual condition itself rather than increasing input modalities. The framework operates through three core components:

1. Ambiguity-Aware Knowledge Graph (AAKG):

   • Sense Disambiguation: The system first identifies polysemous words in the input prompt and selects the most contextually consistent sense using an external knowledge source and the prompt context.
   • Graph Construction & Grouping: The disambiguated semantics are organized into a structured graph $G=(V, E)$. This graph is decomposed into four distinct semantic groups:
     • Identity ($T_{id}$): Core entity definitions.
     • Relation ($T_{rel}$): Relationships between entities.
     • Attribute ($T_{attr}$): Detailed descriptions of appearance and properties.
     • Negative Constraints ($T_{neg}$): Explicit exclusions to suppress incompatible bindings (e.g., "not a financial institution").

2. Structured Semantic Injection Module (SSIM):

   • Instead of relying solely on global embeddings, SSIM injects the disentangled embeddings of the four semantic groups directly into intermediate layers of the diffusion Transformer (DiT).
   • The module formulates a multi-objective optimization problem based on cross-attention maps to enforce consistency. It defines specific loss terms for identity, relational, attribute, and negative constraints, as well as a temporal consistency term.
   • These losses are combined into a unified guidance objective $L_{total}(t)$, which steers the latent reverse process via gradient-based guidance during inference.

3. Temporal-Aware Semantic Control (TASC):

   • Recognizing that different semantic components are critical at different stages of the denoising process, TASC dynamically schedules the weighting of semantic objectives.
   • Early Stage: High weight on Identity ($\lambda_{id}$) and Negative constraints ($\lambda_{neg}$) to establish global structure and suppress incorrect concepts.
   • Middle Stage: Peak weight on Relations ($\lambda_{rel}$) to guide inter-entity spatial composition.
   • Late Stage: Increasing weight on Attributes ($\lambda_{attr}$) to refine fine-grained details and textures.
   • This dynamic scheduling prevents the interference of early-stage structural constraints with late-stage detail refinement, thereby improving temporal stability.

Key Contributions

• KGEdit Framework: A unified, training-free approach for structured semantic control in video generation and editing that improves accuracy and temporal stability without requiring additional model training.
• AAKG and SSIM: The introduction of an Ambiguity-Aware Knowledge Graph to decompose prompts into structured triplets (identity, relation, attribute, negative) and a Structured Semantic Injection Module to inject these signals into key DiT layers, enhancing semantic alignment and generation precision.
• TASC Module: A Temporal-Aware Semantic Control mechanism that dynamically schedules semantic objectives according to the stage-wise characteristics of the denoising process, significantly improving temporal consistency.
• Empirical Validation: Extensive experiments demonstrating that KGEdit outperforms existing training-free methods and unified frameworks (such as VACE) on the VBench benchmark, achieving state-of-the-art results in multiple metrics.

Experimental Results
The authors evaluated KGEdit on the VBench benchmark using the Wan 2.1 1.3B video diffusion backbone.

• Quantitative Performance: KGEdit achieved the highest average score (79.62) across seven automatic metrics, surpassing the unified VACE framework and other baselines like EIDT-V and Free-Bloom.
• Temporal Consistency: The method excelled in temporal metrics, achieving 99.20% Subject Consistency, 99.19% Motion Smoothness, and 98.89% Background Consistency. This highlights the effectiveness of TASC in preventing flickering and semantic drift.
• Qualitative Improvements: In user studies and visual comparisons, KGEdit demonstrated superior ability to handle complex prompts involving rapid motion, multiple attributes, and ambiguous terms. Unlike baselines that exhibited attribute leakage or semantic drift (e.g., changing clothing colors or misinterpreting "bank"), KGEdit maintained precise attribute binding and semantic alignment throughout the video sequence.
• Ablation Studies: Removing any component (AAKG, specific triplet groups, or TASC) led to performance degradation. Notably, removing Identity ($T_{id}$) caused the largest drop in semantic accuracy, while removing Negative constraints ($T_{neg}$) severely impacted ambiguity resolution. Uniform temporal weighting (w/o TASC) resulted in significant drops in temporal consistency, confirming the necessity of dynamic scheduling.

Significance and Claims
The paper claims that KGEdit addresses the fundamental limitations of current training-free video generation methods by shifting the focus from adding control modalities to improving the interpretability and controllability of the textual condition itself. By structuring ambiguous natural language into explicit semantic components and scheduling their influence over time, the framework enables precise, stable, and coherent video generation under complex instructions. The authors position this work as a step toward practical, high-quality video creation systems that do not require costly model retraining, making structured semantic control accessible for diverse applications in advertising, animation, and gaming.

Limitations
The authors acknowledge current limitations, including reliance on Large Language Models (LLMs) for graph construction, the use of fixed injection layers, and hand-designed temporal schedules. Future work is suggested to explore adaptive graph construction, learned scheduling mechanisms, and lightweight guidance strategies.