💻 computer science

InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter

The paper introduces InfinityEdit, a lightweight adapter for pretrained video generators that enables unbounded, instruction-based editing of streaming video by ensuring faithful temporal continuity and stable generation quality across an infinite sequence of edit requests.

Original authors: Yunze Tong, Mushui Liu, Canyu Zhao, Shiyi Zhang, Didi Zhu, Peng Zhang, Wanggui He, Jinlong Liu, Ying Chen, Hao Jiang, Pipei Huang, Bo Zheng

Published 2026-08-24

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yunze Tong, Mushui Liu, Canyu Zhao, Shiyi Zhang, Didi Zhu, Peng Zhang, Wanggui He, Jinlong Liu, Ying Chen, Hao Jiang, Pipei Huang, Bo Zheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine watching a live broadcast of a soccer match, and suddenly you wish the entire stadium looked like a watercolor painting, or you wanted the camera to suddenly swoop down from the sky to follow the ball. In the world of artificial intelligence, creating such changes is usually a rigid process. Current tools can take a short, finished video clip and alter it, but they treat the clip as a closed box. They can rewrite what is already there, but they cannot easily reach into the future to change what happens next. If the video keeps playing, the old rules stop working, and the new style or camera angle vanishes. This limitation makes it difficult to edit live streams or ongoing scenes where the story is still unfolding.

A team of researchers has tackled this problem by developing a new way to edit videos that never end. They call their approach "infinite video editing." Instead of forcing a video to fit into a fixed time slot, their method allows an artificial intelligence to apply a change—like a new visual style or a camera movement—and then keep generating new frames that naturally continue that change forever. The system does not just rewrite the past; it learns to carry the edit forward into the future, ensuring that the video remains consistent and high-quality even as new instructions arrive one after another.

The researchers began by recognizing that existing video editors operate on a simple assumption: the video is already finished. They take a source clip and produce an edited version of the exact same length. This works well for short films or social media posts, but it fails when the video is a continuous stream, like a live game or a security camera feed. In these situations, the video keeps growing. If you want to change the style of the footage halfway through, the edit must extend to every new frame that appears. The old methods cannot do this because they are designed to look backward at a static clip, not forward into an open-ended stream.

To solve this, the team first built a way to teach computers how to handle this specific challenge. They created a massive collection of training examples. They took existing videos and paired them with instructions for how to change them, such as "make this look like a comic book" or "move the camera down." Crucially, they did not just ask the computer to change the existing frames. They generated new video segments that continued from the original footage while applying the requested change. This taught the system that an edit is not just a filter applied to a snapshot, but a new direction for the story to take. They carefully filtered this data to ensure the new video segments looked realistic and that the changes were applied correctly without breaking the flow of the scene.

With this data in hand, they introduced a new system called InfinityEdit. This system works by attaching a small, lightweight "adapter" to a powerful video generator that is already very good at making long videos. Think of the main generator as a skilled artist who can draw a continuous stream of images, and the adapter as a special brush that can change the style of the drawing on the fly. The main generator remains frozen, meaning its core knowledge of how to create stable, high-quality video is never altered. The adapter is the only part that learns. It sits between the generator and the user's instructions, ready to step in only when a new edit request arrives.

When a user sends an instruction, the adapter activates for just one short segment of the video. It takes the instruction and the recent history of the video and generates a new chunk of frames that applies the change. This new chunk acts as a bridge. Once this first edited segment is created, the adapter turns off, and the main generator takes over again. Because the main generator is now looking at the newly edited frames as its history, it naturally continues the video in the new style or with the new camera angle. The edit has been "ignited," and the system carries it forward automatically. This process can repeat endlessly. If a second instruction arrives later, the adapter wakes up again to apply the new change, and the generator continues from there.

The researchers found that this approach solves two major problems that usually plague long video generation. The first is consistency. In many systems, if you try to edit a video that is already being generated, the changes often fade away or get lost as the video progresses. The new frames might drift back to the original look. InfinityEdit prevents this by resetting a key reference point every time an edit happens. This ensures that the video stays anchored to the most recent change, keeping the style or camera angle stable for as long as the video continues.

The second challenge is quality. When a computer generates video frame by frame, small mistakes can pile up over time, making the picture look blurry or distorted. The researchers trained their adapter to handle imperfect history. They deliberately gave the system examples where the previous frames were slightly flawed, teaching it to recover and produce clean results even when the input was not perfect. This training allows the system to remain stable even after many rounds of editing, preventing the video from degrading as it gets longer.

In their tests, the system successfully applied a wide variety of edits to ongoing video streams. It could change the visual style of a scene, alter the camera movement, or modify the appearance of objects, and then continue generating the video with those changes intact. The results showed that the video remained faithful to the instructions and did not drift back to the original state. The system handled these changes repeatedly without losing quality, proving that it is possible to edit a video stream as it happens, rather than waiting for the video to finish.

This work opens the door to a new kind of interaction with video. Instead of editing a finished product, users can now guide a video as it is being created. Whether it is restyling a live sports broadcast in real-time or changing the perspective of a virtual camera during a long recording, the technology offers a way to keep the edit alive as the story unfolds. The researchers have shown that by combining a stable, pre-trained generator with a flexible, lightweight adapter, we can create video editing tools that are not limited by time, allowing for a truly infinite stream of creative possibilities.

Technical Summary of InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter

1. Problem Definition: Infinite Video Editing

Existing instruction-based video editing methods operate under an in-place editing assumption. They align an edited output frame-by-frame with a fixed-duration source clip. While effective for static clips, this paradigm fails for open-ended video streams (e.g., live game restyling or applying camera moves to ongoing shots) where new content arrives continuously.

The paper defines Infinite Video Editing as a task where, given a preceding video segment ( $V_{pcd}$ ) and an edit instruction ( $c_{edit}$ ), a model must generate a subsequent segment ( $V_{tgt}$ ) that:

Faithfully applies the edit requested in $c_{edit}$ .
Continues the stream naturally, generating future frames that were not present in the input, rather than rewriting existing frames.
Maintains stability over an unbounded sequence of edits, preventing error propagation and quality degradation as edits accumulate.

The core challenges identified are:

Faithful Continuation: The model cannot simply transform input frames; it must generate a coherent continuation where entities unaffected by the edit remain consistent with the history.
Stability: As each generated segment becomes the history for the next, errors can compound over time.
Conflict with Streaming Priors: Standard streaming generators use mechanisms like anchor frames and cached history to prevent drift. However, these mechanisms often pull generated content back toward the unedited original, weakening or ignoring new edit instructions.

2. Methodology

Data Collection Pipeline
To train a model for this task, the authors constructed a dataset of triplets $(V_{pcd}, c_{edit}, V_{tgt})$ :

Source: Sampled from the UltraVideo dataset.
Instructions: Generated by expanding abstract edit types (from VAP) into detailed natural language prompts using Gemini 3 Flash, conditioned on source captions.
Targets: Synthesized using Wan2.2-I2V-A14B. The pipeline employs edit-type-aware generation:
- Style/Appearance edits: The connection frame is edited first (using Qwen-Image-Edit) before generating the video.
- Camera/Motion edits: The connection frame is preserved to ensure scene consistency, with changes applied only to the viewpoint.
Filtering: Triplets are manually scored on alignment, consistency, rationality, and visual quality.

Model Architecture: InfinityEdit
The proposed solution attaches a lightweight Edit-Ignition Adapter to a frozen streaming video generator backbone (Helios-Distilled, a 14B autoregressive diffusion transformer). The adapter is the only trainable component.

The adapter consists of three attention modules inserted after each transformer layer:

History Cross-Attention: Anchors the new denoising chunk to the provided input frames. Queries are restricted to the leading frames of the current chunk, while keys/values come from the history tokens. This ensures the edit respects the immediate past.
Temporal Causal Self-Attention: Propagates temporal cues strictly from earlier frames to later frames within the chunk, maintaining the autoregressive flow required for streaming.
Edit Cross-Attention: Injects the edit instruction ( $c_{edit}$ ) into the generation process. All tokens in the current chunk query the edit prompt, allowing the instruction to influence the entire generated segment.

Training Strategy

Objective: Flow-matching objective with a frozen backbone.
History Corruption: To mitigate exposure bias (where the model conditions on its own imperfect outputs at inference), the training process randomly corrupts history frames with noise.
Noise Sampling: Instead of the backbone's pyramid schedule, the adapter uses a Mixture-Gaussian sampling of noise levels ( $\sigma$ ) to decouple edit control from specific resolution stages.
Two-Phase Curriculum:
- Phase 1: Uniform coverage of noise levels to learn basic editing.
- Phase 2: Focuses on middle-to-low noise levels and later frames to refine details and improve quality.

Inference Pipeline: "Ignite-Then-Continue"
The inference process handles infinite edits through a specific strategy:

Ignition: When an edit instruction arrives, the adapter is activated only for the first chunk (the "ignition chunk"). This chunk is generated using the adapter and a single-stage Euler schedule (including a near-zero noise step for detail refinement).
Continuation: Subsequent chunks are generated by the frozen backbone with the adapter disabled. Because the first edited chunk is now part of the history window, the backbone naturally continues the edited stream.
State Management:
- The anchor frame ( $x_0$ ) is reset to the first frame of the new ignition chunk to prevent the model from reverting to the original source.
- The scene prompt is updated to the current edit instruction.
- A sliding window manages memory, dropping old frames to keep costs bounded.

3. Key Contributions

Task Definition: Formalized Infinite Video Editing, distinguishing it from in-place editing and identifying the specific challenges of faithful continuation and long-term stability.
Data Pipeline: Developed a synthesis pipeline for generating edit-type-aware training triplets that bridge the gap between static editing and streaming generation.
Architecture: Proposed InfinityEdit, a lightweight adapter that enables a frozen streaming generator to apply edits without compromising its long-video generation priors.
Inference Strategy: Introduced the "ignite-then-continue" scheme, which applies edits locally while leveraging the backbone's stability mechanisms for the remainder of the stream.

4. Results and Claims

Extensive experiments on out-of-distribution cases demonstrate that:

InfinityEdit faithfully continues the stream under each edit instruction, successfully applying the requested changes to future frames.
The system remains stable over unbounded sequences of edits, avoiding the drift and quality degradation common in approaches that simply re-prompt the model.
The method effectively balances the application of new edits with the preservation of the original model's ability to generate long, coherent videos.

5. Significance

The paper claims that InfinityEdit addresses a critical gap in video generation: the inability to edit open-ended, live streams dynamically. By decoupling the edit injection mechanism from the long-term generation prior, the method allows for on-the-fly editing (e.g., restyling game footage or changing camera angles in real-time) without requiring full model retraining or sacrificing the stability of the underlying generator. This enables practical applications where video streams are continuous and edit requirements are dynamic.