💻 computer science

InfVSR: Breaking Length Limits of Generic Video Super-Resolution

The paper introduces InfVSR, an autoregressive one-step diffusion framework that reformulates video super-resolution to enable efficient, streaming inference of arbitrarily long videos with state-of-the-art quality and temporal consistency, achieving up to a 58x speed-up over existing methods.

Original authors: Ziqing Zhang, Kai Liu, Zheng Chen, Xi Li, Yucong Chen, Bingnan Duan, Linghe Kong, Yulun Zhang

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ziqing Zhang, Kai Liu, Zheng Chen, Xi Li, Yucong Chen, Bingnan Duan, Linghe Kong, Yulun Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very old, blurry home video of a family vacation. It's grainy, shaky, and hard to make out. You want to use a magic tool to make it crisp and clear (High Definition).

For a short clip of 10 seconds, current "magic tools" (AI models) work okay. But what if you want to fix a whole movie that is 2 hours long? That's where the paper's new invention, InfVSR, comes in.

Here is the story of InfVSR, explained simply:

The Problem: The "Memory Overload" and the "Flicker"

Imagine trying to fix a 2-hour movie using today's best AI tools. They face two big headaches:

The Memory Crash: To fix the whole movie at once, the computer tries to hold every single frame in its mind simultaneously. It's like trying to remember every word of a 2-hour speech while reciting it. The computer's memory (RAM) explodes, and it crashes.
The Flickering Ghost: To avoid crashing, other tools chop the movie into tiny 5-second chunks, fix them one by one, and glue them back together. But because they don't talk to each other, the characters might change clothes between chunks, or the background might jump. It's like a movie where the actor's face flickers or the scenery changes randomly every few seconds.

The Solution: InfVSR (The "Streaming Storyteller")

The authors created InfVSR, which they describe as a "Consistency-Driven Streaming" system. Think of it as a master storyteller who can read a book page by page without ever forgetting the beginning, without needing to hold the whole book in their hands.

Here is how it works, using three simple metaphors:

1. The "Rolling Notebook" (Causal Structure & KV-Cache)

Instead of trying to remember the whole movie, InfVSR keeps a rolling notebook.

As it fixes the current scene, it writes down the most important details (like where the sun is, or how the character is moving) in a small notebook.
When it moves to the next scene, it looks at the notebook to remember what happened just before.
The Magic: It doesn't keep the entire movie in the notebook. It only keeps the last few pages. If a new page comes in, the oldest page falls out. This means the computer's memory usage stays the same, whether the video is 1 minute or 1,000 minutes long. It can stream forever without crashing.

2. The "One-Step Leap" (One-Step Diffusion)

Old AI tools fix a video by taking 50 tiny, slow steps to clean up the image (like peeling an onion layer by layer).

InfVSR is trained to take one giant leap. It looks at the blurry frame and instantly jumps to the clear version.
The Result: This makes the process incredibly fast. The paper claims it is 58 times faster than the previous best method. It's the difference between walking up a mountain one step at a time versus taking a helicopter ride.

3. The "Anchor Rope" (Joint Visual Guidance)

To stop the "flickering" problem (where the video looks different in different chunks), InfVSR uses a special Anchor Rope.

It looks at the original blurry video (which is still there) and grabs a "semantic anchor"—a strong clue about what the scene should look like (e.g., "This is a blue sky," "This is a red car").
It ties this anchor to every single chunk it fixes. Even if the AI is working on a new part of the movie, the anchor rope pulls it back to the original reality, ensuring the character's face and the background stay consistent from start to finish.

The New "Long-Video" Test (MovieLQ)

The authors realized that no one was really testing these tools on long videos. Most tests were just 10-second clips.

They built a new test called MovieLQ, which consists of 10 real-world videos that are 1,000 frames long (about 40 seconds of continuous, uncut footage with real-world blur and noise).
They also introduced a new way to grade the videos. Instead of just checking if the pixels look sharp, they check if the story makes sense: Did the character's face stay the same? Did the background stay stable? Did the movement look smooth?

The Result

When they tested InfVSR against the current champions:

Speed: It was the fastest, finishing tasks in seconds that took others minutes.
Quality: It produced the clearest, most realistic images.
Consistency: It didn't flicker. The characters and backgrounds stayed consistent throughout the entire long video.

In short: InfVSR is a new AI tool that can fix unlimited-length videos without running out of memory, without making the video flicker, and doing it 58 times faster than before. It's like having a magic wand that can restore a whole movie in real-time, one frame at a time, without ever losing its place.

Technical Summary: InfVSR

Problem Statement

Real-world video content often extends over thousands of frames, creating a significant bottleneck for existing Generative Video Super-Resolution (VSR) approaches. Current methods face two primary challenges when processing such long sequences:

Inefficiency: Multi-step denoising processes required by full-sequence diffusion models incur prohibitive computational costs and memory usage that scale with video length.
Inconsistency: To manage memory, existing methods often decompose videos into short, independent chunks. This temporal decomposition breaks the global temporal coherence learned by pretrained models, leading to artifacts, flickering textures, and discontinuities that disrupt the viewing experience.

While previous solutions like increased overlap or post-hoc smoothing partially address these issues, they fail to preserve long-range and semantic-level consistency. There is a pressing need for VSR models that are both streamable (handling arbitrary lengths with fixed memory) and consistent (explicitly modeling temporal coherence).

Methodology: InfVSR

The authors propose InfVSR, the first consistency-driven streaming generative VSR framework. It reformulates VSR as an Autoregressive One-Step Diffusion (AR-OSD) paradigm, leveraging pretrained Text-to-Video (T2V) priors while enabling efficient, streaming inference.

1. Causal DiT Architecture with Dual-Timescale Modeling

The core of InfVSR is a causal adaptation of a pretrained 3D Diffusion Transformer (DiT), designed to maintain both local and global coherence without revisiting previous frames.

Rolling KV-Cache (Local Smoothness): Inspired by Large Language Models (LLMs), the method implements a fixed-length, rolling Key-Value (KV) cache within the self-attention layers. By assigning absolute positional embeddings based on the original timeline, the model maintains temporal awareness across chunk boundaries. This ensures smooth motion and appearance transitions while keeping memory and computation constant, regardless of video length.
Joint Visual Guidance (Global Coherence): To prevent the loss of long-range cues due to cache truncation, the method introduces a joint visual guidance module. Low-Resolution (LR) key frames are encoded using a pretrained visual encoder (DAPE) and injected as visual prompts into the cross-attention layers of the DiT. These prompts serve as shared semantic anchors across multiple adjacent chunks, stabilizing long-range semantics and acting as a lightweight alternative to VLM-based prompt extraction.

2. Efficient Autoregressive Post-Training

The training strategy is designed to balance fidelity, consistency, and memory efficiency through a two-stage curriculum and specific loss functions.

Patch-wise Pixel Supervision: To avoid the prohibitive memory overhead of full-frame supervision in 3D VAE decoders, the method employs random spatial cropping. Reconstruction loss is applied to randomly sampled patches from the latent space and corresponding ground-truth patches in pixel space. This strategy includes:
- Fidelity Loss: Combining Mean Squared Error (MSE) and DISTS perceptual loss.
- Temporal Smoothness Loss: Aligning frame-wise differences between predicted and ground-truth sequences to enforce local temporal consistency.
Cross-Chunk Distribution Matching (DMD): To align generated videos with realistic motion patterns over extended ranges, a distribution matching loss is applied. This minimizes the KL divergence between the distribution of generated triplet-chunk sequences and real video distributions learned from a teacher model, ensuring high-level semantic consistency.
Two-Stage Curriculum:
- Stage I: Initialization on high-resolution, long clips using only pixel supervision to fit the one-step diffusion.
- Stage II: Autoregressive adaptation at lower resolution, incorporating the KV-cache, AR inference, and both pixel and distribution matching losses.

3. Benchmarking: MovieLQ

To address the lack of long-form evaluation standards, the authors introduce MovieLQ, a benchmark comprising ten 1000-frame-long, single-shot videos with real-world degradations. Beyond conventional pixel-wise metrics, they adopt semantic-level metrics from VBench (Subject Consistency, Background Consistency, and Motion Smoothness) to comprehensively assess temporal coherence.

Key Contributions

InfVSR Framework: A novel T2V-based autoregressive one-step diffusion framework for real-world VSR that supports ultra-efficient inference on unbounded-length sequences.
Dual-Timescale Mechanism: The introduction of a rolling KV-cache for local smoothness and joint visual guidance for global coherence, enabling constant memory usage.
Training Innovations: A patch-wise pixel supervision strategy and cross-chunk distribution matching loss that effectively balance fidelity, consistency, and efficiency.
New Benchmark: The establishment of MovieLQ and the adoption of semantic-level consistency metrics for long-form VSR evaluation.

Experimental Results

Extensive experiments demonstrate that InfVSR achieves State-of-the-Art (SOTA) performance across various datasets (UDM10, SPMCS, MVSR4x, VideoLQ, and MovieLQ).

Quality: The method achieves top-1 or top-2 scores in fidelity (PSNR, SSIM) and perceptual quality (MUSIQ, CLIP-IQA, DOVER) across multiple benchmarks.
Consistency: InfVSR attains the lowest flow warping errors ( $E^*_{warp}$ ) and superior semantic consistency scores (SC, BC, MS) compared to existing methods like MGLD-VSR, STAR, and SeedVR2.
Efficiency: The method delivers a 58× speed-up over multi-step methods like MGLD-VSR and is significantly faster than recent one-step methods (e.g., 5.48× faster than SeedVR2). Crucially, memory usage remains constant even for long videos (e.g., 1000 frames), whereas full-attention baselines suffer from Out-Of-Memory (OOM) errors or linear memory growth.

Significance

The paper claims that InfVSR breaks the length limitations of existing full-sequence approaches by reformulating VSR as an AR-OSD paradigm. By combining causal DiT adaptation with single-step distillation and consistency-driven training, the method enables ultra-efficient inference while preserving temporal coherence. The authors position InfVSR as a foundational step toward the practical deployment of generative VSR systems for long-range video enhancement, offering a scalable solution that addresses the critical trade-off between quality, consistency, and computational cost.