💻 computer science

Train Short, Inference Long: Training-free Horizon Extension for Autoregressive Video Generation

FLEX is a training-free inference framework that extends autoregressive video generation horizons by addressing spectral bias and noise sampling limitations through Frequency-aware RoPE Modulation, Antiphase Noise Sampling, and an Inference-only Attention Sink, enabling high-quality long-duration video synthesis without additional model training.

Original authors: Jia Li, Xiaomeng Fu, Xurui Peng, Weifeng Chen, Youwei Zheng, Tianyu Zhao, Jiexi Wang, Fangmin Chen, Xing Wang, Hayden Kwok-Hay So

Published 2026-03-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jia Li, Xiaomeng Fu, Xurui Peng, Weifeng Chen, Youwei Zheng, Tianyu Zhao, Jiexi Wang, Fangmin Chen, Xing Wang, Hayden Kwok-Hay So

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a child how to draw a story. You show them a few pages of a comic book (the training phase). They learn how to draw a character, a car, and a tree. But then, you ask them to keep drawing that story for hours, far beyond the few pages they studied.

What usually happens? The child gets confused. The car starts melting, the character's face changes into a blob, and the story loses its plot. In the world of AI video generation, this is called extrapolation failure. The AI knows how to make a 5-second clip, but when asked to make a 60-second or 4-minute video, it falls apart.

The paper you shared introduces a clever solution called FLEX. It's like giving the AI a set of "training wheels" and a "map" so it can keep drawing the story correctly for hours, without needing to go back to school and relearn everything.

Here is how FLEX works, broken down into three simple tricks:

1. The "Zoom Lens" for Time (Frequency-aware RoPE Modulation)

The Problem:
Think of the AI's understanding of time like a radio dial. It has stations for "fast changes" (like a flickering light) and "slow changes" (like a sunset).
When the AI was trained on short clips, it only listened to the "fast change" stations. It never got enough practice with the "slow change" stations. When you ask it to make a long video, it tries to tune into those slow stations but gets static because it's never heard them before.

The FLEX Fix:
FLEX acts like a smart zoom lens.

For the fast changes (high frequency), it says, "You're good at this! Keep doing exactly what you do." It lets the AI use its natural talent for quick details.
For the slow changes (low frequency), it says, "You haven't practiced this much. Let's slow you down and stretch your understanding so you can handle the long timeline."
This keeps the video sharp and detailed without losing the big picture.

2. The "Anti-Boredom" Noise (Antiphase Noise Sampling)

The Problem:
When AI starts generating a video, it starts with a cloud of random static (noise). Usually, to keep the video consistent, the AI is told to make this noise very similar from frame to frame.
Imagine a drummer who is told to keep the beat exactly the same for 100 hours. Eventually, the music becomes a boring, robotic drone. The video looks like a frozen painting that barely moves. The AI gets "too consistent" and loses all the energy and motion.

The FLEX Fix:
FLEX introduces Antiphase Noise. Instead of telling the AI to make the noise similar, it tells it to make the noise opposite to the previous frame.

Think of it like a pendulum. If the previous frame swings left, the next one is nudged to swing right.
This injects energy right at the start. It forces the AI to create movement and dynamic changes, preventing the video from turning into a boring, static slideshow. It's like adding a spark to a campfire so it keeps burning bright instead of dying out.

3. The "Anchor" (Attention Sink)

The Problem:
As the AI draws more and more frames, it starts to forget the beginning of the story. It's like a person telling a long joke who forgets the punchline setup halfway through. The characters might change faces, or the background might shift colors because the AI lost its grip on the original scene.

The FLEX Fix:
FLEX adds an Anchor.

Imagine the AI is a ship sailing on a long ocean. As it sails, it might drift off course. FLEX ties a heavy anchor to the very first frame of the video and keeps it connected to the ship.
No matter how far the video goes (even 4 minutes long), the AI constantly looks back at that first frame to remind itself: "This is who the character is, and this is where the scene started." This stops the characters from morphing into strangers.

The Result: "Train Short, Inference Long"

The magic of FLEX is that it is training-free.

Old Way: To make a 4-minute video, you had to train the AI on 4-minute videos. This takes millions of dollars and weeks of computing power.
FLEX Way: You take an AI trained on 5-second clips, plug in FLEX, and suddenly it can make 4-minute videos that look just as good.

In a nutshell:
FLEX is a plug-and-play upgrade that fixes the AI's "short attention span" and "boredom." It gives the AI a better map for time, a spark of energy for movement, and an anchor to keep it on track. This allows us to generate long, consistent, and dynamic videos without the massive cost of retraining the models.

1. Problem Statement

Autoregressive (AR) diffusion models have emerged as a scalable paradigm for long video generation due to their inference efficiency (using KV caches). However, they suffer from severe extrapolation failure when the inference horizon significantly exceeds the training horizon.

Symptoms: Rapid error accumulation leads to temporal drift, visual artifacts, motion degradation, and loss of subject identity.
Root Causes Identified:
1. Spectral Bias in 3D Positional Embeddings: 3D Rotary Positional Embeddings (RoPE) exhibit imbalanced training exposure. Low-frequency components (responsible for global structure) are often under-trained because they fail to complete a full rotation cycle within short training clips. Standard Position Interpolation (PI) compresses all frequencies uniformly, destroying fine-grained high-frequency temporal details.
2. Lack of Dynamic Priors in Noise Sampling: Existing noise initialization strategies often rely on positive temporal correlations to ensure consistency. This over-stabilizes the latent space, leading to monotonous content, insufficient motion energy, and "frozen" video sequences over long durations.
Limitation of Current Solutions: Existing methods like "Streaming Long Tuning" require fine-tuning on long self-rollout sequences, which is computationally prohibitive and bound by the "train long, inference long" constraint.

2. Methodology: FLEX Framework

The authors propose FLEX (Frequency-aware Length EXtension), a training-free, inference-time framework comprising three core components to bridge the gap between short-term training and long-term inference.

A. Frequency-aware 3D RoPE Modulation

Inspired by "NTK-by-parts" in Large Language Models, this component addresses the spectral bias of positional embeddings.

Mechanism: It quantifies "training exposure" ( $r_m$ ) for each rotary dimension based on the number of completed rotation cycles during training.
Strategy:
- Low-Frequency Interpolation ( $r_m < \alpha$ ): For under-trained low-frequency components (global structure), the method adaptively interpolates them back into the familiar phase domain to maintain global positional capability.
- High-Frequency Extrapolation ( $r_m > \beta$ ): For well-trained high-frequency components (fine-grained details), it preserves the original rotation angles to avoid compressing temporal discriminability.
Result: This prevents the "static video" failure of standard PI while preserving multi-scale temporal resolution.

B. Antiphase Noise Sampling (ANS)

This component addresses the lack of motion dynamics by reshaping the noise distribution.

Mechanism: Instead of standard independent or positively correlated noise, ANS introduces a structured initialization using a first-order autoregressive (AR(1)) process with a negative correlation coefficient ( $\rho < 0$ ).
Theoretical Basis:
- Spectral Seeding: Negative correlation shifts the Power Spectral Density (PSD) toward high frequencies (high-pass characteristic).
- Energy Monotonicity: The adjacent-difference energy is maximized when $\rho < 0$ , injecting high-frequency temporal priors into the latent space.
Result: This seeds the generation with dynamic variations, preventing motion collapse and ensuring rich, diverse motion over long horizons without altering the model architecture.

C. Inference-only Attention Sink

To combat global context degradation over extended horizons.

Mechanism: The first $N$ frames within the local attention window are fixed as persistent "anchors" (sinks) for global semantic and structural consistency.
Advantage: Unlike training-based approaches (e.g., LongLive), this is a plug-and-play inference strategy that requires no architectural changes or fine-tuning.

3. Key Contributions

Systematic Analysis: Identified two fundamental inference-time failure modes in AR video models: spectral bias in 3D RoPE and the lack of dynamic priors in noise sampling.
FLEX Framework: Proposed a training-free solution integrating Frequency-aware RoPE Modulation, Antiphase Noise Sampling, and Attention Sinks.
State-of-the-Art Performance: Demonstrated that principled inference-time modifications can outperform or match computationally expensive training-based baselines.

4. Experimental Results

Evaluations were conducted on VBench-Long and MovieGen prompts, comparing FLEX against SOTA models like CausVid, Self Forcing, LongLive, and Rolling Forcing.

30s Generation (6× Extrapolation):
- FLEX achieved a Quality Score of 83.01, surpassing all baselines (e.g., LongLive: 82.78, Self Forcing: 81.21).
- It showed minimal imaging quality drift (-0.06) compared to the significant drift in baselines.
- Dynamic Degree improved significantly (from 26.24 in Self Forcing to 40.84 in FLEX).
60s Generation (12× Extrapolation):
- FLEX achieved a Quality Score of 82.68, matching the performance of LongLive (which required 60s self-rollout fine-tuning) while using a model trained only on 5s clips.
- It effectively suppressed quality drift (-0.56 vs -2.51 for Self Forcing).
Minute-Level Generation (4 Minutes):
- Integrated into LongLive, FLEX supported consistent generation up to 240 seconds (4 minutes).
- It reduced image drift from -2.19 to -1.17 and maintained subject identity and fine-grained textures where baselines suffered from identity fading or scene collapse.
Ablation Studies: Confirmed that removing any of the three components (RoPE modulation, ANS, or Attention Sink) led to significant drops in consistency, dynamics, or overall quality.

5. Significance

Training-Free Scalability: FLEX breaks the "train long, inference long" bottleneck. It allows models trained on short clips to generate high-quality, minute-long videos without the prohibitive cost of fine-tuning on long sequences.
Plug-and-Play Compatibility: The framework integrates seamlessly into existing autoregressive inference pipelines (e.g., Self Forcing, LongLive) without modifying model weights or architecture.
Theoretical Insight: The work provides a crucial understanding of how spectral bias in 3D RoPE and noise priors affect long-horizon generation, offering a new direction for extending context in diffusion models beyond simple fine-tuning.
Practical Impact: It pushes the boundaries of video generation, enabling consistent, dynamic, and semantically aligned synthesis at scales previously unattainable by standard AR models.