Helios: Real Real-Time Long Video Generation Model

Helios is a pioneering 14B autoregressive diffusion model that achieves real-time, minute-scale video generation with high robustness and quality on a single GPU, eliminating the need for common anti-drifting heuristics, standard acceleration techniques, or parallelism frameworks through novel training strategies and infrastructure optimizations.

The Gist

Imagine you are trying to direct a movie. In the past, making a high-quality video with AI was like trying to paint a masterpiece, but you could only paint for 5 seconds before the paint dried, the colors started to fade, and the characters forgot who they were. If you tried to keep going, the movie would turn into a blurry, glitchy mess. Also, the computer needed a massive supercomputer to do this, and it took hours to render just a few seconds.

Enter Helios. Think of Helios as a super-efficient, real-time movie director that can run on a single powerful laptop (specifically, a single NVIDIA H100 GPU) and keep making a movie forever without losing its mind.

Here is how Helios works, broken down into simple concepts:

1. The "Infinite Scroll" vs. The "Stop Sign"

Most AI video models are like a person reading a book who only remembers the last page. If you ask them to write a story for 10 minutes, they forget the beginning by minute 2, and the characters start acting weird (this is called "drifting").

Helios uses a trick called Unified History Injection. Imagine instead of forgetting the past, the AI has a scroll of paper that it constantly updates. It looks at the "clean" past (the story so far) and the "messy" future (what it's trying to draw next) at the same time. It doesn't just guess; it uses the past to gently guide the future, ensuring the character's face stays the same and the story makes sense for minutes on end.

2. The "Drifting" Problem: The Chameleon Effect

When AI tries to make long videos, it often suffers from three types of "drifting":

• Position Shift: The character suddenly teleports to a different spot.
• Color Shift: The sky turns from blue to purple, or the skin turns green.
• Restoration Shift: The video gets blurry or noisy, like an old TV losing signal.

Helios solves this with three simple "training drills":

• The Anchor: It keeps the very first frame of the video locked in its memory like a compass. No matter how long the video gets, the AI checks this compass to make sure the colors and shapes don't wander off.
• The "Fake It Till You Make It" Drill: During training, the AI is intentionally fed "bad" history (blurry or noisy frames). It learns to fix these mistakes on the fly, making it super robust when it generates real videos later.
• The Relative Map: Instead of counting frames as "Frame 1, Frame 2, Frame 3..." (which confuses the AI if the video gets too long), it uses a relative map. It just knows, "I am 5 steps away from the last thing I saw." This stops the AI from getting confused and repeating the same motion over and over.

3. The Speed Trick: The "Zoom Lens"

Making a 14-billion-parameter model (a huge brain) run in real-time is usually impossible. It's like trying to drive a Ferrari through a traffic jam.

Helios uses Deep Compression Flow, which acts like a smart zoom lens:

• The Past: For things that happened a long time ago, it doesn't need to remember every single detail. It compresses the distant past into a "summary" (like looking at a landscape from a distance vs. up close). This saves massive amounts of memory.
• The Future: When drawing the video, it doesn't start with high-definition details. It starts by sketching a rough, low-resolution outline (like a pencil sketch) and then slowly adds the fine details (like paint). This is much faster than trying to paint every single pixel perfectly from the start.

4. The "Teacher-Student" Hack

Usually, to make an AI fast, you have to "distill" it—teach a small, dumb student to copy a big, smart teacher. But this often makes the video look worse.

Helios uses Adversarial Hierarchical Distillation. Imagine a master chef (the teacher) teaching a sous-chef (the student).

• Instead of just telling the student "copy me," the teacher lets the student cook, and then a tough food critic (the adversarial part) tastes the dish.
• If the dish isn't as good as the real thing, the critic yells at the student to fix it.
• This forces the student to learn not just to copy the teacher, but to actually taste like a real movie. The result? The student becomes incredibly fast (19.5 frames per second!) but still tastes delicious (high quality).

The Result?

• Speed: It generates video at 19.5 frames per second on a single chip. That's faster than many smaller, weaker models. It's essentially "real-time."
• Length: It can generate videos that are minutes long without the characters turning into monsters or the colors melting away.
• Versatility: It can turn text into video, a single image into a video, or even take an existing video and continue it.

In short: Helios is the first AI video model that is fast enough to play a game with you, smart enough to remember the plot for an hour, and cheap enough to run on a single machine, all without needing the complex "cheats" other models rely on. It turns the "impossible" dream of infinite, high-quality video generation into a reality.

Technical Summary

1. Problem Statement

Current video generation models face a critical trade-off between quality, duration, and speed:

• Duration vs. Quality: Most high-quality models (e.g., Wan 2.1, CogVideoX) are limited to short clips (5–10 seconds). Extending generation to minute-scale videos often leads to severe temporal drifting (content degradation, color shifts, repetitive motion) and requires costly training strategies like "self-forcing" (train-as-infer rollouts) or error-banks.
• Speed vs. Scale: Real-time generation (interactive speeds) is typically achieved only by small models (e.g., 1.3B parameters) via aggressive distillation or quantization. Larger models (14B+) are computationally prohibitive, often taking tens of minutes to generate a 5-second clip, even with acceleration techniques like KV-cache or sparse attention.
• Infrastructure: Training 14B video models usually requires complex parallelism (FSDP, TP) and sharding, making them inaccessible to single-GPU setups.

Goal: Develop a 14B video generation model that achieves real-time inference (interactive speeds), supports minute-scale generation without drifting, and can be trained/inferred on a single GPU without standard acceleration heuristics.

---

2. Methodology

Helios is a 14B autoregressive diffusion model built on a Diffusion Transformer (DiT) architecture. It introduces three core pillars to solve the challenges above:

A. Unified History Injection (Architecture)

Instead of using causal masking (which limits bidirectional pre-training benefits), Helios treats long-video generation as infinite video continuation.

• Representation Control: The model takes a concatenation of a Historical Context ($X_{Hist}$) and a Noisy Context ($X_{Noisy}$).
  • If $X_{Hist}$ is zero: Text-to-Video (T2V).
  • If $X_{Hist}$ has only the last frame: Image-to-Video (I2V).
  • Otherwise: Video-to-Video (V2V).
• Guidance Attention: A novel attention mechanism that separates the processing of historical and noisy contexts.
  • Historical context is fixed at timestep 0 (clean) and acts as a guide, not a target for denoising.
  • Head-wise Amplification: Learnable tokens amplify specific attention heads for historical keys, allowing the model to selectively focus on discriminative history while suppressing noise.

B. Easy Anti-Drifting (Training Strategies)

Helios eliminates the need for expensive self-forcing rollouts by explicitly simulating failure modes during training:

1. Relative RoPE: Replaces absolute positional encoding with relative indexing ($0:T_{Hist}$ for history, $T_{Hist}:T_{Hist}+T_{Noisy}$ for noise). This prevents the model from "snapping back" to early positions (repetitive motion) and stabilizes generation at arbitrary lengths.
2. First-Frame Anchor: The first frame of the history is always retained and never corrupted. It acts as a global visual anchor to constrain distribution shifts and prevent color drift.
3. Frame-Aware Corrupt: To prevent "Restoration Shift" (blur/noise accumulation), the model is trained on historical frames that are randomly perturbed (exposure adjustment, noise injection, down/upsampling). This forces the model to be robust to imperfect history.

C. Deep Compression Flow (Efficiency)

To achieve real-time speeds with a 14B model, Helios aggressively reduces token counts and sampling steps:

1. Multi-Term Memory Patchification: Compresses the historical context hierarchically.
   • Short-term: High resolution (fine motion).
   • Mid-term: Medium resolution.
   • Long-term: Low resolution (global context).
   • This keeps the total token count constant regardless of video length, reducing memory and compute by ~8x for history.
2. Pyramid Unified Predictor Corrector: A multi-scale sampling strategy.
   • Sampling starts in a low-resolution latent space (global structure) and progressively upsamples to full resolution (fine details).
   • This reduces the number of tokens processed in early, noisy steps.
3. Adversarial Hierarchical Distillation:
   • Pure Teacher Forcing: Uses the autoregressive Helios-Base as the teacher (unlike methods using bidirectional teachers) and requires only single-section generation per step, avoiding long rollouts.
   • Staged Backward Simulation: Decomposes the backward simulation into stages to match the multi-scale flow.
   • Adversarial Post-Training: Adds a GAN objective to the distillation process to break the performance ceiling of the teacher and improve realism.
   • Result: Reduces sampling steps from 50 to 3.

D. Infrastructure Optimizations

• Single-GPU Training: By compressing contexts and offloading VAE/text embeddings to disk, Helios fits a 14B model (plus distillation components) into 80GB VRAM without parallelism.
• Memory Management: Uses "Cache Grad" for GAN training (freezing activations after forward pass) and asynchronous VRAM freeing to manage peak memory.
• Kernel Fusion: Custom Triton kernels for Flash Normalization and Flash RoPE reduce memory complexity and improve throughput.

---

3. Key Contributions

1. First 14B Real-Time Model: Helios is the first 14B video model to run at 19.5 FPS on a single NVIDIA H100 GPU, outperforming many 1.3B distilled models in speed while maintaining 14B quality.
2. Drifting-Free Long Video: Achieves minute-scale generation without standard anti-drifting heuristics (self-forcing, error-banks, keyframe sampling) by using explicit drift simulation and relative positional encoding.
3. Unified Framework: Supports T2V, I2V, and V2V within a single architecture via Representation Control, eliminating the need for task-specific fine-tuning.
4. Training Efficiency: Enables training a 14B video model on a single GPU with batch sizes comparable to image models, removing the need for complex sharding infrastructure.
5. HeliosBench: Introduces a new benchmark with 240 prompts across four duration regimes (81 to 1440 frames) to evaluate real-time long-video generation.

---

4. Results

• Speed: Helios-Distilled achieves 19.53 FPS on a single H100 (384x640 resolution). This is ~52x faster than Wan 2.1 14B and significantly faster than other 14B models (e.g., Krea runs at 6.7 FPS on H100).
• Quality (Short Video): On 81-frame clips, Helios matches or surpasses base models (like Wan 14B) in Semantic alignment and Naturalness, while outperforming all distilled baselines.
• Quality (Long Video): On 1440-frame clips, Helios maintains high coherence and low drifting. It outperforms the strongest baseline (Reward Forcing) in total score and significantly reduces drifting in Aesthetic, Semantic, and Naturalness metrics.
• User Study: In a side-by-side evaluation with 200 responses, Helios was preferred over existing real-time long-video methods (e.g., CausVid, Self-Forcing, Krea) for both short and long videos.
• Ablation: Removing components like Guidance Attention, First-Frame Anchor, or Frame-Aware Corrupt leads to severe degradation in stability and visual quality, confirming their necessity.

---

5. Significance

Helios represents a paradigm shift in video generation:

• Democratization: It proves that high-capacity (14B) video models can be trained and run on consumer-grade single-GPU hardware, removing the barrier of massive cluster requirements.
• Interactive World Models: By achieving real-time speeds with minute-scale coherence, Helios moves video generation closer to "interactive world models" where users can edit prompts on the fly (Interactive Interpolation) without breaking temporal consistency.
• Efficiency over Heuristics: It demonstrates that architectural innovations (compression, relative RoPE, adversarial distillation) can outperform brute-force methods (long rollouts, massive parallelism) for long-video generation.

The authors plan to release the code, base model, and distilled model to foster further community development.