💻 computer science

Visko Orbis 1.0: A Live Model for Real-Time Interactive Long Video Generation

Visko Orbis 1.0 is a live, interactive model capable of generating real-time, hour-long 4K videos at 24 FPS that allow for dynamic prompt switching and maintain consistent quality and style through a bounded multi-scale memory system.

Original authors: Xiangbo Gao, Siyuan Yang, Ping He, Mingyang Wu, Yuheng Wu, Yushen Zuo, Jiongze Yu, Ryan Cui, Hongyuan Hua, Devin Ma, Xiao Jin, Yubo Yuan, Qing Yin, Jie Yang, Zhengzhong Tu

Published 2026-07-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xiangbo Gao, Siyuan Yang, Ping He, Mingyang Wu, Yuheng Wu, Yushen Zuo, Jiongze Yu, Ryan Cui, Hongyuan Hua, Devin Ma, Xiao Jin, Yubo Yuan, Qing Yin, Jie Yang, Zhengzhong Tu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a movie, but instead of sitting back and letting the story play out, you are the director, the scriptwriter, and the lead actor all rolled into one. You can shout "Cut!" and change the plot mid-scene, and the movie instantly updates the next scene based on your new idea. This is the dream of real-time interactive video generation. For a long time, making videos with computers was like baking a cake: you mixed the ingredients (the prompt), waited for it to bake (the generation), and once it was out of the oven, it was done. If you wanted to add more chocolate chips, you had to start over. But what if you could reach into the oven while it was baking and change the recipe, and the cake would instantly adjust? That is the world of "Live Models." These are computer programs that don't just answer a question and stop; they keep running, like a persistent character in a video game, ready to react to your commands the second you give them. The big challenge has always been keeping the story consistent. If you change the story halfway through, does the character forget who they are? Does the background turn into a blurry mess? Does the video get stuck in a loop? Scientists have been trying to build a system that can handle these long, changing stories without losing its mind or its style.

Enter Visko Orbis 1.0, a new system that acts like a super-smart, tireless video director who never sleeps. Think of it as a magical storyteller that can spin a video tale for an hour straight, and if you suddenly decide, "Actually, make the sky purple and the dog fly," the video doesn't crash or restart. Instead, it seamlessly weaves your new idea into the ongoing story, showing the change in less than a second. The team behind this built a system that doesn't just make short, 5-second clips; it builds a "live stream" of video that can go on for hours. They call it a "Live Model" because it stays alive and active, carrying the memory of what happened in the last minute and the last hour, so the characters and scenery stay consistent even as the plot twists and turns.

So, how does this magic trick work? Imagine the video isn't made all at once, but in tiny, invisible Lego blocks called "chunks." The system builds one chunk, then the next, and the next. The secret sauce is a special "memory bank" that holds onto the important details—like the color of the main character's shirt or the style of the forest—without getting clogged up. As the video gets longer, the system compresses the older, distant memories (like "what happened 30 minutes ago") into a summary, while keeping the recent past (the last few seconds) in high definition. This lets the video run for hours without the computer getting confused or the colors drifting into weird shades of green.

The paper shows that this system can generate 4K video (which is super sharp and clear) at 24 frames per second (the standard speed for smooth movies) in real time. That means you can watch the video appear on your screen as it is being made, and you can type a new prompt at any moment—whether you are making a cartoon, a live stream, or a virtual companion—and the video will update instantly. The researchers tested this against other top systems and found that Visko Orbis 1.0 was the best at keeping the video looking good, following instructions, and staying stable over long periods. In fact, when humans played a game where they had to pick the best long video from a bunch of options, they chose Visko Orbis 1.0 more often than any other system, especially praising how steady and reliable it was over time.

The team didn't just build the engine; they also taught it how to learn from a massive library of videos, filtering out the bad ones and organizing the good ones so the AI could learn to tell stories that make sense. They even added a special "physics check" to make sure that when a ball is thrown, it falls down like a real ball, not like a floating balloon. The result is a tool that turns video generation from a static, one-time task into a living, breathing conversation between a human and a machine. It's not just about making a video; it's about keeping a story alive, ready to change direction whenever you want, without ever losing the thread.

Technical Summary: Visko Orbis 1.0

Problem Statement

Current video generation systems (e.g., Sora, Veo, Kling) predominantly operate in an offline paradigm: a user submits a prompt, waits for processing, and receives a fixed clip that cannot be altered. While these systems excel at visual fidelity and short-form generation, they fail to support live, interactive experiences where video is consumed while being created (e.g., live streaming, interactive storytelling, virtual companions). Existing attempts at streaming or interactive video often address only isolated aspects—such as causal streaming, limited interaction, or long-horizon stability—but lack an end-to-end system that simultaneously supports real-time user control, sustained long-form generation without quality drift, and progressive delivery. Furthermore, serving such models in real-time requires solving inference bottlenecks related to state reuse, parallel execution, and progressive upscaling.

Methodology

Visko Orbis 1.0 is formulated as a Live Model: a persistent process that generates video in real-time under a continuous clock, accepting updates without reinitialization. The system is built upon a chunk-wise latent-video model that generates successive temporal chunks while maintaining a bounded multi-scale memory of the preceding rollout.

1. Data Curation and Training Pipeline

The training pipeline utilizes a multi-stage data engine to filter, annotate, and rebalance a large-scale raw video collection:

Data Processing: Raw videos undergo shot-aware clipping (3–240s), layered safety filtering, and cascaded quality filtering (assessing structural integrity, temporal coherence, and motion). Distribution rebalancing ensures coverage across diverse domains (nature, urban, industrial, gaming) to prevent model collapse into narrow visual styles.
Captioning: A multi-stage captioning system generates single-event captions for holistic supervision and event-aligned temporal captions that bound actions to specific intervals, enabling the model to handle prompt switching within a single video.
Progressive Training:
- Bidirectional Pretraining: Trained on 1–5s videos to establish a strong visual prior.
- Streaming Adaptation: The model is adapted to causal, chunk-wise generation on 3–10s videos using a linear rectified-flow objective.
- Mid-Training: Specialized on single- and multi-event data to improve motion naturalness and temporal coherence.
- Fine-Tuning: High-quality, human-curated subsets are used to refine visual fidelity and event transitions.
- Post-Training: Includes few-step flow post-training via distillation (removing the unconditional branch) and Group-Relative Reinforcement Learning (GRPO) to optimize visual quality, motion, and text-video alignment. A latent world model is used to score candidate futures for physical plausibility.

2. Model Architecture and Memory

Formulation: The model generates chunks $z_k$ conditioned on text instructions $c_k$ , visual references $r_k$ , and a bounded history $H_k$ . Instructions are treated as external controls; updates affect only uncommitted chunks.
Bounded Multi-Scale Memory: To sustain hour-scale rollouts, the system retains recent history at native latent resolution while progressively compressing older spans into a fixed-capacity learned state. This prevents error accumulation and color drift while keeping per-chunk computation constant.
Super-Resolution: A dedicated single-refinement video super-resolution model transforms native 832 × 480 chunks into 4K output. It employs temporally distilled autoencoders and spatial-window attention to preserve motion consistency without introducing autoregressive dependencies between chunks.

3. Inference and Serving

The inference stack is designed for progressive delivery and real-time interaction:

Live Control: Prompts are encoded asynchronously and applied at chunk boundaries. A rolling prompt summary maintains context across interactions.
Optimized Execution: The system utilizes compiled transformer execution, fused kernels, and full-sequence multi-GPU execution (using Ulysses-style sequence sharding) to minimize latency.
Drift Stabilization: Content-adaptive controls (e.g., additive perturbation, temporal RoPE scaling, and motion-aware negative-prompt routing) are applied to stabilize long rollouts against accumulated errors.
Output: The pipeline delivers 4K video at 24 FPS with an average visible response to prompt updates of under one second.

Key Contributions

Unified Live-Video Formulation: A persistent, chunk-wise latent-flow process where visual state is carried across chunks, allowing time-indexed instructions to change without restarting the rollout. This supports T2V, I2V, and V2V continuation under a single interface.
Event-Aligned Data and Training: A multi-stage data engine producing temporally localized captions and a curriculum that moves from short-video priors to history-conditioned streaming, including specific training for prompt switching and long-horizon stability.
Bounded Memory for Long-Horizon Generation: A mechanism that retains recent history at high resolution while compressing older data into a fixed state, enabling hour-scale rollouts without evident quality or color drift.
Few-Step Flow Post-Training: A pipeline combining guidance distillation, self-forcing distribution matching, and GRPO-based reinforcement learning to align the model with video-quality rewards and physical plausibility.
Streaming High-Resolution System Co-Design: An optimized serving pipeline integrating state reuse, batch-one sequence parallelism, and a reference-aware super-resolution model to achieve real-time 4K delivery.

Results

The paper evaluates Visko Orbis 1.0 on 74 cases involving 1–3 minute videos with multiple event switches, comparing it against state-of-the-art systems like Helios, Self-Forcing, LongLive, and Causal-Forcing.

Quantitative Metrics:
- DOVER: Achieved the highest aesthetic (0.8101) and technical (0.5572) scores.
- VideoAlign: Secured the best visual quality (1.5777) and motion quality (1.8646) scores, along with the highest text-alignment logit (0.1043).
- Human Preference (HPSv3): Recorded a frame mean of 8.1018 and a minimum of 6.9719.
Human Preference Arena: In a side-by-side study across 9 systems, Visko Orbis 1.0 achieved the highest Overall Elo (1838) and the highest Temporal Stability Elo (1940). While other systems (e.g., HappyOyster) showed competitive visual fidelity, Orbis demonstrated a distinct advantage in maintaining coherent long-form viewing experiences and handling instruction switches.
Long-Horizon Consistency: Figure 1 illustrates that over a 3600-second (1-hour) rollout, the system maintains stable aesthetic quality, prompt alignment, and color saturation (HSV trajectories) compared to baseline systems which exhibit drift.

Significance and Claims

The paper positions Visko Orbis 1.0 as the first realization of the Live Model paradigm for video generation. It claims to shift the boundary of video generation from producing bounded, static clips to sustaining a controllable visual stream. By integrating stateful generation, event-aligned training, and efficient real-time serving, the system enables users to steer video generation continuously as it unfolds. The authors assert that this work advances video generation toward a responsive creative medium, supporting applications ranging from live streaming and interactive storytelling to robotics and virtual companionship, all while maintaining high visual fidelity and temporal stability over extended durations.