💻 computer science

EcoVideo: Entropy-Orchestrated Video Generation Paradigm in Cloud-Edge Dynamics

EcoVideo is an entropy-orchestrated cloud-edge framework that dynamically selects high-entropy keyframes for cloud-based denoising and reconstructs remaining frames on the edge using motion-aware interpolation, thereby optimizing the quality-efficiency trade-off and achieving up to 2.9x speedup in video generation under resource-constrained conditions.

Original authors: Jiayu Chen, Hengyi Zhang, Maoliang Li, Minyu Li, Zihao Zheng, Xuanzhe Liu, Guojie Luo, Xiang Chen

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiayu Chen, Hengyi Zhang, Maoliang Li, Minyu Li, Zihao Zheng, Xuanzhe Liu, Guojie Luo, Xiang Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to paint a 50-frame animated movie. In the old way (the "Cloud-Only" method), a super-talented but slow artist (the Cloud) has to sit down and carefully paint every single frame from start to finish. This takes forever and costs a lot of money.

To speed things up, previous attempts at "Cloud-Edge" collaboration tried a different approach: The super-artist paints the first half of the movie, and then a less experienced, faster apprentice (the Edge) tries to finish the rest. The problem? The apprentice often messes up the details. The transition between the master's work and the apprentice's work looks jumpy, with flickering textures or characters suddenly losing their shape. It's like a movie where the lighting changes abruptly halfway through.

EcoVideo proposes a completely new way to collaborate, which the authors call "Entropy-Orchestrated Generation." Here is how it works, using simple analogies:

1. The "Entropy" Detective (Finding the Important Moments)

Instead of splitting the work by time (first half vs. second half), EcoVideo splits it by importance.

The Concept: Think of a video as a story. Some moments are boring and static (a person sitting still), while others are chaotic and full of action (a car crashing or a dancer spinning).
The Trick: EcoVideo uses a "detective" that looks at the video's "attention" (how much the AI is thinking about specific parts of the image). It calculates a score called Entropy.
- High Entropy: The scene is changing fast or is complex. This is a "Keyframe."
- Low Entropy: The scene is calm and similar to the previous frame. This is a "Redundant Frame."

2. The New Division of Labor

EcoVideo changes the rules of the game:

The Cloud (The Master Artist): Instead of painting half the movie, the Cloud only paints the Keyframes. These are the critical moments where big changes happen. Because there are fewer of these, the Cloud finishes its job much faster.
The Edge (The Smart Apprentice): The Edge device doesn't just guess the rest. It uses the Cloud's Keyframes as anchors. It fills in the "boring" frames in between using a special technique called Motion-Aware Interpolation.
- Analogy: Imagine the Cloud draws the start and end of a jump. The Edge doesn't just draw a straight line; it understands the physics of the jump and fills in the smooth arc in between, ensuring the character doesn't flicker or glitch.

3. The "Smart Traffic Controller" (Dynamic Adaptation)

The real world is messy. Sometimes your internet is slow; sometimes your phone battery is low.

The Problem: Old methods used a fixed plan. If the internet slowed down, they would still try to send the same amount of data, causing the video to freeze.
The EcoVideo Solution: It has a built-in "Traffic Controller." It constantly checks the network speed and the computer's power.
- If the internet is slow, it tells the Cloud: "Paint fewer keyframes, we need to send less data."
- If the Edge computer is weak, it tells the Cloud: "Paint more keyframes so the Edge doesn't have to work as hard."
- It constantly re-calculates the perfect balance to keep the video running smoothly without crashing.

The Results: Why It Matters

The paper tested this against the old methods using popular video AI models.

Speed: In slow internet or weak computer scenarios, EcoVideo was up to 2.9 times faster than the best existing methods.
Quality: The old methods often caused "texture flickering" (like a TV with bad signal) or "detail collapse" (where a face turns into a blob). EcoVideo kept the video smooth and stable because the Edge model wasn't trying to "invent" new details from scratch; it was just smartly filling in the gaps between the Cloud's high-quality Keyframes.
Data: It sent about 15 times less data over the network compared to the old methods, saving bandwidth.

In Summary:
EcoVideo stops trying to force a slow master and a fast apprentice to split a movie in half. Instead, it lets the master paint only the most dramatic scenes and lets the smart apprentice fill in the calm moments, all while a traffic controller adjusts the plan in real-time based on how fast the internet is. The result is a faster, smoother, and cheaper way to generate AI videos.

Technical Summary: EcoVideo

Problem Definition

Diffusion Transformer (DiT) based video generation models have achieved high visual fidelity but suffer from severe computational bottlenecks due to their inherent step-by-step full-frame denoising paradigm. This makes large-scale deployment difficult, particularly in latency-sensitive scenarios.

Recent attempts to alleviate this via cloud-edge collaboration (e.g., HybridSD, EC-Diff) utilize inter-step decoupling, where a large cloud model performs early denoising steps to establish semantics, and a small edge model completes the remaining refinement steps. The authors identify three critical limitations in this paradigm:

Temporal Redundancy: Inter-step decoupling assumes all frames are equally important, leading to redundant computation even when adjacent frames are highly similar.
Quality Degradation: Small edge models often lack the capacity to perform late-stage denoising effectively, resulting in artifacts like texture flickering and detail collapse. Furthermore, switching models mid-trajectory introduces prediction gaps that break temporal coherence.
Static Adaptation: Existing methods use fixed workload splitting points. They cannot adapt to dynamic real-world conditions (fluctuating bandwidth, variable cloud/edge compute), often leading to unstable or degraded end-to-end efficiency.

Methodology: EcoVideo

The authors propose EcoVideo, an entropy-orchestrated framework that shifts the collaboration paradigm from static inter-step decoupling to dynamic inter-frame decoupling. The system consists of three core components:

1. Inter-frame Attention Entropy Analysis

To identify temporal redundancy without training, EcoVideo analyzes self-attention entropy during the early denoising steps (warm-up stage).

Mechanism: Token-level attention entropy is aggregated to the frame level using an Exponential Moving Average (EMA) over the first 10% of denoising steps.
Function: High entropy indicates complex dynamics and high information density (candidates for keyframes), while low entropy indicates temporal redundancy (candidates for interpolation). This provides a training-free signal to estimate frame-wise information density.

2. Frame-level Entropy-Orchestrated Generation

Based on the entropy signal, the generation process is split into two collaborative stages:

Cloud-side (Large Model): The large model (e.g., Wan2.1-14B) generates only a sparse set of high-entropy keyframes (including the first and last frames). To preserve global temporal context, the model injects frozen non-keyframe latents as context tokens during attention computation. This allows the cloud model to anchor the motion skeleton without processing every frame.
Edge-side (Small Model): A lightweight model (EcoVFI-160M) reconstructs the remaining low-entropy intermediate frames.
- Information Reuse: The edge model reuses cues from the cloud-generated keyframes (motion, structure, texture, and entropy density) to guide interpolation.
- Greedy Refinement: Instead of uniform interpolation, EcoVideo employs an entropy-aware greedy strategy. It calculates a "difficulty score" for each interval between keyframes based on optical flow, entropy gaps, and feature distances. It then iteratively inserts intermediate frames into the most challenging intervals to minimize temporal artifacts like ghosting and blur.

3. Cloud-Edge Dynamics-aware Decoupling Adaptation

To handle system variability, EcoVideo includes a scheduler that dynamically adjusts two knobs: the keyframe budget ( $K$ ) and the interpolation depth ( $D$ ).

Adaptation Logic: The scheduler monitors real-time bandwidth and compute resources. It solves a discrete optimization problem to maximize a utility function (balancing quality and latency) subject to resource constraints.
Outcome: This enables closed-loop optimization, ensuring the system remains efficient even when network bandwidth drops or edge compute becomes contended.

Key Contributions

Paradigm Shift: The paper introduces the first inter-frame decoupling paradigm for cloud-edge video generation, moving from step-level model switching to frame-level workload partitioning that exploits temporal redundancy.
Training-Free Entropy Analysis: A novel module that estimates frame-wise information density using attention entropy, enabling the identification of critical keyframes without additional training.
Dynamic Orchestration Framework: A generation framework that preserves temporal consistency through model collaboration and information reuse, coupled with a system-aware adaptation mechanism that achieves up to 2.9× end-to-end speedup in constrained environments.

Experimental Results

The authors evaluated EcoVideo on mainstream DiT models (Wan2.1, Wan2.2, CogVideoX) using the VBench benchmark.

Quality: EcoVideo consistently outperforms step-splitting baselines (HybridSD, EC-Diff) in VBench Overall scores (average improvement of 0.084). It significantly mitigates temporal artifacts such as texture flickering and detail collapse, maintaining better subject and background consistency.
Efficiency:
- Communication: EcoVideo drastically reduces data transmission. For example, under the Wan2.1-14B setting, communication volume dropped from 17.23 MB (baselines) to 1.10 MB.
- Latency: In low-bandwidth, compute-limited edge scenarios, EcoVideo achieves up to 2.9× end-to-end speedup compared to cloud-only generation, whereas baselines often degrade below cloud-only performance under resource contention.
Robustness: Ablation studies confirm that the combination of entropy-based keyframe selection, model collaboration, and dynamic adaptation is essential for achieving the best quality-efficiency trade-off.

Significance and Claims

The paper claims that EcoVideo addresses the fundamental inefficiencies of current cloud-edge video generation by recognizing that inter-frame sparsity exists and can be exploited. By shifting from a static, step-based division of labor to a dynamic, frame-based orchestration driven by information density, EcoVideo achieves:

Reduced Redundancy: Avoiding the generation of similar frames by the large model.
Improved Coherence: Preventing the temporal artifacts caused by model switching and insufficient edge refinement.
System Resilience: Adapting to dynamic network and compute conditions to maintain stable performance.

The authors position EcoVideo as a practical direction for robust video generation in cloud-edge systems, noting that it is orthogonal to other acceleration techniques (like parallelism or caching) and can be combined with them. They acknowledge limitations in handling extremely fast motion or heavy occlusion without additional keyframes but frame the current work as a significant step toward efficient, dynamic video generation services.