💻 computer science

EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration

EverAnimate is an efficient post-training method that enables minute-scale human animation by combining persistent latent propagation and restorative flow matching to anchor generation in a latent context memory, thereby eliminating accumulated visual and semantic drift while preserving character identity and background quality.

Original authors: Wuyang Li, Yang Gao, Mariam Hassan, Lan Feng, Wentao Pan, Po-Chien Luan, Alexandre Alahi

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wuyang Li, Yang Gao, Mariam Hassan, Lan Feng, Wentao Pan, Po-Chien Luan, Alexandre Alahi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to film a long, continuous movie of a dancer performing a complex routine. You have a photo of the dancer (the reference) and a script of their moves (the poses). Your goal is to generate a video that follows the script perfectly while keeping the dancer looking exactly like the person in the photo, even after 90 seconds of dancing.

The paper EverAnimate tackles a specific problem: when AI tries to make these long videos, they tend to "drift" apart, much like a copy of a copy of a copy eventually becomes blurry and unrecognizable.

Here is how the paper explains the problem and its solution, using simple analogies:

The Problem: The "Copy-Paste" Glitch

Current methods try to make long videos by stitching together short clips (chunks). Imagine you are copying a paragraph of text, then copying that copy to make the next paragraph, and so on.

The Background Drift (Low-Level): The background (like a wall or a tree) is supposed to stay still. But because the AI keeps re-copying the image to start the next clip, the wall starts to look fuzzy, the colors shift, and eventually, it looks like a bad photocopy.
The Identity Drift (High-Level): The dancer is supposed to look the same. But as the video gets longer, the dancer's face might slowly change shape, their clothes might change color, or their hair might look different. They lose their "identity."

The paper argues that existing methods try to fix this by showing the AI the original photo over and over again (like a "sink" or anchor), but this isn't enough. The AI still gets confused by the repeated copying process.

The Solution: EverAnimate

The authors propose a new way to generate these videos that happens entirely inside the AI's "brain" (its latent space) rather than by re-photographing the video. They use two main tricks:

1. The "Persistent Backpack" (Persistent Latent Propagation)

Instead of taking the video output, turning it back into a picture, and then feeding that picture back into the AI for the next clip (which causes the "copy-paste" errors), EverAnimate keeps a backpack of memory inside the AI.

How it works: When the AI finishes one chunk of the video, it doesn't look at the final picture. Instead, it passes a "backpack" of raw data (latent codes) to the next chunk.
What's in the backpack?
- Short-term memory: The last few seconds of movement to keep the dance smooth.
- Long-term memory: A collection of "ID cards" (multi-view images) of the dancer's face and clothes to ensure they never change their look.
The Analogy: Imagine a relay race. Instead of the runner handing you a blurry photo of the previous runner to copy, they hand you a direct, high-quality data chip that tells you exactly how to continue the race without losing the original runner's features.

2. The "Self-Correcting Compass" (Restorative Flow Matching)

Even with a good backpack, the AI might occasionally take a wrong turn during the generation of a single clip.

The Problem: If the AI starts to drift slightly off-course (e.g., the dancer's arm looks a bit weird), standard methods just keep going, making the error worse.
The Fix: EverAnimate trains the AI with a special "compass." During training, the AI is intentionally given a slightly "broken" or distorted path to follow. It learns to recognize that it's off-track and actively steers itself back to the correct, clean path.
The Analogy: Think of a hiker walking in fog. A normal hiker might wander off a cliff because they can't see the path. EverAnimate is like a hiker with a GPS that vibrates whenever they step off the trail, gently nudging them back to the safe path before they get lost.

The Results

The paper claims that by using these two methods (the memory backpack and the self-correcting compass), EverAnimate can generate videos that are minutes long (up to 90 seconds in their tests) without the background getting blurry or the character changing faces.

Short videos (10 seconds): It is already better than the best existing methods.
Long videos (90 seconds): The improvement is massive. The video stays sharp, the background stays stable, and the character looks exactly the same from start to finish.

Summary

In short, EverAnimate stops the AI from making "copies of copies" of a video. Instead, it keeps a high-quality digital memory of the character and the scene, and it trains the AI to fix its own mistakes as it goes, allowing for smooth, high-quality, minute-long animations that don't fall apart.

Technical Summary of EverAnimate: Minute-Scale Human Animation via Latent Flow Restoration

1. Problem Statement

The paper addresses the challenge of generating long-horizon, pose-guided human animation videos (minute-scale) while maintaining high visual fidelity and consistent character identity. Existing methods, which typically rely on autoregressive, chunk-wise generation, suffer from accumulated drift over time. The authors identify two distinct types of drift:

Low-level Quality Drift: Progressive degradation of static backgrounds and textures caused by repeated decoding and re-encoding (VAE round-trips) of carry-over frames between chunks.
High-level Semantic Drift: Inconsistencies in character identity, facial appearance, and clothing details, particularly in regions undergoing significant motion.

The authors argue that current solutions, such as attention sinks (using a reference frame as a global anchor) or error recycling, are insufficient. Attention sinks act as passive signals that specify what to preserve but fail to actively correct trajectory deviations, while repeated image-space continuation inevitably damages visual details in static regions.

2. Methodology: EverAnimate

EverAnimate is an efficient post-training framework designed to stabilize long-form generation by operating directly in latent space rather than image space. It introduces two complementary mechanisms:

A. Persistent Latent Propagation (PLP)
Instead of decoding the previous chunk's video, extracting the last frame, and re-encoding it (the standard carry-over pipeline), EverAnimate propagates latent memory directly across chunks.

Motion Memory ( $M_{mot}$ ): Preserves short-term temporal continuity by propagating the last $r$ latent slices from the previous chunk.
Identity Memory ( $M_{id}$ ): Anchors multi-view identity across all chunks. This is constructed by encoding a small set of sampled frames from the context chunk.
Memory Augmentation: To prevent spatial bias where the memory context inadvertently dictates the spatial layout of the generated frame, the authors apply mild spatial augmentations (random translation and rescaling) to the identity memory during training.
Injection: These memories are concatenated with pose-guided target latents to form the input for the Diffusion Transformer (DiT), ensuring the model conditions on persistent semantic information without destructive image-space reconstruction.

B. Restorative Flow Matching (RFM)
To address drift occurring within a chunk, EverAnimate introduces an implicit restoration objective during the sampling process.

Concept: Standard Flow Matching (FM) trains the model to transport Gaussian noise to a clean data manifold. However, in long-horizon generation, the model may encounter perturbed states due to accumulated errors.
Mechanism: During training, the target endpoint is perturbed ( $\tilde{X}_1$ ) to simulate drift. The model is then trained to predict a velocity field that not only follows the clean path but also actively corrects the trajectory from the perturbed state back to the clean endpoint.
Velocity Adjustment: The target velocity is adjusted to include a restoration term: $\tilde{U}_t = U_t + \lambda(t)(X_t - \tilde{X}_t)$ .
Rescheduling: To prevent training instability (where the correction term becomes disproportionately large near the clean endpoint), a bounded time weight $\lambda(t)$ (a bell-shaped schedule) is applied. This ensures restoration is strongest in intermediate noise regions where perturbations are informative, while avoiding over-correction in high-noise or low-noise extremes.

Training and Inference
The framework uses lightweight LoRA tuning on a pre-trained DiT backbone (Wan-2.2-Animate). Training occurs in two stages: (1) Memory adaptation to learn the latent propagation, and (2) Anti-drift adaptation using the Restorative Flow Matching loss. During inference, the model rolls out chunk-by-chunk in latent space, reusing latent slices for the next chunk without decoding intermediate frames.

3. Key Contributions

Problem Identification: The authors empirically reveal that image-space continuation and attention sinks are insufficient for minute-scale animation, identifying specific limitations in how current methods handle VAE round-trip degradation and semantic anchoring.
Latent-Space Principle: They derive a principle for stable long-horizon generation: semantic memory must be propagated autoregressively in latent space, and drift must be corrected via intrinsic restoration capabilities.
EverAnimate Framework: A post-training method comprising Persistent Latent Propagation (for cross-chunk continuity) and Restorative Flow Matching (for within-chunk fidelity), which requires only lightweight LoRA tuning.

4. Experimental Results

The method is evaluated on a dataset combining Champ, UBC, Seedance, and self-collected YouTube videos, benchmarked at 10s, 30s, 60s, and 90s durations.

Quantitative Performance: EverAnimate outperforms state-of-the-art methods (including Wan-Animate, UniAnimate-DiT, and SteadyDancer) across all metrics.
- At 10 seconds, it improves PSNR by 8% and SSIM by 7% over Wan-Animate, while reducing LPIPS by 22% and FID by 11%.
- At 90 seconds, the gains increase significantly: PSNR/SSIM improve by 15% each, and LPIPS/FID reduce by 32% and 27%, respectively.
- It achieves the highest Face-PSNR (F-PSNR), indicating superior identity consistency.
Qualitative Performance: Visual comparisons show that while baselines suffer from background degradation and identity flickering over time, EverAnimate maintains stable backgrounds and consistent character identity throughout minute-long sequences.
Ablation Studies: Removing Restorative Flow Matching (RFM) significantly lowers perceptual quality, while removing Persistent Latent Propagation (PLP) weakens cross-chunk continuity, confirming the necessity of both components.

5. Significance and Claims

The paper claims that EverAnimate represents a shift from image-space continuation to latent-state control for long-form animation. By decoupling the propagation of semantic memory from the destructive cycle of image decoding/re-encoding, and by equipping the model with an intrinsic ability to correct trajectory drift, EverAnimate enables the generation of minute-scale videos that were previously unattainable with high fidelity. The authors position this as a solution to the specific train-test asymmetry in pose-driven animation, where long-horizon generation fails due to accumulated errors in both motion structure and visual synthesis.