💻 computer science

Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation

This paper proposes a training-free, three-stage pipeline framework that preserves subject identity in sequential-action video generation by rewriting prompts for terminal states, generating keyframes through joint conditioning on reference identity and predecessor frames, and enhancing intermediate segments with multi-reference guidance and identity-driven noise searching.

Original authors: Zhenjie Liu, Binyan Chen, Hao Chen, Tong Pan, Shangfei Wang

Published 2026-07-21

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhenjie Liu, Binyan Chen, Hao Chen, Tong Pan, Shangfei Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a director trying to film a movie with a very special, stubborn actor. This actor is a digital character created by a computer program called a "diffusion model." Think of this model as a magical, chaotic artist who is incredibly good at painting pictures based on your descriptions, but it has a terrible memory. If you ask it to draw a picture of a "man with a beard reading a book," it might draw a perfect man. But if you then ask it to draw the same man standing up and waving, the artist might get confused. The beard might disappear, the face might change shape, or the man might suddenly look like a completely different person. This is the problem of "identity drift." In the world of AI video generation, keeping a character looking exactly the same while they move and change actions is like trying to keep a snowflake from melting while you walk it across a hot sidewalk.

Now, imagine you don't just want the actor to do one thing; you want them to perform a whole script. First, they read a book. Then, they look up at the sky. Then, they look back down. Doing this in a single, continuous burst is a nightmare for the AI because the longer the video gets, the more the character's face tends to "drift" and change. This paper tackles a specific challenge where the goal is to generate a video of a specific person performing a sequence of different actions, all while ensuring the person in the video looks exactly like the reference photo you started with. The researchers from the University of Science and Technology of China decided to stop trying to film the whole movie in one take. Instead, they came up with a clever way to break the movie down into manageable scenes, anchor the character's face at the end of each scene, and then stitch the pieces together so the actor never forgets who they are.

The Three-Stage Magic Trick

The authors propose a "training-free" pipeline, which is a fancy way of saying they didn't need to re-teach the AI how to draw; they just needed to give it better instructions and a better workflow. They call their method "Keyframe-Anchored Identity Preservation," which sounds complicated but is actually quite logical. Think of it like building a bridge across a river. Instead of trying to lay the whole road at once (which might collapse), you build strong, solid pillars (keyframes) at specific points and then fill in the road between them.

Stage 1: The Script Doctor (Action-Aware Prompt Polishment)
First, the team realized that the instructions given to the AI were too vague. The original prompts were like a director shouting, "Okay, now he reads, then he looks up!" to an artist who needs a clear picture of the end result of each action. The AI was getting confused about when to stop reading and how to look up.

To fix this, the researchers used a smart AI assistant (a Large Language Model) to rewrite the script. Instead of describing the action of reading, the assistant rewrote the prompt to describe the state of the character after reading. It's the difference between saying "The man is turning his head" and "The man is now looking to the left." By focusing on the "terminal state" (the final pose) of each action, the AI gets a clear target to aim for. This ensures that when the AI generates the "keyframe" (the anchor point), it knows exactly what the character should look like at that specific moment.

Stage 2: The Chain Reaction (ID-Preserving Keyframe Generation)
Next comes the actual drawing of the anchor points. The researchers didn't ask the AI to draw the whole video at once. Instead, they asked it to draw a chain of still images, one after another.

To draw the first image, they showed the AI the reference photo (the original person) and the first rewritten prompt.
To draw the second image, they showed the AI the same reference photo (to keep the face the same) AND the first image they just made (to keep the body position continuous).
They repeated this for every action in the script.

This is the "chained" part. By feeding the previous image into the next step, the AI knows how to move the body naturally without losing the connection to the original face. It's like a game of "telephone" where you whisper the instructions, but instead of the message getting garbled, you are passing along a physical sketch that ensures the next artist knows exactly where the last one left off. This separates the "identity" (the face, which stays the same) from the "pose" (the body, which changes), solving the drift problem.

Stage 3: The Safety Net (ID-Aware Inference Enhancement)
Finally, the AI has a series of still images (the keyframes), but it needs to fill in the movement between them to make a smooth video. This is where the magic happens. The researchers added two special tricks to the video generation process to make sure the character doesn't slip up during the movement.

Multi-Reference Guidance: Imagine you are trying to draw a picture while someone is whispering three things in your ear: the text description, the previous frame, and the original photo. The AI usually listens to the text and the previous frame. The researchers forced the AI to listen extra hard to the original photo. They tweaked the math so that the "identity" signal was amplified, making it much harder for the AI to accidentally change the character's face while moving the body.
Noise Searching: When AI generates video, it starts with a screen full of static noise (like TV snow) and slowly cleans it up to reveal the image. Usually, the AI just picks a random patch of snow to start with. The researchers decided to be picky. They generated several different patches of "snow" and tested them quickly to see which one would result in the best-looking face. They picked the winner and used that as the starting point. It's like trying out different seeds to see which one grows the best flower before you plant the whole garden.

The Results: Did It Work?

The team tested their method in a competition called IPVG26 (Track 2), where the goal was to generate videos of specific people performing sequences of actions. They didn't just guess; they measured the results against other top teams.

Their approach ranked third on the official leaderboard. While they didn't take first place, the results were impressive, especially in the areas that mattered most: keeping the person's face recognizable (Identity Preservation) and making sure the actions happened at the right time (Temporal Alignment).

When they tested their system, they found that the "Multi-Reference Guidance" was the heavy lifter. When they removed this feature, the score for keeping the identity consistent dropped significantly (from 0.4055 to 0.3520). This proved that explicitly telling the AI to focus on the reference photo was the key to stopping the face from drifting. The "Noise Searching" helped too, but it was a smaller boost.

The paper also compared two different "backbones" (the underlying video engines they used). They found that a model called LTX-2.3 performed better than another model called Wan2.1-VACE across almost every metric. The LTX-2.3 model achieved an overall score of 0.4971, compared to 0.4818 for the other model.

What This Means

The paper doesn't claim to have solved the problem of AI video generation forever. It doesn't say the character will never change, or that the method works perfectly for every single possible scenario. Instead, it suggests that breaking a complex video task into smaller, anchored steps is a very effective way to handle it.

By treating the video as a chain of "terminal states" (the end of each action) rather than a continuous, flowing stream, the researchers managed to keep the character's identity intact even when the actions were complex and sequential. They showed that you don't always need to retrain the AI from scratch; sometimes, you just need to give it a better map, a stronger anchor, and a little bit of extra help when it's choosing where to start. For anyone trying to create digital stories with consistent characters, this "keyframe-anchored" approach offers a reliable, training-free way to keep the actor in the role, no matter how many scenes they have to perform.

Technical Summary: Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation

Problem Definition

The paper addresses the challenge of Identity-Preserving Text-to-Video Generation (IPVG) within the context of the IPVG26 Challenge Track 2. Unlike traditional IPVG tasks that rely on a single holistic prompt, this setting requires generating a video based on:

A reference identity image ( $I_{ref}$ ).
A global scene description ( $P_{global}$ ).
A temporally structured sequence of timestamped action captions $\{(s_i, e_i, c_i)\}_{i=1}^N$ , where each $c_i$ describes a specific sub-action with start and end times.

The core difficulty lies in maintaining the subject's recognizability (identity) while executing a scripted sequence of distinct actions. Standard end-to-end video generators often suffer from appearance drift as motion accumulates and the depicted actions change, causing the subject to lose likeness over the course of a multi-action sequence.

Methodology

The authors propose a training-free, three-stage pipeline that decomposes the video generation task into a chain of identity-preserving keyframes, avoiding the spatiotemporal trade-offs of joint optimization.

1. Action-Aware Prompt Polishment

The raw textual inputs (global description and temporal captions) are often narrative and lack the spatial precision required for image generation models.

Process: A multimodal Large Language Model (LLM), specifically Qwen3.6-27B, rewrites the inputs.
First Frame: The global prompt is rewritten into a prompt ( $\hat{P}_{global}$ ) that anchors the subject's identity attributes (face shape, skin tone, etc.) and establishes the scene, ensuring a plausible opening moment.
Terminal States: Each temporal caption $c_i$ is rewritten into an end-frame prompt ( $\hat{c}_i$ ). Crucially, these prompts describe the terminal state of the action (final pose, hand position, gaze) rather than the progression of motion. This provides unambiguous spatial constraints for the subsequent keyframe generation.

2. ID-Preserving Chained Keyframe Generation

Instead of generating the full video at once, the system generates a sequence of keyframes $\{K_0, K_1, \dots, K_N\}$ , where each $K_i$ represents the terminal state of the $i$ -th action segment.

Chained Conditioning:
- $K_0$ is generated conditioned on $I_{ref}$ and $\hat{P}_{global}$ .
- Subsequent keyframes $K_i$ $K_{i}$ are generated conditioned on three inputs:
  1. $I_{ref}$ : Provides time-invariant identity features.
  2. $K_{i-1}$ : Provides the preceding visual state (pose, orientation, scene layout) to ensure temporal continuity.
  3. $\hat{c}_i$ : Specifies the target terminal state for the current action.
Mechanism: This design decouples time-invariant appearance (from $I_{ref}$ ) from time-varying pose (from $K_{i-1}$ ), allowing the model to modify only the aspects described in the new prompt while inheriting the rest.

3. ID-Aware Inference Enhancement

Once the keyframes are established, the video segments between them are synthesized using a diffusion-based video model (specifically LTX-2.3 in keyframe-interpolation mode). Two training-free enhancements are applied during inference:

Multi-Reference Guidance: Extends standard Classifier-Free Guidance (CFG) by adding specific guidance terms. The sampling trajectory is guided by:
- $\Delta_{cfg}$ : Standard text alignment.
- $\Delta_{stg}$ : Spatiotemporal guidance (via attention perturbation) for temporal coherence.
- $\Delta_{mod}$ : Modality guidance that explicitly amplifies the influence of the visual conditions (boundary keyframes) to enforce identity consistency.
Identity-Driven Noise Searching: Treats the initial diffusion noise $z_T$ as an optimization variable rather than a random draw. For each segment, $K$ candidate noise tensors are sampled, and a short lookahead denoising is performed. The candidate yielding the highest reward (cosine similarity between the decoded first frame's face embedding and $I_{ref}$ ) is selected for full denoising.
Variable-Length Allocation: Segments are allocated frame counts proportional to their duration, rounded to satisfy VAE constraints ( $8k+1$ ).

Key Contributions

Action-Aware Prompt Polishment Module: A method to rewrite global and temporal captions into terminal-state image prompts, providing clear spatial targets for keyframe synthesis.
ID-Preserving Chained Keyframe Generation: A generation strategy that conditions each keyframe on both the reference identity and the preceding keyframe, effectively separating identity from pose to maintain recognizability and motion continuity.
ID-Aware Inference Enhancement: A training-free stage integrating multi-reference guidance and identity-driven noise searching to reinforce identity fidelity during sampling.
Keyframe-Anchored Decomposition: A paradigm shift from end-to-end generation to interpolating between identity-anchored endpoints, addressing the specific challenges of sequential-action tasks.

Results

The proposed method was evaluated on the IPVG26 Track 2 test set (200 samples from the ReactID dataset).

Leaderboard Performance: The method ranked 3rd on the official leaderboard (Score: 2.9375), behind USTC-CMI and WislabGDUT.
Backbone Comparison: The authors compared LTX-2.3 (keyframe-interpolation mode) against Wan2.1-VACE (first/last-frame mode). LTX-2.3 outperformed Wan2.1-VACE across all metrics, particularly in identity preservation and alignment, leading to its selection as the final backbone.
Ablation Studies:
- Removing Multi-Reference Guidance caused the most significant drop in identity score (from 0.4055 to 0.3520), confirming it as the primary driver of subject consistency.
- Removing Noise Searching resulted in a moderate but consistent decline in identity and overall scores.
- Both enhancements improved identity metrics without significantly affecting motion smoothness or image quality.

Significance and Claims

The paper claims that the keyframe-anchored decomposition is an effective solution for the specific challenge of sequential-action identity preservation. By decoupling the time-invariant identity layout from temporal motion, the method avoids the appearance drift common in end-to-end generators. The authors emphasize that their approach is training-free, requiring no fine-tuning of off-the-shelf models, yet it achieves competitive performance on a challenging benchmark. The results suggest that explicitly separating the generation of terminal states (keyframes) from the interpolation of motion, combined with inference-time identity reinforcement, is a viable strategy for controllable, multi-action video synthesis.