🤖 AI

ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation

The paper introduces Argus, a Wan-based framework that overcomes the limitations of point-reference paradigms in subject-preserving video generation by employing Stacked Multi-View Identity Mosaic Injection (SMII) and counterfactual self-supervision to achieve state-of-the-art robustness across diverse motions, viewpoint changes, and occlusions.

Original authors: Zijie Meng, Jiwen Liu, Yufei Liu, Chengzhuo Tong, Xiaoqiang Liu, Yuanxing Zhang, Yulong Xu, Pengfei Wan

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zijie Meng, Jiwen Liu, Yufei Liu, Chengzhuo Tong, Xiaoqiang Liu, Yuanxing Zhang, Yulong Xu, Pengfei Wan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to draw a video of your best friend, "Alex," based on a single photo and a description like "Alex walking a dog."

The old way of doing this was like handing the robot one single snapshot of Alex. The robot would look at that one photo and try to copy it perfectly. But here's the problem: if the photo shows Alex wearing sunglasses on a sunny day, the robot might get confused and think "sunglasses" and "sunlight" are part of Alex's face. If you ask the robot to show Alex walking in the rain or turning his head to the side, the robot often fails. It either forgets what Alex looks like, or it stubbornly keeps the sunglasses on even when they shouldn't be there. It treats Alex's identity as a single, frozen point in time.

Argus is a new system that changes the game. Instead of giving the robot one photo, Argus gives it a dynamic memory bank.

Here is how it works, broken down into simple concepts:

1. The "Identity Director" (The Smart Librarian)

Before the robot starts drawing, a smart AI librarian (called the MLLM Identity Director) looks through a whole album of Alex's photos and videos.

The Job: It doesn't just pick the "best" photo. It picks a diverse set of moments: Alex smiling, Alex looking sideways, Alex in the dark, Alex with a hat, and Alex without a hat.
The Conflict Resolver: If the user's prompt says "Alex in a red shirt" but the photos show Alex in a blue shirt, the librarian knows to prioritize the user's text over the photos for the shirt color, but keep the face from the photos. It acts like a director on a movie set, making sure the script (the prompt) and the actor's look (the identity) don't fight each other.

2. The "Mosaic Injection" (The 3D Memory Block)

This is the core magic trick, called SMII.

The Old Way: Imagine trying to paste a photo onto a video stream. It often looks like a sticker that doesn't quite fit the flow.
The Argus Way: The librarian arranges those 9 selected moments into a 3x3 grid mosaic (like a tic-tac-toe board).
The "Time Travel" Trick: Instead of pasting this grid into the video, Argus places it before the video starts in the computer's memory. Think of it like a "negative time" memory. The video generation process can look back at this grid to remember what Alex looks like, but the grid itself never gets mixed up or "polluted" by the new video being created. It's a read-only memory that the robot can peek at whenever it needs to remember Alex's nose shape or eye color, regardless of how Alex is moving in the new scene.

3. The "Counterfactual Training" (The "What If" Gym)

Usually, to teach a robot to recognize a person, you need pairs of videos: one of the person standing still, and one of the same person running. These are hard to find.

Argus's Secret: It doesn't need those pairs. Instead, it takes one video of Alex and creates thousands of "fake" versions. It randomly swaps the background, adds digital noise, changes the lighting, or even digitally adds a hat or glasses.
The Lesson: By forcing the robot to recognize Alex despite all these random changes, the robot learns what is truly "Alex" (the face structure) and what is just "noise" (the background or accessories). It learns to ignore the distractions.

4. The "Adaptive Guidance" (The Volume Knob)

When the robot is actually drawing the video, it has to balance two things: following the text instructions and keeping the face looking like Alex.

The Problem: If you push the "keep the face" button too hard, the video looks stiff and frozen. If you push it too soft, the face changes into someone else.
The Solution: Argus uses a smart "volume knob" called Adaptive Self-Likeness Guidance.
- Early in the video: It focuses on the big picture (layout, movement).
- Middle of the video: It turns up the volume on the identity to make sure the face is correct.
- End of the video: It turns it down slightly to let the textures look natural and not too sharp or plastic.

The Results

The paper tested Argus on a "stress test" called HardID-Celeb, which includes tricky scenarios like:

Large Yaw: The person turns their head almost 90 degrees (showing their profile).
Occlusion: The person's face is partially covered in the first frame.

In these tough tests, Argus outperformed all other methods. While other systems would lose the person's identity when they turned their head or when their face was covered, Argus kept the person recognizable, preserving their unique features even in difficult angles and lighting.

In short: Argus stops treating a person's identity as a single, static photo and starts treating it as a living, breathing memory that can be accessed from any angle, at any time, without getting confused by the background or the lighting.

Technical Summary: Argus – Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation

1. Problem Definition

The paper addresses the challenge of subject-preserving video generation, a task distinct from standard text-to-video or image-to-video synthesis. The goal is not merely to generate realistic motion but to ensure the generated subject remains the same specific individual across complex variations, including:

Large viewpoint changes (e.g., large yaw angles).
Facial expression shifts and micro-motions.
Partial occlusions and scale variations.
Conflicts between text prompts, first-frame conditions, and identity references.

The authors argue that existing methods suffer from a "point-reference paradigm." Current approaches typically condition generation on a single static image or a few frames via adapters or token concatenation. This collapses identity into a single observation entangled with nuisance variables (pose, lighting, accessories, background). Consequently, models either over-trust the reference (copying unwanted details) or under-trust it (causing identity drift). The paper posits that human identity in video is a distribution rather than a point, requiring exposure to diverse, dynamic, and nuisance-randomized evidence.

2. Methodology

Argus is a framework built upon the Wan 2.1 video foundation model (a 14B-parameter Diffusion Transformer trained with Flow Matching). It introduces three core components to transition from static reference matching to dynamic identity memory:

A. Argus Identity Director (AID)

A perception front-end utilizing a frozen Multimodal Large Language Model (MLLM) (Qwen3-VL).

Function: It compiles the most informative identity evidence from a reference bank (images or videos) and resolves conflicts between the text prompt, first frame, and identity references.
Selection Strategy: Instead of selecting top-scoring frames, it optimizes for diversity across axes such as viewpoint, expression, motion, scale, and illumination, while minimizing redundancy and nuisance risks.
Output: It produces a set of nine diverse identity observations (a mix of static images and micro-clips), a conflict-aware identity caption, and a global identity descriptor vector ( $h_D$ ) that conditions the DiT alongside text and first-frame embeddings.

B. Stacked Multi-View Identity Mosaic Injection (SMII)

This is the core injection mechanism, designed to integrate identity directly into the model's native token space without external adapters.

Mosaic Construction: Selected identity observations are arranged into a $3 \times 3$ stacked mosaic. Each cell can be a static image or a short micro-clip.
Diffusion-Synchronized Tensorization: Unlike methods that inject clean reference tokens, SMII places the identity mosaic on the same Flow Matching path as the target video. The identity latent is noised at the current timestep ( $\sigma_t$ ) to match the noise level of the video latent. This ensures the identity stream aligns with the feature prior learned by Wan's patch embedding.
Negative-Time Positional Anchoring: Identity tokens are assigned negative temporal coordinates (e.g., $-n$ to $-1$), positioning them as "pre-video history" adjacent to the first frame ( $t=0$ ). This allows the model to attend to identity evidence without interpreting it as generated content.
Read-Only Memory: A block attention mask ensures identity tokens attend only to other identity tokens, while video tokens can attend to both. This prevents the reference memory from being contaminated by the generated video content.

C. Training and Guidance Strategies

No-Cross-Pair Counterfactual Training: Argus does not rely on expensive paired data (reference video + target video of the same subject). Instead, it uses same-clip or same-identity references with strong counterfactual augmentation. Backgrounds, accessories, lighting, and crops are randomized while the stable facial identity is preserved. This forces the model to learn identity-invariant features.
Temporal Identity Annealing (TIA): A gating mechanism that modulates video-to-identity attention based on diffusion time. It concentrates identity reading in the "structural middle phase" of denoising, preventing identity injection from over-constraining the global layout (early) or over-sharpening faces (late).
Adaptive Self-Likeness Guidance (ASLG): At inference, this method decouples text guidance from identity fidelity. It constructs a "weak model" branch via a learnable skip mask, allowing the system to interpolate between conditional and unconditional predictions specifically for identity, preventing identity control from overwhelming the text prompt.

3. Key Contributions

Argus Framework: A subject-preserving video generation system that shifts from point-reference conditioning to dynamic identity memory, accepting both image and video references.
SMII Module: A native injection mechanism that represents multi-view evidence as diffusion-synchronized, negative-time, read-only memory tokens, eliminating distribution mismatches associated with clean reference-token concatenation.
Training Paradigm: A no-cross-pair counterfactual training strategy that achieves robustness through large-scale nuisance randomization rather than requiring paired subject-video data.
Benchmarks and Metrics:
- HardID-Celeb: A new public-figure benchmark designed to stress-test identity preservation under large yaw, first-frame occlusion, and small faces.
- YawScore & OccScore: New metrics specifically measuring robustness to large viewpoint changes and occluded first-frame conditions.

4. Results

Argus achieves state-of-the-art performance on both general and stress-tested benchmarks:

OpenS2V-Eval Human-Domain: Argus achieves a Total Score of 64.38, improving upon the strongest baseline by 4.18 points. It also leads in FaceSim (71.86), NexusScore (51.62), and NaturalScore (79.14).
HardID-Celeb: Argus significantly outperforms baselines in stress scenarios, achieving a FaceSim of 76.80. Notably, it improves YawScore by 12.60 points and OccScore by 15.10 points over the strongest baselines, demonstrating superior ability to recover identity under viewpoint and occlusion stress.
Ablation Studies: Removing SMII (replacing it with naive token concatenation) drops the Total Score by 10.52 points. Replacing counterfactual training with 2K cross-pair samples reduces the score by 5.76 points, validating the efficacy of the proposed training strategy.

5. Significance and Claims

The paper claims that dynamic identity memory and large-scale counterfactual self-supervision are highly effective alternatives to static reference conditioning and paired data supervision.

By reframing identity not as a single static point but as a compact, dynamic distribution accessible via negative-time memory, Argus resolves the entanglement of identity with nuisance variables. The authors argue that this approach allows the model to recognize familiar subjects across motion, expression shifts, and occlusions more effectively than methods relying on frontal-face similarity or single-frame adapters. The release of HardID-Celeb and the associated metrics aims to provide a more faithful evaluation of identity preservation in challenging, real-world scenarios.