💻 computer science

MoZoo:Unleashing Video Diffusion power in animal fur and muscle simulation

MoZoo is a generative dynamics solver that leverages novel architectural innovations like Role-Aware RoPE and Asymmetric Decoupled Attention, alongside a synthetic-to-real data pipeline and a new benchmark, to efficiently synthesize high-fidelity animal videos with realistic fur and muscle dynamics from coarse meshes.

Original authors: Dongxia Liu, Jie Ma, Xiaochen Yang, Jiancheng Zhang, Bin Xia, Zhehan Kan, Nisha Huang, Jun Liang, Wenming Yang, Jin Li

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dongxia Liu, Jie Ma, Xiaochen Yang, Jiancheng Zhang, Bin Xia, Zhehan Kan, Nisha Huang, Jun Liang, Wenming Yang, Jin Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to turn a rough, gray clay sculpture of a panda into a photorealistic, fluffy, living animal for a movie.

The Old Way (Traditional CGI):
In the past, this was like building a house brick by brick. Artists had to manually sculpt every muscle, rig the skeleton so it moved correctly, and then individually place millions of tiny hairs to simulate fur. It was slow, expensive, and required a team of experts to tweak the physics of every single strand of hair.

The New Way (MoZoo):
The paper introduces MoZoo, a new AI tool that acts like a "magic paintbrush" for 3D animals. Instead of building the fur and muscles from scratch, MoZoo takes a rough, untextured 3D model (the "clay") and a reference (like a photo, a text description, or a video of a real animal) and instantly paints on realistic fur and muscle movement.

Here is how it works, broken down into simple concepts:

1. The "Recipe" Problem: MoZoo-Data

To teach an AI how to paint fur, you need thousands of examples of "Rough Clay" paired with "Real Animal." But real movies don't come with the "clay" version; they only have the final movie.

The Solution: The team built a special pipeline called MoZoo-Data. They used a video game engine (Unreal Engine 5) to create thousands of fake animal videos. Then, they used a clever "reverse trick" (an inverse model) to look at real-world wildlife documentaries and mathematically "peel off" the fur and skin to guess what the underlying rough clay skeleton looked like.
The Result: They created a massive library of 62,000 pairs of "Rough Clay" and "Real Animal" videos to train their AI.

2. The "Conductor" Problem: Role-Aware RoPE

When you ask an AI to turn a clay panda into a real panda, the AI gets confused about when things happen.

The Confusion: If you show the AI a video of a real panda running and a video of a clay panda running, the AI might think the real panda's fur should move at a different time than the clay panda's body. It's like a conductor trying to lead an orchestra where the drums and violins are playing different songs.
The Solution: They created a system called Role-Aware RoPE. Think of this as a strict conductor who assigns specific "time slots" to different inputs.
- The Clay Video (the body) and the Target Video (what we are making) get the exact same time slots so they move perfectly together.
- The Reference Video (the fur texture) gets a "delayed" time slot. This tells the AI: "Use the fur from this reference, but apply it to the body's current movement, not the reference's movement." This prevents the fur from sliding around or looking out of sync.

3. The "Noise" Problem: Asymmetric Decoupled Attention

When the AI tries to learn, the loud, obvious details of the clay body (the big muscles and bones) tend to drown out the quiet, tiny details of the fur.

The Confusion: It's like trying to hear a whisper (fur details) while someone is shouting (the clay body structure). The AI often ignores the whisper and just copies the shouting, resulting in smooth, bald-looking animals.
The Solution: They built a filter called Asymmetric Decoupled Attention. Imagine a one-way glass window.
- The AI can look at the clay body to know where to put the fur.
- The AI can look at the reference video to know what the fur looks like.
- Crucially, the clay body cannot "shout" at the fur details, and the fur details cannot "shout" at the body structure. They stay in their own lanes. This ensures the AI keeps the body's shape solid while adding the delicate, high-quality fur texture without blurring it.

4. The Results

The team tested MoZoo on a new benchmark called MoZooBench.

What it does: It can take a rough 3D model of a lion and turn it into a realistic tiger, or a clay bear into a real panda, just by showing it a picture or video of the target animal.
Why it's better: Compared to other AI tools, MoZoo keeps the animal's body moving correctly (no jittery fur) and preserves tiny details like individual hairs and muscle flexes that other tools usually smooth over.

In Summary:
MoZoo is a generative AI that skips the hard, manual work of animating fur and muscles. It uses a massive, self-made dataset and a smart "traffic control" system to instantly transform rough 3D models into high-definition, realistic animal videos, keeping the movement perfect and the fur detailed.

Technical Summary: MoZoo – Unleashing Video Diffusion Power in Animal Fur and Muscle Simulation

1. Problem Statement

Creating cinematic-quality animal effects traditionally relies on labor-intensive and computationally expensive workflows involving skeletal rigging, muscle modeling, and high-fidelity fur synthesis. These processes require experienced artists to manually adjust subtle deformations and complex physical simulations to ensure realism. While generative diffusion models have advanced artistic workflows, their application to high-fidelity animal simulation remains underexplored due to two primary bottlenecks:

Data Scarcity: There is a lack of sufficient high-quality paired data (coarse meshes paired with realistic fur/muscle videos) required to train diffusion models effectively.
Simulation Complexity: Existing reference-based video generation methods often rely on single images, which fail to capture the geometric complexity of dense fur fibers and intricate light-hair interactions. Furthermore, standard diffusion models struggle to maintain structural consistency (skeletal motion) while transferring complex textural details (fur/muscle) without feature interference.

2. Methodology

MoZoo is a generative dynamics solver designed to bypass conventional refinement stages, synthesizing photorealistic animal videos directly from coarse mesh inputs under multimodal guidance (text, images, or videos). The framework is built upon the Wan2.1 DiT (Diffusion Transformer) backbone and introduces three core innovations:

A. MoZoo-Data: A Synthetic-to-Real Pipeline

To address the data bottleneck, the authors constructed MoZoo-Data, a large-scale dataset of 62,000 paired sequences.

Synthetic Generation: Using Unreal Engine 5 (UE5), they rendered diverse animal meshes with and without textures to create perfectly aligned pairs of coarse geometry ( $V_{mesh}$ ) and high-fidelity videos ( $V_{tar}$ ).
Inverse Mapping for Real Data: Recognizing the domain gap between synthetic and real footage, they trained an inverse generative model on the synthetic data. This model extracts estimated coarse mesh sequences from high-definition real-world wildlife documentaries.
Reference Construction: Reference videos ( $V_{ref}$ ) are generated by sampling different action clips of the same species to enforce the disentanglement of identity and motion.

B. Role-Aware RoPE (RAR-RoPE)

Standard 3D Rotary Positional Embeddings (RoPE) assign continuous temporal indices, which creates artificial temporal priors when concatenating different input streams (target, mesh, reference). MoZoo proposes RAR-RoPE to handle heterogeneous alignment requirements:

Synchronization: Target and source mesh videos share synchronized temporal indices to ensure precise motion alignment.
Decoupling: Reference content (images or videos) is assigned a fixed temporal offset ( $\Delta$ ), effectively positioning them prior to the noisy target tokens. This prevents the model from confusing the temporal dynamics of the reference with the target.
Spatial Anchoring: Masks are anchored at index zero to maintain their structural role.

C. Asymmetric Decoupled Attention (ADA)

To prevent feature interference between the strong structural signals of the mesh and the subtle high-frequency details of fur, MoZoo introduces Asymmetric Decoupled Attention.

Partitioning: The latent sequence is partitioned into four segments: target ( $x_{tgt}$ ), mesh guidance ( $x_{mesh}$ ), mask ( $x_{msk}$ ), and reference ( $x_{ref}$ ).
Unidirectional Flow: The mechanism enforces a unidirectional information flow. Target queries aggregate information from all modalities, but the guidance branches (mesh and reference) are isolated, computing attention only within their own subspaces.
Frame-wise Alignment: Target frames attend exclusively to their corresponding structural keys ( $k_{mesh, i}$ ) rather than the entire mesh sequence, ensuring strict temporal synchronization while preventing noise in the target sequence from corrupting clean guidance signals.

3. Key Contributions

MoZoo Framework: A novel video diffusion framework that replaces expensive traditional CG simulation stages, directly generating high-fidelity animal videos with realistic muscle and fur dynamics from coarse meshes. It supports multimodal guidance for flexible control.
MoZoo-Data & MoZooBench: The introduction of a large-scale dataset (62K clips) constructed via a synthetic-to-real pipeline and a comprehensive benchmark (120 mesh-video pairs) to evaluate animal fur rendering, which are made publicly available.
Architectural Innovations: The proposal of RAR-RoPE to resolve multi-modal alignment challenges and Asymmetric Decoupled Attention to separate structural guidance from textural information, significantly reducing computational overhead while improving fidelity.

4. Experimental Results

The authors evaluated MoZoo on MoZooBench against state-of-the-art methods (VACE, Refacade) and ablated versions of their own model.

Quantitative Performance: MoZoo consistently outperformed baselines in structural fidelity (PSNR, SSIM) and reference alignment (LPIPS). Specifically, the video-conditioned version achieved a PSNR of 20.75 and an LPIPS of 0.070, surpassing image-conditioned baselines.
Qualitative Performance: Visual comparisons demonstrated that MoZoo preserves intricate high-frequency fur details and maintains spatio-temporal consistency, whereas baselines often produced over-smoothed results or failed to map textures to correct anatomical regions.
Ablation Studies: Removing RAR-RoPE led to spatial drift in anatomical features, while removing ADA resulted in the loss of fine details due to structural signals overwhelming fur textures. The video-conditioned modality proved superior to image-conditioned settings, particularly under significant viewpoint variations.
Cross-Species Transfer: The model demonstrated zero-shot re-skinning capabilities, successfully transferring biological textures from one species to a different species' mesh while maintaining anatomical coherence.

5. Significance and Claims

The paper claims that MoZoo significantly reduces the manual effort and computational cost associated with high-fidelity animal character production. By integrating muscle and fur dynamics into an end-to-end generative process, it bridges the gap between low-cost production and industrial-grade visual fidelity.

The authors note that while the method achieves superior textural fidelity and consistency across diverse species, challenges remain in handling complex multi-animal interactions and severe occlusions. Future work aims to generalize the framework to universal hair and fur simulation, including human hair and synthetic fibers. The authors also acknowledge the potential ethical risks regarding the generation of unauthorized digital avatars or deepfakes as the technology generalizes to human subjects.