💻 computer science

ReGenHuman: Re-Generating Human Appearances for Realistic Full-Body Video Anonymization

The paper introduces ReGenHuman, a novel full-body video anonymization pipeline that synthesizes realistic and temporally consistent human appearances from identity-free structural cues using a "regenerate, don't edit" paradigm, thereby achieving an optimal trade-off between privacy, visual quality, and downstream task utility.

Original authors: Adam Sun, Eshaan Barkataki, Arnold Milstein, Gordon Wetzstein, Ehsan Adeli

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Adam Sun, Eshaan Barkataki, Arnold Milstein, Gordon Wetzstein, Ehsan Adeli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a home video of a busy hospital hallway or a crowded street. You want to share this video to help train AI systems (like self-driving cars or medical robots), but you can't because the people in the video are recognizable. Their faces, their clothes, the way they walk, and even the text on their shirts give them away.

Currently, the way people handle this is like taking a permanent marker and scribbling over the people's faces or blurring the whole screen. It protects their identity, but it ruins the video. The AI can't learn anything from a blurry mess, and the video looks terrible.

Enter "ReGenHuman."

Think of ReGenHuman not as an editor with a marker, but as a magical, super-fast painter who watches the video and then re-paints the entire scene from scratch, replacing the real people with brand-new, fake ones.

Here is how it works, broken down into simple steps:

1. The "Ghost Blueprint" (The Input)

Instead of showing the painter the real people, the system first strips the video down to a "ghost blueprint." It extracts three things that describe where people are and what they are doing, but contain zero information about who they are:

The Skeleton: A stick-figure drawing of the person's pose (arms up, walking, sitting).
The Outline: A silhouette showing exactly where the person's body is in the room.
The Depth Map: A 3D map showing how far away things are, giving the scene shape without showing textures or colors.
The Description: A text caption describing the scene (e.g., "Three people are standing on a stage giving an award").

Crucially, the painter never sees the original faces, clothes, or skin. It only sees these anonymous blueprints.

2. The "Re-Painting" (The Generation)

The system uses a powerful AI (a video diffusion model) to look at these blueprints and generate a completely new video.

It paints new people who fit perfectly into the stick-figure skeletons.
It paints new clothes and hair that look realistic.
It keeps the background and the movement exactly the same as the original video.

Because the painter never saw the original person, it is mathematically impossible for the original person to appear in the new video. It's like if you gave an artist a photo of a tree and asked them to paint a new tree based only on a wireframe drawing; they could never accidentally paint the exact same tree from the photo.

3. The Result: A "Safe" Video

The final video looks incredibly realistic. The new people look like real humans, they move smoothly (no jittery flickering between frames), and they interact with the environment naturally.

Privacy: The original people are gone. You can't recognize them by their face, their walk, or their clothes.
Quality: The video looks high-definition and natural, not blurry or pixelated.
Utility: Because the actions and movements are preserved, AI systems can still "watch" this video and learn how people move, how doctors interact with patients, or how crowds behave.

Why is this a big deal?

The paper compares this to old methods:

Old Way (Blurring/Redacting): Like putting a black bar over a face. It's safe, but the video is useless for learning.
Old Generative Way (Inpainting): Like trying to paint over a face on a photo. It often looks weird, the person might look different in every frame (flickering), and sometimes the original face still leaks through.
ReGenHuman: It's the first method that is safe by design (because it never sees the original) and high quality (because it generates a whole new, realistic video).

The authors tested this on thousands of videos, including medical footage, and found that their method is the best at balancing privacy (keeping people anonymous) with utility (keeping the video useful for AI). They even showed that AI models can still answer questions about the video (like "What is the doctor doing?") just as well as they could with the original, un-anonymized video.

In short: ReGenHuman is a tool that takes a video of real people, erases them completely, and paints in new, fake people who act exactly the same way, allowing us to share and study video data without ever risking anyone's privacy.

Technical Summary: ReGenHuman

Problem Statement

Anonymizing human-centric video data is a critical yet understudied challenge. Existing privacy regulations (e.g., HIPAA, GDPR) and the vulnerability of trained models to privacy breaches (e.g., membership inference) necessitate robust anonymization. However, current techniques face a fundamental trade-off:

Redaction/Blurring: Traditional methods (blurring, pixelation, blacking out) preserve temporal consistency but destroy semantic utility, rendering the data useless for downstream tasks like action recognition or video question answering (VQA).
Generative Editing: Prior generative approaches often edit pixels frame-by-frame or condition on original images. This leads to temporal incoherence (flickering identities), anatomical unrealistic outputs, or insufficient privacy guarantees because original identity cues (clothing, gait, background) often remain or leak through.

The core problem is the lack of a pipeline that simultaneously achieves Privacy (unreidentifiable), Quality (temporally consistent and photorealistic), and Utility (preserves semantic content for downstream AI).

Methodology: ReGenHuman

The authors propose ReGenHuman, a "regenerate, don't edit" paradigm. Instead of modifying original pixels, the system synthesizes human regions entirely from scratch using identity-free structural cues. The pipeline consists of three stages:

1. Conditioning Signal Extraction

From an input video, the system extracts four anonymous components:

Human Instance Masks ( $M$ ): Generated by SAM3, defining the exact spatial region of the person.
2D Pose Skeletons ( $P$ ): Generated by DWPose, capturing the subject's movement and posture.
Monocular Depth ( $D$ ): Generated by Video Depth Anything, providing 3D scene geometry and context without revealing textures or text.
Text Caption ( $y$ ): A detailed description of the scene generated by a vision-language model.

2. Structural Conditioning Construction

The extracted signals are composited into two distinct conditioning videos to guide the generation:

StructHuman: Combines the pose skeleton with the original background and objects (non-human pixels). This preserves rich scene context while anonymizing the human.
StructAll: Replaces the background with the depth map. This yields a fully anonymized frame where no original pixels remain, preserving only geometry and pose.
Privacy Guarantee: In both cases, the human region is grayed out before the pose is overlaid. Crucially, the conditioning $c_t$ depends only on the pose $P_t$ within the human mask, not the original video pixels $x_t$ . This provides an architectural privacy guarantee: no original human appearance can flow to the output.

3. Video-to-Video Diffusion Fine-Tuning

The system utilizes VACE (Video Anytime Controllable Editing), an extension of the Wan video diffusion model.

Architecture: The model uses a Video Condition Unit (VCU) to inject structural signals (depth, pose, mask) alongside text prompts.
Training Strategy: The pre-trained DiT backbone and Context Blocks are frozen. Only LoRA (Low-Rank Adaptation) adapters are fine-tuned on the HOIGen-1M dataset (a corpus of 1M+ human-object interaction videos).
Inference: The model generates the anonymized video by denoising a latent representation conditioned on the structural overlays and text caption, ensuring temporal coherence and anatomical realism without ever "seeing" the original human pixels.

Key Contributions

First Realistic, Temporally Consistent Pipeline: ReGenHuman is the first method to generate full-body video anonymizations that are both photorealistic and temporally stable, avoiding the flickering and anatomical errors of prior generative methods.
"Regenerate, Don't Edit" Paradigm: The paper introduces a new approach where the human region is synthesized from identity-free structural inputs. This shifts the privacy guarantee from statistical obfuscation to architectural design, ensuring original pixels cannot reach the output.
Unified Three-Axis Evaluation: The authors establish a comprehensive evaluation protocol measuring Privacy (identity/text similarity, clothes-changing re-ID), Quality (VBench metrics, realism, anatomy), and Utility (zero-shot Video Question Answering).

Experimental Results

Evaluated on the HOIGen-1M dataset and clinical footage (MedVideoCap-55K), ReGenHuman outperforms current baselines (including DeepPrivacy2, FADM, and classical redaction methods) across all three axes:

Privacy: ReGenHuman drives body re-identification (using CSCI-V on CCVID) to near-chance levels (mAP ~2.5%), significantly outperforming even "blackout" methods. It is the only method to effectively anonymize text and background cues simultaneously.
Quality: The method achieves state-of-the-art scores in Subject Consistency, Human Anatomy, and Realism. Unlike frame-by-frame generators, it maintains temporal smoothness and geometric consistency.
Utility: In zero-shot Video Question Answering (VQA) tasks, ReGenHuman maintains high accuracy (only ~4-5% degradation compared to the original video), whereas classical redaction methods suffer massive drops (>12%). Notably, the model generalizes to clinical settings without re-training, demonstrating robustness in high-stakes domains.

Significance and Claims

The paper claims that ReGenHuman sets a new standard for video anonymization by resolving the long-standing trade-off between privacy and utility. By regenerating humans from structural cues, the method enables the large-scale release of sensitive human-centric data (e.g., in hospitals, classrooms, and surveillance) that was previously unusable due to privacy concerns.

The authors emphasize that their architectural privacy guarantee makes the anonymization process auditable; privacy professionals can verify that the conditioning inputs contain no private information, ensuring the output is free of identity leakage. While the authors acknowledge limitations regarding generation speed (non-real-time) and potential pose-based re-identification risks, they posit that this work provides a foundational baseline for future research in diffusion-based video anonymization and ethical data sharing.