💻 computer science

Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps

Pro-Pose is a novel method that transforms unpaired, in-the-wild full-body portraits into controllable, high-fidelity avatars with standardized minimal clothing by leveraging canonical UV maps and multi-image fine-tuning to preserve identity while enabling robust reposing for downstream applications like Virtual Try-On.

Original authors: Sandeep Mishra, Yasamin Jafarian, Andreas Lugmayr, Yingwei Li, Varsha Ramakrishnan, Srivatsan Varadharajan, Alan C. Bovik, Ira Kemelmacher-Shlizerman

Published 2026-03-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sandeep Mishra, Yasamin Jafarian, Andreas Lugmayr, Yingwei Li, Varsha Ramakrishnan, Srivatsan Varadharajan, Alan C. Bovik, Ira Kemelmacher-Shlizerman

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a single, casual selfie of yourself. Maybe the lighting is weird, you're wearing a bulky winter coat, and you're standing in a messy room. Now, imagine you want to see what you'd look like in a completely different outfit, or perhaps doing a handstand, or even just standing in a professional studio with perfect lighting.

Usually, doing this is a nightmare for computers. They get confused by your clothes, the background, and the weird angles. They often end up making your face look like a stranger's or your body shape all wrong.

Enter Pro-Pose, a new AI tool developed by researchers at Google and the University of Texas. Think of Pro-Pose as a "Digital Tailor and Photographer" rolled into one. Here is how it works, explained simply:

1. The Problem: The "Messy Room" vs. The "Studio"

Most photos we take are "in-the-wild." They are messy. If you want to try on a new shirt digitally (Virtual Try-On), the computer gets confused because your current clothes are hiding your body shape. It's like trying to measure a person's size while they are wearing three heavy coats and a hat.

2. The Solution: The "Digital Mannequin" (Canonical UV Maps)

Pro-Pose has a clever trick. Instead of trying to edit the messy photo directly, it first turns you into a standardized digital mannequin.

The Metaphor: Imagine taking a photo of a person, then instantly peeling off their clothes and skin to create a flat, 2D "map" of their body (like a paper doll pattern). This is called a UV Map.
The Magic: On this map, Pro-Pose paints you wearing a simple, standard black tank top and shorts. It strips away the messy background, the heavy coats, and the bad lighting.
Why this helps: Now, the computer knows exactly what your body shape and face look like, without any distractions. It's like having a clean, blank canvas.

3. The "Donor" Trick: Learning to Move Without Cheating

Here is the hardest part: How do you teach a computer to move this mannequin into a new pose (like a handstand) without just "copying and pasting" parts of the original photo?

If you just tell the AI, "Move this person," it might cheat by looking at the edges of the original photo to guess where the body goes. That's bad because it doesn't actually learn how the body works.

The Analogy: Imagine teaching a student to draw a person lying down. If you show them a photo of a person standing, they might just trace the outline.
The Fix: Pro-Pose uses a "Donor" strategy. It takes the "map" of your body and covers parts of it with "masks" taken from other random people in different poses.
The Result: The AI can no longer cheat by looking at the edges of your original photo. It is forced to actually understand your 3D body shape to fill in the missing pieces. It's like a puzzle where the pieces keep changing shape, forcing the AI to learn the true structure of a human body.

4. The "Personal Touch" (Fine-Tuning)

Sometimes, the AI might get your nose slightly wrong or your hair color a bit off. To fix this, Pro-Pose has a "Personalization" mode.

The Metaphor: Think of the base AI as a master sculptor who knows how to make a perfect human statue. But if you want your specific statue, you give the sculptor a few more photos of you.
The Action: The AI quickly "fine-tunes" itself using just a few extra photos of you. It adjusts its memory to lock onto your specific face and body details, ensuring the final result looks exactly like you, not just a generic person.

5. The Payoff: Virtual Try-On and More

Once Pro-Pose has created this clean, pose-controlled "digital twin" of you, the real magic happens:

Virtual Try-On: Because your body is now a clean, standardized shape (the black tank top), any new clothes the AI wants to put on you will fit perfectly. No more weird glitches where the shirt blends into your original jacket.
Pose Control: You can ask the AI to make you dance, jump, or pose like a model, and it will do it while keeping your face and body shape perfectly intact.

Summary

Pro-Pose is like a magical photo studio that:

Takes your messy, everyday photo.
Strips it down to a clean, standard "body map."
Uses a clever "donor" trick to learn how to move that body into any pose without cheating.
Lets you try on any outfit or strike any pose, all while keeping your face and identity 100% real.

It turns a single, imperfect snapshot into a versatile, high-quality digital avatar that you can control however you like.

1. Problem Statement

The paper addresses the challenge of generating high-fidelity, pose-controllable human avatars from a single "in-the-wild" photograph. Existing methods face three primary limitations:

Data Scarcity & Overfitting: State-of-the-art pose-guided synthesis models (e.g., MCLD, LEFFA) rely on small-scale paired datasets (like DeepFashion with ~100 identities). This leads to severe overfitting, "identity drift" (where the generated face resembles a training model rather than the input), and poor generalization to unseen individuals.
Pose-Texture Coupling: In standard image-space training, the model learns to copy pixels based on visible boundaries. When the pose changes, the model struggles to synthesize occluded regions or correctly warp textures because the visible boundaries in the input image leak information about the source pose.
Downstream Interference: Current Virtual Try-On (VTO) systems often fail on unconstrained photos because the original clothing creates geometric interference and structural leakage, preventing the clean application of new garments.

The goal is to create a system that can canonicalize a person into a standardized, minimal outfit (a "blank canvas") while preserving their unique identity (face, skin texture, body shape) and allowing arbitrary pose control, even when training data is largely unpaired.

2. Methodology

Pro-Pose introduces a unified framework that operates in Canonical UV Space rather than image space, leveraging a hybrid training strategy of paired and massive unpaired data.

A. Canonical UV Space Representation

Instead of generating images directly, the method maps the input image to a SMPL-X based UV texture space.

Decoupling: This theoretically decouples pose from appearance. The UV map represents the person's texture independent of their current pose.
Partial Observation: Since a single image only reveals a partial UV map (due to self-occlusion), the system must learn to reconstruct the full texture and synthesize new views.

B. Donor-Based UV Reposing (The Core Innovation)

To prevent the model from learning trivial "copy-paste" shortcuts based on occlusion boundaries, the authors introduce Donor-Based UV Reposing:

Mechanism: For a single-view training sample, the system extracts the partial UV texture ( $T_p$ ). Instead of using the original visibility mask, it applies a random visibility mask ( $\tilde{M}_p$ ) from a "donor" image with a different pose.
Effect: This creates a hybrid texture ( $T_{p \to \tilde{p}}$ ) where the visible boundaries no longer correspond to the source pose. The model is forced to learn robust geometric warping and inpainting to reconstruct the full identity, rather than relying on boundary cues.
Face Dropout: In the single-view branch, the face crop condition is explicitly dropped during training to prevent the model from simply copying the face pixels, forcing it to learn identity representation from the warped body texture.

C. Generative Framework

Architecture: The generator is a Latent Rectified Flow model (based on the Flux.1 transformer backbone) operating in a compressed 16-channel latent space.
Training Strategy:
- Paired Branch: Uses ground-truth pose pairs (source/target) conditioned on the source texture, target pose, and face crop.
- Unpaired Branch: Uses the Donor-Based Reposing mechanism on single images, conditioning on the donor-masked texture and target pose (without the face crop).
Base Clothing (BC) Standardization: To ensure consistency, the authors use Gemini 2.5 Flash Image to preprocess the entire dataset (DeepFashion, FFHQ, e-commerce) into a standardized "Base Clothing" state: a black tank top and shorts. This removes garment variation as a confounding factor.

D. Test-Time Personalization

To handle extreme pose changes and further enhance identity fidelity, the method employs a few-shot adaptation strategy. Given a small set of reference images of a specific subject, the model fine-tunes its LoRA adapters at test time using a paired objective, significantly reducing identity drift.

3. Key Contributions

Scalable Hybrid Training Framework: A novel approach that integrates scarce paired data with abundant unpaired single images (approx. 470K unpaired + 30K paired), overcoming the identity diversity bottleneck of existing benchmarks.
Donor-Based UV Reposing: A self-supervised mechanism that breaks the correlation between pose and texture boundaries, enabling effective learning from unpaired data without pose leakage.
Base Clothing (BC) Dataset: A large-scale, standardized dataset synthesized via AI to ensure uniform visual conditions (minimal black clothing) across diverse sources.
Test-Time Personalization: A lightweight LoRA fine-tuning strategy that adapts the pre-trained model to specific subjects, drastically improving identity preservation in challenging scenarios.
Emergent Capabilities: The framework enables direct texture editing (e.g., adding tattoos) that propagates geometrically to new poses, and serves as a superior preprocessing step for Virtual Try-On.

4. Results

The paper evaluates Pro-Pose on DeepFashion (in-domain) and WPose (out-of-domain/in-the-wild) datasets.

Quantitative Performance: Pro-Pose achieves State-of-the-Art (SOTA) results across all metrics, including PSNR, SSIM, LPIPS, and identity preservation (FaceSim, DINO).
- On the WPose dataset, Pro-Pose significantly outperforms baselines (e.g., MCLD, LEFFA, UniHuman), demonstrating superior generalization to real-world imagery.
- Identity Preservation: In extreme pose changes (e.g., >60° rotation), Pro-Pose maintains a FaceSim score >0.4 in 86.5% of cases, whereas baselines fail in over 97% of cases.
Ablation Studies:
- Hybrid Data: The combination of paired and unpaired data is crucial; "Paired Only" overfits, while "Unpaired Only" fails on large pose shifts.
- Donor Reposing: Replacing donor masks with random patch masking leads to significant performance drops, proving that realistic occlusion simulation is necessary to force geometric learning.
- Fine-Tuning: Few-shot adaptation improves FaceSim by ~18% over the zero-shot base model.
Downstream Application: When used as a preprocessing step for Virtual Try-On, Pro-Pose significantly improves the fidelity of garment transfer compared to applying VTO directly to raw in-the-wild images.

5. Significance

Pro-Pose represents a paradigm shift in human image synthesis by moving away from the reliance on expensive, small-scale paired datasets. By operating in canonical UV space and utilizing a donor-based masking strategy, it successfully leverages the vast availability of unpaired internet images to learn robust 3D geometric priors.

Its ability to generate a "clean," standardized avatar from a single photo makes it a powerful foundational tool for the fashion industry (Virtual Try-On), digital identity management, and metaverse applications. It solves the critical problem of identity drift in pose transfer, ensuring that the generated avatar remains a faithful representation of the specific individual, regardless of the target pose or clothing.