BarbieGait: An Identity-Consistent Synthetic Human Dataset with Versatile Cloth-Changing for Gait Recognition

1. Problem Statement

Gait recognition is a robust biometric technology, but its real-world application is severely hindered by clothing variations. Existing datasets (e.g., CASIA-B, GREW, Gait3D) suffer from two critical limitations:

• Scarcity of Cross-Clothing Data: Most real-world datasets offer only a few clothing changes per subject (e.g., 3–7 outfits), which is insufficient to train models that generalize across diverse, extensive clothing styles.
• Identity Inconsistency in Synthetic Data: Existing synthetic datasets often fail to preserve the unique gait identity of a subject when changing clothes. They typically reuse motion sequences across different virtual characters or apply diverse motion patterns to the same character, breaking the link between the gait and the identity.

The core challenge is to generate a massive dataset where a single real-world subject's identity is preserved across hundreds of distinct clothing combinations to learn cloth-invariant features.

2. Methodology

The paper proposes a two-pronged approach: the creation of the BarbieGait dataset and the development of the GaitCLIF model.

A. BarbieGait: The Synthetic Dataset

BarbieGait is a synthetic dataset containing 521 subjects, each with 100 unique clothing combinations, resulting in over 1.2 million gait sequences. The generation pipeline ensures identity consistency through three key modules:

1. Identity Mapping (Skeleton & Shape Matching):

   • Real subjects are captured via a 6-camera synchronized array.
   • 3D poses and meshes are estimated using HRNet and EasyMoCap.
   • Virtual humans (generated via MakeHuman) are aligned to real subjects by matching skeleton lengths (e.g., thigh, lower leg) and body shape parameters (12 static circumference measurements like chest, waist, hip) to ensure the virtual avatar physically matches the real person.

2. Random Dressing:

   • Each subject is assigned 100 random full-body outfits (hair, tops, bottoms, shoes, accessories) following seasonal and practical dressing strategies.
   • This creates high intra-class variance while maintaining the underlying identity.

3. Kinematic Motion Matching (The Core Innovation):

   • To preserve gait identity, the system does not simply replay a motion on a new mesh. Instead, it uses rig-to-rig motion transfer.
   • It constructs local coordinate systems for each bone based on the source 3D pose.
   • It calculates quaternion rotations for each bone relative to its parent and transfers these local rotations frame-by-frame to the target virtual skeleton.
   • This ensures that the walking pattern (joint dynamics) remains identical to the real subject, regardless of the clothing worn.

4. Rendering & Modalities:

   • Scenes include diverse indoor/outdoor environments, lighting conditions (day/night), and occlusions.
   • The dataset provides RGB images, 2D Human Poses, and Silhouettes.

B. GaitCLIF: The Recognition Model

To address the high intra-class variance caused by clothing, the authors propose GaitCLIF (Gait-oriented CLoth-Invariant Feature). It focuses on two strategies:

1. Removing Cloth-Specific Statistics (Gait-Oriented Normalization - GON):

   • Standard normalization (like Instance Normalization) fails because silhouette features contain noise and clothing-specific statistics.
   • GON is a custom normalization layer inspired by Layer Normalization but applied to horizontally segmented regions of the feature map.
   • It computes mean and variance across channels and spatial dimensions for each segment, effectively removing style-specific statistics while preserving the relative motion patterns.

2. Preserving Fine-Grained Motion Details:

   • Frame-Level: Uses GON-P3D and GON-3D blocks to apply temporal convolutions, enhancing motion representation while suppressing clothing fluctuations.
   • Sequence-Level: Uses GON-FC (a Fully Connected layer with GON applied after each layer) to enhance non-linear mapping and reduce clothing variance in the aggregated sequence features.

3. Key Contributions

1. BarbieGait Dataset: The first identity-consistent synthetic gait dataset with massive clothing diversity (100 outfits/subject). It sets a new benchmark for cross-clothing research, offering 521 subjects and over 1.2M sequences with multimodal ground truth (3D joints, meshes, silhouettes).
2. Identity-Preserving Generation Pipeline: A novel kinematic motion matching technique that aligns real subjects to virtual avatars in both static (shape) and dynamic (motion) levels, solving the identity drift problem in synthetic data.
3. GaitCLIF Model: A robust baseline that introduces Gait-Oriented Normalization (GON) to eliminate clothing-specific statistics and preserve fine-grained motion, significantly improving cross-clothing recognition.
4. Upstream & Downstream Utility: Demonstrated that BarbieGait serves as an effective pre-training source not only for gait recognition but also for upstream human pose estimation, improving generalization on real-world datasets.

4. Experimental Results

Performance on BarbieGait

• Baseline: Standard models (e.g., DeepGaitV2) drop significantly when moving from ideal silhouettes to noisy, real-segmented silhouettes (R1 drops from ~91% to ~67%).
• GaitCLIF Performance:
  • GaitCLIF-3D achieved 80.4% Rank-1 and 65.7% mAP on the challenging THK1-THK9 clothing thickness levels.
  • GaitCLIF-P3D (using heatmaps) achieved 78.1% Rank-1 and 73.3% mAP, outperforming all other appearance and pose-based methods.
• Ablation: Both GON-P3D and GON-FC contributed positively, with their combination yielding the best results, confirming the necessity of removing clothing statistics and preserving motion details.

Performance on Real-World Datasets

• CCPG & SUSTech1K: GaitCLIF improved Rank-1 accuracy by 1.9% and 2.4% respectively compared to state-of-the-art baselines.
• Gait3D & GREW: Even on datasets with limited clothing variation, GaitCLIF achieved 76.5% R1 on Gait3D and 80.2% R1 on GREW, demonstrating robustness.
• Cross-Domain Transfer: Pre-training on BarbieGait and fine-tuning on CCPG yielded 95.8% R1, significantly outperforming training from scratch (94.8%), proving the dataset's value as a universal pre-training source.

Upstream Pose Estimation

• Retraining ViTPose on BarbieGait (Barbie-17 keypoint format) improved pose estimation on real-world CCPG data, achieving 83.0% R1, a 19.5% improvement over SkeletonGait using standard COCO-trained models.

5. Significance

• Solving the Data Bottleneck: BarbieGait provides the first large-scale, identity-consistent solution to the "lack of cross-clothing data" problem, allowing researchers to rigorously test and train models for extreme clothing variations.
• Methodological Advancement: The GaitCLIF approach demonstrates that normalizing out clothing statistics while preserving fine-grained motion is a more effective strategy than simply relying on pose or silhouette data alone.
• Future Impact: The dataset and model pave the way for robust gait recognition in surveillance and security scenarios where subjects frequently change attire. Furthermore, the identity-consistent nature of the dataset opens new avenues for training upstream tasks like pose estimation that are robust to appearance changes.

The source code and dataset are publicly available at https://github.com/BarbieGait/BarbieGait.