Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification

1. Problem Statement

Domain Generalized Person Re-Identification (DG Re-ID) aims to train models on source domains (e.g., specific camera datasets) and deploy them on unseen target domains without access to target data.

• The Challenge: Pure vision-based models often fail to generalize because they learn to associate identity (ID) with domain-specific artifacts (illumination, resolution, background) rather than intrinsic ID features.
• The Limitation of Current VLMs: While Vision-Language Models (VLMs) like CLIP offer strong generalization, directly adapting them to Re-ID yields suboptimal results. Existing methods typically rely on single-grained alignment (global image-text matching). This approach ignores fine-grained nuances (e.g., specific clothing patterns, hairstyles, body parts) that are crucial for distinguishing similar IDs across different domains.
• The Gap: There is a need for a framework that leverages the generalization power of language while capturing multi-grained (global + local) visual details to handle domain shifts effectively.

2. Methodology: MUVA Framework

The authors propose MUVA (MUlti-grained Vision-language Alignment), a CLIP-based framework that aligns vision and language at multiple granularities. The architecture consists of three core components:

A. Multi-Grained Vision-Language Prompt Learning

Instead of using fixed text descriptions, the model learns learnable prompts for different granularities:

• Global Prompt: Describes the whole person (e.g., "A photo of a [X] person").
• Local Prompts: Specifically describe body parts (Head, Upper Body, Legs) using templates like "A photo of a [H] head of a person."
• Training Strategy:
  • Stage 1: Optimizes the learnable prompts to align with corresponding images using Prototypical Contrastive Loss (PCL). This "squeezes" ID information from images into the language modality, creating generalizable textual prototypes.
  • Stage 2: Trains the image encoder and a BN neck for Re-ID. It uses the learned prompts to enforce a cross-modal constraint, injecting the generalizable language knowledge back into the visual features.

B. Adaptively Masked Multi-Head Self-Attention (AM-MSA)

To extract fine-grained visual features without rigid partitioning (which fails with varying poses), the authors introduce the AM-MSA module:

• Mechanism: It replaces standard self-attention with a bypass branch containing a Residual Mask Prediction (RMP) module and a Mask Gating (MG) module.
• Function: The RMP module uses cross-attention to predict foreground masks for specific body parts (Head, Upper, Legs). The MG module then applies these masks to the attention mechanism, forcing the model to focus on specific regions for feature aggregation.
• Advantage: Unlike fixed stripes, AM-MSA adaptively locates parts regardless of pose or deformation.

C. Visual Grounding Expert (VGE) Supervision

Training the AM-MSA module requires ground truth for body part locations, which is expensive to annotate manually.

• Solution: The authors employ a pre-trained Multi-modal Large Language Model (MLLM), specifically CogVLM, as a "Visual Grounding Expert."
• Process: Before training, the VGE generates bounding boxes for body parts based on text queries (e.g., "locate the head").
• Calibration: To handle VGE hallucinations (e.g., incorrect bounding boxes), a statistical calibration step corrects extreme values.
• Pseudo-Labels: These calibrated boxes are converted into binary masks (pseudo-labels) to supervise the AM-MSA module via a combination of Binary Cross-Entropy and Dice Loss.

3. Key Contributions

1. Multi-Grained Alignment Framework: First to propose a CLIP-based DG Re-ID framework that aligns both global and local (part-level) vision-language features, addressing the nuance perception deficiency of single-grained methods.
2. Adaptive Part Extraction: Design of the AM-MSA module, which adaptively predicts and focuses on body parts using learned masks, overcoming the rigidity of traditional stripe-based partitioning.
3. MLLM-Based Supervision: Introduction of a Visual Grounding Expert to automatically generate pseudo-labels for part localization, eliminating the need for tedious manual annotation while providing strong prior knowledge.
4. State-of-the-Art Performance: Extensive experiments demonstrate significant improvements over existing methods in both single-source and multi-source DG Re-ID protocols.

4. Experimental Results

The model was evaluated on standard datasets: Market1501, DukeMTMC-reID, MSMT17, and CUHK03-NP.

• Single-Source DG Re-ID: MUVA significantly outperforms state-of-the-art (SOTA) methods.
  • Example (MA→MS): Achieved 27.6% mAP and 56.8% Rank-1, surpassing the previous best (DCAC) by a large margin.
  • Example (MS→MA): Achieved 59.5% mAP and 82.7% Rank-1.
• Multi-Source DG Re-ID: Demonstrated robust generalization across all target domains.
  • Example (MA+D+MS→C3): Achieved 48.6% mAP and 49.6% Rank-1, outperforming the runner-up (SVIL) by 14.6% mAP.
• Ablation Studies:
  • Granularity: Combining Global + Head + Upper + Legs yielded the best results, proving the synergy of multi-grained features.
  • Language Modality: Vision-Language alignment provided consistent gains over Vision-Only baselines, especially in multi-grained settings.
  • AM-MSA vs. Stripes: The adaptive masking module significantly outperformed traditional fixed stripe partitioning.
  • Calibration: The calibration of VGE pseudo-labels was proven necessary to handle hallucinations and improve performance.

5. Significance and Impact

• Bridging the Gap: The paper successfully bridges the gap between the generalization capabilities of VLMs and the fine-grained requirements of Person Re-ID.
• Efficiency: While the training process utilizes a heavy MLLM (VGE) for supervision, the inference phase is lightweight, requiring only the image encoder. This makes the model deployable in real-world scenarios where VGE inference would be too slow.
• Future Directions: The framework is extensible to other Re-ID tasks (animals, vehicles) and video-based Re-ID by incorporating temporal information into the prompts.

In summary, MUVA represents a significant advancement in DG Re-ID by leveraging the semantic richness of language and the adaptability of attention mechanisms to learn robust, fine-grained identity features that generalize well across unseen domains.