💻 computer science

CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification

This paper introduces the new task of Cross-Modality Clothing-Change Person Re-Identification (CMCC-ReID), proposes the SYSU-CMCC benchmark dataset, and presents a Progressive Identity Alignment Network (PIA) with disentangling and prototype learning modules to effectively match pedestrians across both modality and clothing variations.

Original authors: Haoxuan Xu, Hanzi Wang, Guanglin Niu

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haoxuan Xu, Hanzi Wang, Guanglin Niu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard at a busy airport. Your job is to find a specific person, "John," who might be walking through the terminal.

Usually, finding someone is easy if they haven't changed their clothes and the lighting is the same. But in the real world, two things make this incredibly hard:

The "Clothing Change": John might walk in wearing a red jacket in the morning, but by evening, he's wearing a blue hoodie.
The "Modality Shift": The morning sun is bright (Visible light), but at night, you only have thermal cameras that see heat (Infrared light). In thermal cameras, clothes look like blurry, glowing blobs, making it hard to tell a red jacket from a blue one.

Most current security systems are like guards who are trained to handle only one of these problems. Some are great at spotting people in the dark (but assume they never change clothes). Others are great at spotting people who change outfits (but assume the lighting is always perfect).

The Problem:
The authors of this paper realized that in the real world, both problems happen at the same time. A person changes clothes while the sun goes down and the camera switches to night vision. Existing systems get confused and fail because they try to match the "look" of the clothes, which changes, rather than the "soul" of the person, which stays the same.

The Solution: CMCC-ReID and the "PIA" System

The team created a new challenge called CMCC-ReID (Cross-Modality Clothing-Change Re-Identification) and built a new dataset (SYSU-CMCC) to test it. To solve this, they invented a smart AI system called PIA (Progressive Identity Alignment Network).

Here is how PIA works, using a simple analogy:

1. The "Detox" Phase (Dual-Branch Disentanglement)

Imagine you are trying to recognize a friend, but they are wearing a giant, flashy clown nose and a wig.

Old AI: Tries to memorize the clown nose and the wig. If the friend takes them off, the AI says, "I don't know you!"
PIA's Approach: It has a special "Detox" filter. It looks at the image and says, "Okay, that wig and nose are just noise. Let's ignore them." It separates the "Friend" (Identity) from the "Costume" (Clothing).
The Trick: It does this for both day (color) and night (heat) cameras. Even in the dark, blurry thermal images, it learns to ignore the "glowing clothes" and focus on the shape of the person's body and face.

2. The "Bridge" Phase (Bi-Directional Prototype Learning)

Now that the AI has a clean picture of "Who the person is" (ignoring the clothes), it needs to connect the "Day Version" of the person to the "Night Version."

Imagine you have a "Mugshot" for the day and a "Heatmap" for the night. They look totally different.
PIA builds a Bridge between them. It creates a "Master Profile" (a prototype) for John.
It forces the Day Picture and the Night Picture to both look like that Master Profile. It's like telling the Day Camera and the Night Camera: "Stop arguing about the jacket color. Just agree on who the person is."

3. The "Progressive" Strategy

This is the secret sauce. PIA doesn't try to do everything at once.

Step 1: First, it learns to ignore the clothes completely. (Don't worry about the camera type yet).
Step 2: Once it knows who the person is regardless of clothes, then it learns to match the Day camera to the Night camera.
Why? If you try to match the cameras first, the AI gets confused by the clothes and learns the wrong things. By cleaning the "clothes" problem first, the "camera" problem becomes much easier to solve.

The Results

The team tested this system on their new dataset.

The Result: PIA crushed all previous methods. It was like upgrading from a guard who guesses based on a hat, to a super-spy who recognizes a person's gait and bone structure, no matter what they are wearing or what time of day it is.
Visual Proof: When they looked at the AI's "attention maps" (where the AI is looking), the old systems were looking at the clothes. PIA was looking at the face and body, ignoring the outfit entirely.

In a Nutshell

This paper is about teaching computers to recognize people by their essence, not their outfit or their lighting conditions. By breaking the problem into two steps—first ignoring the clothes, then matching the cameras—they created a system that works much better for real-world security, where people change clothes and the sun goes down.

1. Problem Definition

The paper addresses a critical gap in Person Re-Identification (ReID) research: the simultaneous occurrence of modality discrepancy (e.g., Visible vs. Infrared) and clothing variation in long-term surveillance scenarios.

Current Limitations: Existing research typically isolates these challenges into two separate tasks:
- VI-ReID (Visible-Infrared): Assumes clothing remains constant while modality changes.
- CC-ReID (Clothing-Change): Assumes lighting/modality remains constant while clothing changes.
The Real-World Challenge: In reality, these factors coexist. A pedestrian may change outfits over time (clothing variation) and be captured by both day (visible) and night (infrared) cameras (modality discrepancy).
Specific Difficulties in CMCC-ReID:
1. Information Degradation in Infrared: Infrared images have low contrast, making it difficult to distinguish clothing from the body. Existing clothing-agnostic methods (designed for high-contrast visible images) fail here because attention mechanisms often shift toward clothing regions due to the lack of texture cues.
2. Interference during Alignment: Naive cross-modality alignment without suppressing clothing interference leads the model to align "clothing features" rather than "identity features," causing performance degradation.

2. Methodology: Progressive Identity Alignment Network (PIA)

The authors propose PIA, a framework designed to progressively mitigate clothing variation and modality discrepancy through a two-stage learning paradigm.

A. Dual-Branch Disentanglement Learning (DBDL)

Goal: To extract identity-consistent features while suppressing clothing-related noise in both visible and infrared modalities.
Mechanism:
- Dual-Branch Architecture: The network splits into an Identity Branch and a Clothing Branch.
- Spatial Attention: A lightweight spatial attention mechanism generates a clothing mask ( $m_c$ ).
- Smooth Suppression: Instead of hard exclusion, a learnable coefficient $\lambda$ creates an identity mask ( $m_{id} = 1 - \lambda \cdot m_c$ ). This allows the model to suppress clothing regions while retaining subtle identity cues that might exist within those regions.
- Orthogonality Constraint: An orthogonal loss ( $\mathcal{L}_{orth}$ ) is applied to ensure the feature subspaces for identity and clothing are decorrelated, preventing information leakage between them.

B. Bi-Directional Prototype Learning (BPL)

Goal: To align identity representations across modalities while further refining intra-modality consistency.
Mechanism:
- Prototype Maintenance: The model maintains modality-specific prototypes ( $P^V$ for visible, $P^I$ for infrared) updated via a momentum strategy to stabilize learning.
- Intra-Modality Contrast: Pulls samples closer to their own modality's identity prototype and pushes them away from others, handling pose and illumination variations within a single modality.
- Inter-Modality Contrast: Aligns a visible sample directly with the infrared prototype of the same identity (and vice versa). This bridges the modality gap using purified, clothing-agnostic features.

C. Progressive Learning Strategy

The training is divided into two stages to prevent the model from learning biased representations:

Stage I: Optimizes only the DBDL module. The goal is to "purify" features by learning clothing-irrelevant representations before any cross-modality alignment occurs.
Stage II: Jointly optimizes DBDL and BPL. Once features are disentangled, the model aligns them across modalities using the prototype learning losses.

3. Key Contributions

New Task Definition: Introduced CMCC-ReID (Cross-Modality Clothing-Change ReID), a realistic task requiring joint handling of modality shifts and outfit changes.
New Benchmark (SYSU-CMCC): Constructed a new dataset by integrating PRCC and SYSU-MM01. It contains 214 identities with 18,375 images, where each identity is captured in both visible and infrared domains with distinct outfits.
Novel Architecture (PIA): Proposed a progressive network that first disentangles features (DBDL) and then aligns them (BPL), solving the issue where standard alignment fails under clothing variation.
Comprehensive Validation: Demonstrated that existing VI-ReID and CC-ReID methods fail in this setting, while PIA establishes a strong baseline.

4. Experimental Results

Experiments were conducted on the SYSU-CMCC dataset using Rank-1 accuracy and mean Average Precision (mAP).

Performance: PIA significantly outperforms all state-of-the-art VI-ReID and CC-ReID methods.
- Visible-to-Infrared (V2I): Achieved 57.0% Rank-1 and 46.5% mAP (vs. ~23% Rank-1 for the best VI-ReID baseline).
- Infrared-to-Visible (I2V): Achieved 50.4% Rank-1 and 50.3% mAP.
Ablation Studies:
- Removing the progressive learning strategy (training DBDL and BPL simultaneously) caused a massive performance drop (~10% Rank-1), confirming the necessity of the two-stage approach.
- The DBDL module alone provided significant gains by suppressing clothing noise.
- The Inter-modality Prototype Loss was crucial for bridging the modality gap.
Visual Analysis: Attention maps show that PIA successfully focuses on body parts (head, torso) and suppresses clothing regions, whereas baseline models often attend to clothing patterns, especially in low-contrast infrared images.

5. Significance

This paper is significant because it moves ReID research from controlled, idealized benchmarks to a more realistic, complex surveillance scenario.

Practical Impact: It addresses the "long-term surveillance" problem where cameras operate 24/7 (requiring VI-ReID) and subjects change clothes over days/weeks (requiring CC-ReID).
Methodological Insight: It proves that disentanglement must precede alignment. Attempting to align features before removing clothing bias leads to the model learning the wrong semantic cues.
Future Direction: The SYSU-CMCC dataset and PIA framework provide a foundation for developing robust, real-world surveillance systems capable of tracking individuals across time and varying sensor types.