Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition

1. Problem Statement

The paper addresses the critical challenge of data scarcity in privacy-sensitive domains, specifically identity recognition (person re-identification and face recognition).

• The Paradox: High-fidelity generative models are most needed in scenarios where data access is restricted by regulations (e.g., GDPR) and copyright. However, these models typically require large-scale real data for training.
• The Vicious Cycle: Limited real data leads to poor generative models, which in turn fail to produce high-quality synthetic data to compensate for the scarcity.
• Limitations of Existing Methods:
  • Traditional Augmentation: (e.g., flipping, cropping) offers limited diversity and cannot bridge the domain gap.
  • Simulation/Virtual Environments: Often suffer from a large domain gap between synthetic and real images.
  • Standard Generative Models: Existing GANs and Diffusion models often mimic the source distribution but fail to optimize for task utility (i.e., the generated data may look real but does not improve downstream classification accuracy).

2. Methodology

The authors propose a Reinforcement-Guided Synthetic Data Generation Framework that adapts general-domain generative priors (pre-trained on large datasets like ImageNet) to specific, data-scarce target domains. The framework consists of three sequential stages:

A. Cold-Start Initialization

To overcome the instability of direct reward optimization on small datasets, the authors first perform a lightweight adaptation:

• Base Model: Uses a pre-trained Diffusion Transformer (DiT) trained on ImageNet.
• Adaptation: The class embedding layer is replaced with a task-specific head aligned to the target label space.
• Process: The backbone is frozen, and only the new head is fine-tuned on the limited target data using a standard denoising objective. This establishes semantic relevance and initial fidelity.

B. Reinforcement Learning (RL) Optimization

The core innovation is formulating the synthesis process as an RL problem where the generator acts as a policy. The model is fine-tuned using a multi-objective reward function to guide the generation toward task-effective samples. The total reward ($R_{norm}$) is a weighted sum of three components:

1. Semantic Consistency ($R_{sem}$): Ensures the generated image preserves the identity of the target class. It measures cosine similarity between the generated feature and the class prototype (mean feature vector) stored in a memory bank.
2. Distributional Coverage ($R_{cov}$): Encourages the generator to cover the variability of the target domain without collapsing to a single mode. It uses a Radial Basis Function (RBF) kernel to maximize alignment with reference features while penalizing redundancy among generated samples.
3. Expressive Diversity ($R_{exp}$): Controls the global dispersion of features to prevent over-concentration. It uses the trace of the feature covariance matrix to ensure the generated variance matches a target expansion relative to the reference distribution.

The rewards are standardized and combined using a hyperbolic tangent ($\tanh$) activation to ensure numerical stability.

C. Dynamic Sample Selection (DSS)

During the downstream training of the recognition model (e.g., ResNet-50 or ViT), not all synthetic samples are equally useful.

• Mechanism: A "lookahead" strategy is employed. For a mixed batch of real and synthetic data, a virtual one-step gradient update is simulated.
• Selection: The utility of a synthetic sample is estimated by the change in identity-consistency loss ($\Delta l$) after this virtual update. Samples with the smallest $\Delta l$ (indicating they align best with the current optimization trajectory) are prioritized for the actual training step.
• Goal: This mitigates distributional bias and ensures the model learns from the most "task-relevant" synthetic data.

3. Key Contributions

1. Paradigm Shift: Moves from relying solely on scarce source data to leveraging general-domain priors (pre-trained DiTs) guided by task-specific rewards.
2. Multi-Objective Reward Design: Introduces a novel reward function that jointly optimizes for fidelity (semantic consistency), diversity (distributional coverage), and richness (expression diversity), breaking the trade-off between realism and task utility.
3. Dynamic Sample Selection: Proposes a lookahead-guided mechanism to filter synthetic data during downstream training, improving generalization under domain shifts.
4. Framework Integration: Combines cold-start adaptation, RL-based fine-tuning, and dynamic selection into a continuous, adaptive pipeline.

4. Experimental Results

The method was evaluated on Person Re-identification (Market-1501, CUHK03-NP) and Face Recognition (CASIA-WebFace subset, verified on LFW, AgeDB, etc.).

• Person Re-ID:
  • Achieved 88.6% mAP on Market-1501 (a 3.2% gain over the baseline) and 76.6% mAP on CUHK03.
  • Outperformed state-of-the-art synthetic augmentation methods (e.g., GIF-SD, IDiff) and simulation-based approaches.
• Face Recognition:
  • Achieved an average verification accuracy of 79.07% on small-scale datasets, surpassing the real-data baseline by 0.60% and the best diffusion-based competitor (NegFaceDiff) by 0.94%.
  • Bias Mitigation: Showed superior performance across ethnic groups (Caucasian, Indian, Asian, African) in the RFW benchmark, reducing cross-ethnicity bias compared to other methods.
• Qualitative Analysis:
  • Generated images exhibited richer variations in pose, lighting, and expression while maintaining strict identity consistency.
  • Feature visualizations (via DOSNES) showed that synthetic samples expanded the intra-class manifold without drifting from the true identity center.
• Ablation Studies: Confirmed that each component (Cold-start, Semantic Reward, Coverage Reward, Expression Reward, and Dynamic Selection) contributed incrementally to the final performance.

5. Significance

• Privacy-Preserving AI: Provides a viable solution for training high-performance identity recognition models in environments where collecting real data is legally or ethically impossible.
• Breaking the Data Scarcity Loop: Successfully demonstrates that general-domain priors, when guided by reinforcement learning, can overcome the limitations of small datasets, turning a "vicious cycle" of poor data and poor models into a virtuous one.
• Generalizability: The framework is model-agnostic regarding the downstream task (works with both CNNs and ViTs) and offers a blueprint for utility-driven synthetic data generation beyond just identity recognition.

In conclusion, this work establishes a new standard for utility-driven synthetic data generation, proving that combining pre-trained generative priors with reinforcement learning can effectively solve data scarcity in highly regulated domains.