Improving Low-Resolution Face Recognition under Limited Data: How Synthetic Data Generation Can Close the Domain Gap

Technical Summary: Improving Low-Resolution Face Recognition under Limited Data

Problem Statement
Face recognition (FR) systems deployed in surveillance often encounter Low-Resolution (LR) faces, where the facial region falls below the standard 112 × 112 input size (e.g., 16–32 pixels). While High-Resolution (HR) training data is abundant, labelled native-LR data and, crucially, paired native-LR/HR data are scarce. This scarcity hinders the training of robust models for edge devices. A common workaround is to synthesize LR data from available HR faces; however, it remains unclear whether the effort invested in complex synthesis strategies translates to improved recognition accuracy on real-world LR data. Furthermore, this setting raises fairness concerns, as degradation may affect demographic groups unequally.

Methodology
The authors present a systematic study of synthetic data generation strategies for a compact, edge-device-oriented backbone (EdgeFace-S) under limited LR data. They evaluate five adaptation strategies across three levels of "synthesis effort":

1. Low Effort (Interpolation Augmentation): A deterministic chain involving downsampling HR faces to specific resolutions (56, 28, 14, or 7 px) using cubic or area interpolation, followed by upsampling back to 112 px. The model is retrained end-to-end on this augmented data.
2. Medium Effort (Generative Degradation): Utilizing a Real-ESRGAN-style stochastic degradation pipeline (random blur, resizing, noise, and JPEG compression) to synthesize realistic LR training faces. This includes training a native 32 px stem (replacing the standard stem with a 1×1 convolution) to accept the degraded images directly.
3. High Effort (Learned Super-Resolution Front-End): A two-stage approach where an identity-aware super-resolution (SR) front-end (using ESPCN or RRDB architectures) is pre-trained to reconstruct HR-like images from 32 px inputs. This is followed by joint contrastive training with a Prepended Domain Transformer (PDT) translator, all feeding into a frozen, stronger backbone (EdgeFace-base).

The study also evaluates Knowledge Distillation (KD), where an HR-trained teacher guides an LR student.

Evaluation Protocol
The authors address the "synthetic–real gap" by evaluating all strategies on two distinct types of data:

• Synthetic Benchmarks: Standard verification sets (LFW, CFP-FP, AgeDB-30) degraded synthetically to various resolutions.
• Real Native-LR: The TinyFace dataset, a real-world collection of LR faces, evaluated under two alignment pipelines (aligned crops with padding and the Differentiable Face Aligner).

Key Results

• The Synthetic–Real Gap: The degradation setting that yields optimal performance on synthetic benchmarks (28 px with cubic-down/area-up, denoted 28 ↓c/↑a) performs poorly on real native-LR data (TinyFace), often worse than the HR-trained baseline. Conversely, a milder synthetic setting (56 ↓c/↑a) proves most effective on real LR data.
• Non-Monotonic Returns on Effort: Increased synthesis effort does not guarantee better results.
  • On the compact backbone (EdgeFace-S), simple interpolation augmentation (56 ↓c/↑a) is the only strategy that improves upon the direct-feed baseline.
  • Knowledge Distillation yields the largest gains on synthetic data but fails to transfer to real LR.
  • Learned SR Front-Ends: Even with identity-aware pretraining and joint training, the learned SR+PDT pipeline fails to surpass a direct feed of the aligned LR image into a strong, frozen backbone. The visual quality of the reconstructed faces does not translate into recognition gains.
• Fairness: While LR-aware synthesis improves average accuracy, it does not systematically reduce demographic bias. The False Match Rate (FMR) disparity across African, Asian, Caucasian, and Indian groups on the RFW dataset remains inconsistent or worsens depending on the training resolution.

Significance and Conclusions
The paper concludes that generative methods for LR face recognition must be validated on real LR data and against a direct-feed baseline, rather than relying solely on synthetic degradation benchmarks. The authors argue that:

1. Optimal settings differ by domain: The "best" synthetic degradation is not the best for real-world deployment.
2. Simplicity often wins: For retrainable compact models, low-effort interpolation augmentation is superior to complex generative pipelines.
3. Baseline importance: A learned SR front-end is not useful as a standalone solution if a strong backbone can be fed the aligned image directly; restoration pipelines should only be claimed as beneficial if they outperform this direct-feed baseline.
4. Fairness limitation: Accuracy gains from synthesis do not automatically resolve demographic bias.

The authors release their pipeline to encourage validation on real native-LR data, emphasizing that matching synthesis effort to deployment constraints (compact vs. strong backbone) is critical for practical application.