TransFIRA: Transfer Learning for Face Image Recognizability Assessment

1. Problem Statement

Face recognition systems deployed in unconstrained environments (surveillance, video, web imagery) face severe challenges due to variations in pose, blur, illumination, and occlusion.

• The Gap: Conventional Face Image Quality Assessment (FIQA) methods rely on visual heuristics (sharpness, pose, illumination) or complex generative pipelines. These metrics often fail to predict whether an image is actually recognizable by a specific deployed face encoder.
• The Limitation: Existing FIQA methods are often detached from the encoder's decision geometry, rely on curated human annotations, or require computationally expensive inference. There is a lack of a lightweight, self-supervised framework that defines recognizability directly within the embedding space of the specific encoder being used.

2. Methodology: TransFIRA

TransFIRA (Transfer Learning for Face Image Recognizability Assessment) is a framework that grounds recognizability directly in the embedding space of a pretrained encoder. It operates in three main stages:

A. Defining Recognizability via Embedding Geometry

Instead of using visual proxies, TransFIRA derives supervisory labels directly from the encoder's embeddings ($z_i$) and class centers ($\mu_k$).

1. Class Center Angular Similarity (CCS): Measures the cosine similarity between an image embedding and its own identity's class center.
   $$CCS_{x_i} = \cos(\theta(z_i, \mu_{y_i}))$$
2. Nearest Non-match Class Center Angular Similarity (NNCCS): Measures the maximum similarity between the image and any other class center (the most confusing impostor).
   $$NNCCS_{x_i} = \max_{j \neq y_i} \cos(\theta(z_i, \mu_j))$$
3. Class Center Angular Separation (CCAS): The difference between CCS and NNCCS.
   $$CCAS_{x_i} = CCS_{x_i} - NNCCS_{x_i}$$
   • Significance: A positive CCAS ($>0$) indicates the sample lies on the correct side of the decision boundary (closer to its own class than any impostor). This provides a natural, parameter-free cutoff for filtering unrecognizable frames.

B. Recognizability Prediction Network

• Architecture: A lightweight regression head (a single linear layer) is attached to a pretrained face encoder.
• Training: The model is fine-tuned end-to-end using Mean Squared Error (MSE) to predict the precomputed ground-truth CCS and CCAS values from raw images.
• Key Advantage: No human quality annotations or external IQA supervision are required. The supervision is deterministic and derived solely from the encoder's own geometry.

C. Recognizability-Informed Aggregation

For template-based recognition (where multiple frames represent one subject), TransFIRA introduces two operations:

1. Filtering: Discard any frame where predicted $\hat{CCAS} \leq 0$. This removes frames that the encoder cannot confidently classify.
2. Weighting: When aggregating the remaining frames into a single template, weight each embedding by its predicted $\hat{CCS}$. Frames closer to the class center (higher CCS) contribute more to the final template.

3. Key Contributions

1. Geometrically Grounded Supervision: The first definition of recognizability based on CCS and CCAS, which are intrinsic to the encoder's decision boundary. This replaces heuristic proxies with mathematically principled criteria.
2. Lightweight Transfer Learning: The framework allows any pretrained encoder to be extended for recognizability prediction with only a lightweight regression head, requiring no retraining of the backbone or architectural changes.
3. Principled Aggregation Strategy: Introduces a natural cutoff ($CCAS > 0$) for filtering and CCS-based weighting for aggregation, achieving state-of-the-art results without external labels.
4. Cross-Modality Extension: Demonstrates the framework's applicability beyond faces to body recognition by introducing a sigmoid calibration strategy to handle saturation in body embedding spaces.
5. Encoder-Grounded Explainability: The method provides insights into why an image is unrecognizable (e.g., specific degradations or subject difficulty) based on the encoder's internal geometry.

4. Experimental Results

The authors evaluated TransFIRA on BRIAR Protocol 3.1 (surveillance video, extreme degradation) and IJB-C (unconstrained web/video).

• Template Verification Accuracy (TAR):
  • On BRIAR, TransFIRA significantly outperformed state-of-the-art methods (MagFace, CR-FIQA, DifFIQA).
  • Using the Swin-B backbone, the TAR at $10^{-6}$ FMR improved by +0.1385 over uniform averaging.
  • Using the ArcFace backbone, it nearly tripled the TAR at $10^{-6}$ FMR compared to the baseline.
• Image-Level Correlation:
  • TransFIRA achieved a Spearman Correlation (SC) of 0.8566 with ground-truth recognizability on BRIAR, nearly double the performance of the next best method (CLIB-FIQA at ~0.50).
  • It demonstrated superior False Non-Match Rate (FNMR) vs. Error Rate of Classification (ERC) curves, indicating better filtering of unrecognizable frames.
• Body Recognition:
  • Applied to the SemReID body encoder, the framework (with sigmoid calibration) successfully improved recognition accuracy, proving its generalizability across modalities.

5. Significance and Impact

• Paradigm Shift: Moves FIQA from "visual quality" (is the image clear?) to "encoder-specific recognizability" (can this specific model identify this person?).
• Efficiency: Unlike generative or diffusion-based FIQA methods (e.g., DifFIQA), TransFIRA is lightweight and computationally efficient, making it suitable for real-time deployment.
• Interpretability: By tying scores to geometric margins (CCAS), the system offers transparent decision rules. It can identify "hard" subjects or specific failure modes (blur, occlusion) that cause the encoder to fail.
• Robustness: The method remains effective under cross-dataset shifts and out-of-distribution evaluation, establishing a unified framework for recognizability-aware modeling in biometrics.

In conclusion, TransFIRA establishes a new standard for Face Image Quality Assessment by directly leveraging the geometry of the deployed recognition model, offering a highly accurate, interpretable, and annotation-free solution for robust face recognition in the wild.