A Systematic Failure Analysis of Vision Foundation Models for Open Set Iris Presentation Attack Detection

Technical Summary: A Systematic Failure Analysis of Vision Foundation Models for Open Set Iris Presentation Attack Detection

Problem Statement
Iris recognition is a high-security biometric modality, yet it remains vulnerable to Presentation Attacks (PAs) using instruments such as printed images, textured contact lenses, electronic displays, and synthetic irises. While deep learning-based Presentation Attack Detection (PAD) has achieved high accuracy in closed-set scenarios (where test attacks are represented in training), real-world deployments face "open-set" conditions involving unseen attack instruments, different sensors, and varying spectral conditions. Recent work has introduced Vision Foundation Models (VFMs)—large-scale models pretrained on massive datasets via self-supervised or weakly supervised objectives—into biometrics. However, their suitability for open-set iris PAD, particularly regarding their ability to generalize to unseen distribution shifts, remains insufficiently examined. This paper addresses the gap between the strong closed-set performance of VFMs and their reliability under realistic, open-set operational conditions.

Methodology
The study employs a unified experimental framework to evaluate five representative vision foundation models: CLIP, DINOv2, DINOv3, EVA02, and OpenVision. The evaluation utilizes a diverse corpus of 81,024 Near-Infrared (NIR) periocular images and 31,200 Visible Spectrum (VIS) periocular images.

• Input Representation: To ensure consistency across heterogeneous datasets and avoid confounding factors related to segmentation quality, the primary experiments use periocular images (cropped to 224×224 pixels) rather than segmented iris regions.
• Adaptation Strategies: Two adaptation regimes are assessed:
  1. Frozen Features: The backbone is frozen, and a lightweight Multi-Layer Perceptron (MLP) is trained to classify bona fide vs. attack samples.
  2. Parameter-Efficient Adaptation (LoRA): Low-Rank Adaptation is applied to attention layers in the final transformer blocks, allowing for task-specific fine-tuning with a minimal number of trainable parameters.
• Open-Set Protocols: Three distinct protocols isolate specific sources of distribution shift:
  • Protocol 1 (Cross-PAI): Evaluates generalization to unseen Presentation Attack Instruments (PAIs) by holding out entire PAI categories (e.g., synthetic irises, textured lenses, paper prints, diseased eyes) during training.
  • Protocol 2 (Cross-Dataset): Evaluates generalization to unseen datasets/sensors using a leave-one-dataset-out approach across NIR corpora.
  • Protocol 3 (Cross-Spectral): Evaluates transfer from NIR (training) to VIS (testing) imagery.
• Metrics: Performance is measured using Attack Presentation Classification Error Rate (APCER), Bona Fide Presentation Classification Error Rate (BPCER), and Detection Equal Error Rate (D-EER), with 95% confidence intervals estimated via non-parametric bootstrapping.
• Feature Analysis: A geometric separability analysis is conducted using the Separability Ratio Drop (SRD) and Dispersion Drop Percentage (DDP) to understand how shifts affect the embedding space structure.

Key Contributions

1. Large-Scale Evaluation: The first systematic evaluation of five diverse vision foundation models for iris PAD using a unified framework and a large, multi-source corpus of NIR and VIS periocular data.
2. Protocol-Driven Failure Analysis: The design and application of three complementary open-set protocols that isolate attack-level, dataset-level, and spectral shifts, allowing for precise attribution of failure modes.
3. Adaptation Impact Assessment: A controlled comparison of frozen versus LoRA-adapted representations, revealing that parameter-efficient fine-tuning does not universally improve open-set robustness.
4. Geometric Insight: An empirical analysis linking PAD performance degradation to changes in feature manifold geometry (inter-class distance and intra-class dispersion).
5. Extended Validations: Additional experiments using segmented iris inputs, full backbone fine-tuning, joint cross-dataset/PAI shifts, and reverse (VIS→NIR) spectral transfer to rule out alternative explanations for the observed failures.

Results
The study reveals a critical discrepancy between closed-set/cross-dataset performance and open-set security:

• Unseen PAIs (Protocol 1): Foundation models fail to generalize to unseen attack instruments. D-EER values frequently approach chance levels (50%), particularly for synthetic irises and textured lenses. LoRA adaptation often exacerbates this failure, increasing D-EER and BPCER, suggesting over-specialization to training PAIs.
• Unseen Datasets (Protocol 2): Models demonstrate better transferability across datasets when the sensing modality (NIR) remains constant. Frozen models like DINOv2 and OpenVision achieve low D-EER on certain held-out datasets. LoRA frequently improves performance in this regime, reducing D-EER to single digits for specific datasets. However, this success is uneven and does not extend to synthetic data or all sensor types.
• Cross-Spectral Transfer (Protocol 3): Performance collapses significantly when transferring from NIR to VIS. Most models exhibit high D-EER and saturated BPCER. LoRA adaptation removes the partial transfer benefits of frozen models, driving performance to chance levels.
• Feature Space Analysis: The separability analysis confirms that under unseen PAI and spectral shifts, the geometric separation between bona fide and attack classes degrades (high SRD). LoRA adaptation often increases intra-class dispersion (negative DDP), further entangling the classes.
• Extended Validations:
  • Using segmented iris inputs did not resolve the open-set failures.
  • Full backbone fine-tuning failed to provide consistent robustness, often performing worse than frozen or LoRA variants under unseen PAI and cross-dataset shifts.
  • Joint cross-dataset and cross-PAI shifts resulted in persistent failures, confirming that isolated success does not imply robustness to combined shifts.
  • Reverse spectral transfer (VIS→NIR) also failed, indicating the issue is spectral domain mismatch rather than a specific directionality.

Significance and Claims
The paper argues that strong performance in closed-set or limited cross-dataset evaluations should not be interpreted as evidence of robust open-set security for iris PAD. The findings highlight that general-purpose vision foundation models, optimized for semantic invariance, tend to suppress the subtle, low-level artifacts (e.g., print patterns, contact lens textures) that are critical for PAD.

The study concludes that current foundation models lack the necessary "PAD-aware" representations to handle realistic deployment variations. Specifically:

• Invariance vs. Sensitivity: The invariance properties engineered into VFMs (to handle illumination and sensor noise) are detrimental to PAD, which relies on detecting those very variations.
• Adaptation Risks: Parameter-efficient adaptation (LoRA) is not a general solution; it can amplify failures under attack-level and spectral shifts by overfitting to training distributions.
• Design Implications: Future PAD systems based on foundation models require explicit strategies to preserve artifact-sensitive cues, such as PAD-aware pretraining, constrained adaptation, and spectrum-aware decision-layer calibration.

The work serves as a methodological warning and a call for evaluation protocols that explicitly reflect open-set conditions, ensuring that robustness claims are qualified by the specific types of distribution shifts they address.