Calibrated Similarity and Graph Clustering for Open-Set Animal Re-Identification

Technical Summary: Calibrated Similarity and Graph Clustering for Open-Set Animal Re-Identification

Problem Statement
The paper addresses the challenge of open-set animal re-identification within the context of the AnimalCLEF 2026 competition. Unlike closed-set recognition, this task requires a system to perform two distinct functions simultaneously: (1) matching query images to known individuals in a reference database, and (2) discovering and clustering previously unseen individuals into new identity groups. The system must operate across four visually diverse species—Eurasian lynx, fire salamander, loggerhead sea turtle, and Texas horned lizard—under challenging field conditions involving variations in pose, lighting, occlusion, and background. Performance is evaluated using the Adjusted Rand Index (ARI), which penalizes both over-segmentation and under-segmentation.

Methodology
The authors propose a "similarity-to-clustering" pipeline that integrates segmentation, species-specific preprocessing, calibrated global-local fusion, and graph-based clustering.

1. Segmentation and Preprocessing:

   • Segmentation: The pipeline first isolates the target specimen using the Segment Anything Model 3 (SAM 3) to reduce background noise. Lynx images, which were provided as pre-cropped specimens, were used directly.
   • Species-Specific Preprocessing: To enhance identity-relevant visual cues, lightweight preprocessing is applied to lynx (sharpening, HSV-based CLAHE, bilateral filtering), sea turtles (red channel boosting, gamma correction for underwater color degradation), and salamanders (Sobel edge enhancement for yellow-black patterns). Texas horned lizard images undergo segmentation only to preserve original ventral spot patterns.

2. Feature Extraction and Similarity Estimation (WildFusion):

   • The core similarity engine is WildFusion, which fuses a global descriptor with two local matching branches.
   • Global Descriptor: The pipeline utilizes MiewID (Multi-species Individual Wildlife ID). The authors evaluated three variants: the original pretrained MiewID-msv3, a version fine-tuned with Dynamic ArcFace, and a version fine-tuned with SphereFace2-Focal loss. These fine-tuning strategies aimed to create more compact and discriminative embedding spaces for open-set discovery.
   • Local Matching: Two fixed local branches are used without task-specific fine-tuning: ALIKED + LightGlue and DISK + LightGlue. These capture fine-grained anatomical patterns (e.g., fur markings, shell texture).
   • Calibration: The scores from the global and local branches are calibrated using Isotonic Calibration and fused via a weighted averaging rule to produce a unified similarity score.

3. Clustering and Identity Attachment:

   • Graph Construction: Pairwise query-query similarities are converted into a sparse weighted graph.
   • Re-ranking: A k-reciprocal re-ranking step is applied to the query-query similarity matrix to refine neighborhood relationships before clustering.
   • Clustering: Chinese Whispers, a graph-based clustering algorithm, is used to infer identity clusters without a predefined number of clusters. This is crucial for the open-set nature of the task.
   • Known-Identity Attachment: For species with labeled databases, query images are matched to known identities using a query-database similarity matrix. Attachment occurs at the cluster level: if a cluster contains confident matches (based on score and margin thresholds) to a known identity, the entire cluster is assigned that identity. Otherwise, the cluster is treated as a newly discovered individual.

4. Ensembling and Optimization:

   • The final system employs an ensemble of the three MiewID variants (pretrained, Dynamic ArcFace, and SphereFace2-Focal) using majority voting.
   • Hyperparameters for re-ranking, graph construction (edge thresholds, k-core), and identity attachment are optimized per species using Optuna.

Key Contributions

• Pipeline Design: A unified pipeline that effectively combines specimen segmentation, species-aware preprocessing, and calibrated global-local fusion for open-set re-identification.
• Preprocessing Strategy: Demonstration that lightweight, species-specific preprocessing (e.g., color correction for underwater imagery, edge enhancement for patterned skin) significantly improves identity-relevant cues without heavy computational cost.
• Calibration and Fusion: The application of WildFusion to calibrate and fuse MiewID global descriptors with local keypoint matchers, showing that this combination outperforms using global descriptors alone.
• Fine-Tuning Adaptations: Exploration of Dynamic ArcFace and SphereFace2-Focal fine-tuning strategies for MiewID to improve embedding separation in an open-set context, contributing to ensemble diversity.
• Cluster-Level Attachment: A robust strategy for assigning known identities to clusters based on accumulated evidence from multiple confident query images, rather than relying on single-image matches.

Results
The proposed system substantially outperforms the competition baseline (agglomerative clustering with WildFusion) on the Adjusted Rand Index (ARI):

• Baseline (WildFusion): Public ARI 0.20342, Private ARI 0.21221.
• Main Pipeline (Pretrained MiewID + Clustering): Public ARI 0.71919, Private ARI 0.66695.
• Final Ensemble (Pretrained + Fine-tuned variants): Achieved the best Public ARI of 0.72124 and a Private ARI of 0.70393.
• Alternative Configuration: A simpler variant using only the pretrained MiewID but applying preprocessing before WildFusion calibration achieved the best Private ARI of 0.71087, suggesting that fitting calibration on the final preprocessed distribution aids generalization.

Significance and Claims
The paper claims that the primary source of performance gain stems from the overall similarity-to-clustering design rather than fine-tuning alone. The training-free pipeline (using only pretrained MiewID) already significantly outperforms the baseline, indicating that specimen segmentation, targeted preprocessing, and calibrated fusion are sufficient for a strong solution.

The authors emphasize that while fine-tuned variants (Dynamic ArcFace, SphereFace2-Focal) do not individually outperform the training-free model, they provide complementary information that enhances robustness when ensembled. Furthermore, the results highlight that calibration strategy (specifically fitting calibration on the preprocessed image distribution) is a critical factor for generalization, as evidenced by the strong private leaderboard performance of the simpler preprocessing-before-calibration configuration.

Limitations
The authors acknowledge three main limitations:

1. Reproducibility: The use of Chinese Whispers, which updates labels in randomized order, can lead to slight variations in cluster assignments across runs.
2. Computational Cost: The system is computationally expensive due to pairwise local matching and the need to run the pipeline three times for the ensemble.
3. Calibration Scope: The calibration stage was fitted using only Lynx images, which may limit the transferability of calibrated similarities across species with different visual statistics.

Future work is suggested to focus on more deterministic clustering methods, more efficient similarity computation, and broader calibration strategies to cover species diversity.