Frozen Vision Transformers for Dense Prediction on Small Datasets: A Case Study in Arrow Localization

1. Problem Statement

The paper addresses the challenge of automating the scoring and analysis of competitive archery targets. While manual scoring records only scalar point values, it discards valuable spatial data (e.g., grouping tightness, aiming bias, and performance trends) embedded in the 2D distribution of arrow impacts.

Key Challenges:

• Small Dataset: The training data consists of only 48 high-resolution images, far fewer than the thousands typically required for supervised deep learning.
• Perspective Distortion: Photos are taken from varying angles, causing circular scoring rings to appear as ellipses.
• Class Imbalance: Arrow punctures are tiny relative to the target face, creating a severe foreground/background pixel imbalance.
• Compound Punctures: Multiple arrows striking the same location create overlapping tears that are difficult to resolve individually.
• Precision Requirement: The system requires sub-millimeter localization accuracy to apply official "line-cutting" scoring rules.

2. Methodology

The proposed system combines a classical computer vision preprocessing pipeline with a modern, frozen self-supervised vision transformer architecture.

A. Canonical Rectification Pipeline

Before neural network processing, a geometric rectification stage transforms perspective-distorted images into a standardized coordinate system.

• Mechanism: It exploits the known color structure of the target (yellow, red, blue, black, white rings) in CIELAB color space.
• Process: Adaptive thresholds identify color boundaries, fit convex hulls, and then ellipses. A confidence-based scoring system selects the best nested ellipses.
• Output: A radial warp transforms the image into a canonical 2048×2048 output where rings are concentric circles and scale is uniform (~0.195 mm/pixel). This eliminates the need for the model to learn perspective invariance.

B. Model Architecture

The detection model uses a "frozen backbone" approach to prevent overfitting on the small dataset.

1. Feature Extraction (Frozen): A DINOv3 ViT-L/16 (pre-trained via self-supervised learning) serves as the backbone. It is entirely frozen. The 2048×2048 image is resized to 512×512, producing a 32×32 grid of 1024-dimensional tokens.
2. Guided Upsampling (Frozen): AnyUp is used to bridge the resolution gap. It takes the low-resolution (32×32) features and a high-resolution (1024×1024) guide image to produce a 512×512 feature map. This recovers spatial precision lost by the ViT's patch tokenization without fine-tuning the backbone.
3. Detection Heads (Trainable): Two lightweight parallel heads operate on the upsampled features (3.8M trainable parameters total):
   • Heatmap Head: Predicts a single-channel map of arrow-center logits using a Gaussian Focal Loss.
   • Offset Head: Predicts sub-pixel offsets ($dx, dy$) to refine center localization (though ablation showed this was less effective in this specific setup).

C. Training & Optimization

• Data Augmentation: Aggressive geometric (rotation, affine, flip) and photometric (color jitter, blur, noise) augmentations were applied to the 48 images.
• Hyperparameter Tuning: A bi-level optimization strategy using Optuna (TPE sampler) tuned learning rates, loss exponents, and decoder settings (NMS kernel, thresholds).
• Loss Functions:
  • Heatmap: Gaussian Focal Loss (penalizing hard negatives near true centers).
  • Offset: Smooth L1 loss within a masked radius of the arrow center.

3. Key Contributions

1. Frozen Foundation Models for Small Data: Demonstrates that a frozen self-supervised ViT (DINOv3) combined with lightweight heads can achieve high-performance dense prediction on a dataset of only 48 images, avoiding overfitting.
2. Geometric Preprocessing: Introduces a robust, color-based rectification pipeline that converts arbitrary perspective shots into a canonical coordinate system, simplifying the learning task and enabling direct physical measurement.
3. Guided Upsampling: Successfully applies AnyUp to recover sub-millimeter spatial precision from coarse ViT features without requiring task-specific fine-tuning of the upsampler.
4. Ablation on Offset Heads: Reveals that in high-precision, small-data regimes with guided upsampling, offset regression heads may introduce noise rather than refinement, suggesting a "heatmap-only" configuration is superior for this specific task.

4. Results

The system was evaluated using 3-fold cross-validation on the 48-image dataset.

• Detection Performance: Achieved a mean F1 score of 0.893 ± 0.011.
• Localization Accuracy: Achieved a mean localization error of 1.41 ± 0.06 mm.
  • Comparison: This outperforms the previous state-of-the-art (Kim et al., 1.77 mm error) despite using a much smaller dataset and a different target type (indoor vs. outdoor).
• Downstream Metrics:
  • Scoring: The system recovered average arrow scores with negligible bias (mean signed error of -0.001 points).
  • Grouping: The centroid error was 4.90 mm, well within a single ring width (20 mm), allowing for the detection of systematic aiming bias.
• Efficiency: The model uses only 3.8M trainable parameters out of 308M total, fitting on consumer GPUs and training in ~30 minutes.

5. Significance and Limitations

Significance:
The paper validates a paradigm where frozen foundation models act as powerful feature extractors for specialized, low-data vision tasks. By offloading geometric complexity to a classical preprocessing step and leveraging self-supervised features, the system achieves high precision without the data hunger of traditional supervised deep learning. This is highly relevant for niche applications where collecting large datasets is impractical.

Limitations:

• Generalization: The dataset comes from a single archer and venue; performance on different lighting, cameras, or target wear patterns is unverified.
• Target Specificity: The rectification pipeline is hardcoded for the 40cm indoor target; other target types would require re-engineering the color boundary definitions.
• Failure Modes: The system struggles with "compound punctures" (overlapping arrows) where Gaussian responses merge, and occasionally produces false positives on heavily damaged paper.
• Evaluation: The study lacks a completely held-out test set, relying on cross-validation which may introduce slight optimistic bias in hyperparameter selection.

Conclusion

The authors successfully built a system for automated archery scoring that achieves sub-millimeter accuracy on a tiny dataset. The combination of geometric rectification, frozen DINOv3 features, and guided upsampling proves to be a robust solution for dense prediction in data-scarce environments, offering a practical alternative to training massive models from scratch.