💻 computer science

Oh Deer, How Should I Handle This? Seasonal Priors for Selective Wildlife Annotation and Classification

This paper demonstrates that integrating biologically grounded seasonal priors with multi-modal (RGB and thermal) data significantly enhances the reliability of wildlife annotation and classification for red deer by identifying predictable windows of evidence, resolving ambiguous labels through human review, and optimizing selective prediction to achieve high accuracy while managing uncertainty.

Original authors: Hugo Markoff, Christoph Praschl, Anton Hjalte Jørgensen, Christian Emil Mogensen, Mathias Bech Skadhauge, Sara Beery, Michael Ørsted, David C. Schedl

Published 2026-08-05

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Hugo Markoff, Christoph Praschl, Anton Hjalte Jørgensen, Christian Emil Mogensen, Mathias Bech Skadhauge, Sara Beery, Michael Ørsted, David C. Schedl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a wildlife detective trying to solve a mystery from the sky. You have a drone hovering high above a forest, snapping pictures of deer. Your job isn't just to count them; you need to figure out who is a grown-up male, who is a female, and who is a baby. This is the world of aerial wildlife monitoring, a field where scientists use drones to count animals without scaring them away. But there's a catch: from high up, a deer looks like a tiny speck, and telling a boy deer from a girl deer is incredibly hard. This is called fine-grained classification. It's like trying to tell if a person in a crowd is wearing a red hat or a blue hat when they are a mile away.

To make this even trickier, nature plays hide-and-seek. Male deer grow big, bony antlers on their heads, which are the main clue to their identity. But these antlers aren't always there. They fall off in winter, grow back slowly in spring, and get hard and shiny by summer. This is the antler cycle, a seasonal rhythm that changes what the deer look like. If you try to identify them when they have no antlers or fuzzy new ones, even a human expert might get confused. This paper asks a simple but vital question: Can we use our knowledge of these seasons to help computers and humans make better guesses, and when should we just admit, "I don't know"?

The Great Deer Identity Crisis

Meet the red deer. In the wild, telling a grown-up male from a female or a baby is usually a job for the antlers. But when you are looking at them from a drone, the view is tiny and the details are fuzzy. The researchers behind this study decided to treat this not just as a computer problem, but as a "seasonal puzzle." They realized that the clues change depending on the month, just like the weather.

They set up a massive experiment with 7,295 tiny pictures of deer. They didn't just take one photo; they took two for every deer: one in normal color (RGB) and one in heat-vision (thermal). Think of it like taking a photo with a regular camera and a night-vision camera at the same time. Then, they brought in three human experts to look at these pictures and guess the deer's gender.

Here is the big surprise they found: Looking at both cameras together is a game-changer.

When the humans looked at the color photos alone, they could only confidently solve the gender mystery for about 32% of the deer. When they looked at the heat-vision photos alone, it was even worse, at 27.3%. But when they were allowed to look at both photos of the same deer at the same time? Suddenly, they could solve 52.8% of the cases. It's like trying to solve a jigsaw puzzle in the dark; with one light, you see a few pieces, but with two lights from different angles, the whole picture snaps into place. The heat camera could see the shape of the head when the color camera was blinded by shadows, and the color camera could see the texture of the antlers when the heat camera was confused by the warm fur.

The Seasonal Calendar of Clues

The paper introduces a clever idea called a "seasonal calendar." Imagine a clock face where each hour represents a month of the year. The researchers mapped out exactly when the deer's antlers are easy to see and when they are invisible.

Winter (The "Lost" Season): In late winter and early spring, the antlers fall off or are just tiny nubs. During this time, the color camera is basically useless for spotting males because there's nothing to see. The heat camera is also confused because the new, fuzzy antlers (called "velvet") are warm and look like the rest of the head.
Summer (The "Hard" Season): By late summer, the antlers are hard, clean, and white. Now, the color camera is the star of the show. The heat camera, however, gets less helpful because the antlers cool down and blend in with the rest of the body.
The "Velvet" Phase: In spring and early summer, the antlers are growing fast and are covered in fuzzy skin that is very hot. This is the heat camera's moment to shine.

The authors found that this calendar predicts exactly when humans and computers will struggle. When the calendar says "uncertain," the computer's confidence drops, and the human experts are more likely to say, "I'm not sure." It's as if the deer are wearing a disguise that changes with the seasons, and the calendar tells you which disguise is the hardest to see through.

The "I Don't Know" Button

One of the most interesting parts of the study is how they handled the times when the clues were too weak. Instead of forcing the computer to guess and risk being wrong, they gave it an "abstention" button. This is like a student raising their hand in class and saying, "I don't know the answer," instead of guessing and getting it wrong.

When the computer was allowed to say "I don't know" for the tricky cases (those inside a specific "uncertainty band"), the accuracy of the answers it did give skyrocketed to 98.9%. That is incredibly high! However, there was a catch. By saying "I don't know" so often, the computer stopped making guesses on about 25% of the deer.

Here is the twist: The computer was much more likely to say "I don't know" about the male deer than the females. Since there are fewer males in the group to begin with, this meant that when the computer did make a guess, it was almost always right, but it was also skipping a lot of the males. It's like a security guard who is so careful they only let in people they are 100% sure are guests, but they accidentally turn away half the actual guests because they look a little suspicious.

What the Computer Learned (and What It Didn't)

The researchers tested different types of "brains" (computer models) to see which one was best at this job. They found that:

Fusion is King: Combining the color and heat images always worked better than using just one.
The "Frozen" Brain: They used a pre-trained computer brain (DINOv3) that was already good at looking at pictures. When they used this brain, the color camera was usually better than the heat camera.
The Surprise: But when they trained a different kind of brain (YOLO) from scratch specifically for this task, the heat camera actually became the winner, beating the color camera by a significant margin. This suggests that the "best" camera depends entirely on how you teach the computer to look.

The Verdict

So, what is the takeaway for our deer detectives?

The paper suggests that we shouldn't just throw all our data into a computer and hope for the best. Instead, we should use biology to guide our technology. By knowing the "seasonal calendar" of the deer's antlers, we can:

Tell human annotators when to be extra careful (like during the shedding season).
Understand why a computer might be confused at a specific time of year.
Decide when to combine our camera views to get the best possible answer.

The study doesn't claim to have solved the problem of identifying every single deer perfectly. In fact, it explicitly rules out the idea that we can just rely on one camera or one type of computer model. It shows that uncertainty is real and seasonal. The best strategy is to accept that sometimes the clues are missing, to use multiple cameras to fill in the gaps, and to have the courage to say "I don't know" when the evidence isn't there. It's a reminder that in the wild, nature is messy, and the smartest tool we have is knowing when to trust our eyes and when to wait for a better view.

Technical Summary: Oh Deer, How Should I Handle This? Seasonal Priors for Selective Wildlife Annotation and Classification

Problem Statement

Fine-grained wildlife classification in aerial imagery is hindered not only by model performance but fundamentally by unreliable labels. Animals often occupy few pixels, and key visual cues vary significantly across seasons. In the specific context of red deer (Cervus elaphus), the primary visual cue for sex identification—antler visibility—follows a predictable annual cycle (casting, regrowth, hardening, and shedding). This seasonality creates windows where evidence is ambiguous or absent, leading to systematic labeling errors by human annotators and low confidence in downstream models. Current approaches often treat these ambiguities as static noise rather than leveraging the biological calendar to guide annotation protocols and modality weighting. The core challenge is to determine how seasonal structure links annotation quality, downstream classification, and selective prediction, particularly when distinguishing adult males from females/juveniles using RGB and thermal imagery.

Methodology

Dataset and Annotation Protocol

The study utilizes a public aerial dataset containing 7,295 aligned crop sets of red deer captured via UAVs equipped with co-mounted RGB and non-radiometric thermal sensors. The data spans nine months (January, February, March, April, May, July, September, November, December).

Three-View Annotation: Each sample is independently reviewed by three ecologically trained annotators under three conditions: RGB-only, thermal-only, and a matched RGB+thermal view.
Labeling Scheme: Annotators assign one of four labels: male, female/juvenile, uncertain, or occluded.
Resolution Strategy: A "resolved" label is established via majority vote within a view. The matched view (combining both modalities) serves as the reference standard for the evaluation pool, resolving samples that remain ambiguous in single-modality views. Samples with no majority are excluded from the evaluation pool but tracked to quantify disagreement.

Seasonal Priors

The authors define "soft seasonal priors" based on the biological antler cycle (Figure 1). These priors do not assign labels or shift predictions at inference but define expected evidence quality:

RGB: Most informative during "hard antler" periods (late summer/autumn).
Thermal: Most informative during "velvet growth" (spring/early summer) due to high vascular activity and heat contrast.
Uncertain Periods: Late winter/early spring (casting/regrowth) where antlers are absent or indistinct in both modalities.

Models and Representations

The study evaluates three classifier families on frozen embeddings derived from a DINOv3 ViT-H+ backbone:

Linear SVM: Trained on concatenated RGB and thermal embeddings (late fusion) or single-modality embeddings.
Triplet Metric Learning: Learns a metric space using class centroids.
YOLO26x: A task-trained image classifier (encoder varies) to test if findings are representation-specific.

Channel Replacement: An early-fusion control where thermal intensity replaces one RGB channel (R, G, or B) before encoding.

Selective Prediction

The study employs an uncertainty-band abstention mechanism. For a scalar score $s(x)$ , samples falling within $|s(x) - 0.5| \le \delta$ are abstained (deferred) rather than forced into a binary class. This allows for the analysis of covered accuracy (accuracy on non-abstained samples) versus coverage (fraction of samples retained).

Key Results

1. Annotation Quality and Matched Review

Matched RGB+thermal review significantly outperforms single-modality annotation:

Resolution Rate: Matched review resolves 52.8% of crop sets, compared to 32.0% for RGB-only and 27.3% for thermal-only.
Label Changes: Matched review recovers 99 majority-male labels missed by RGB and 249 missed by thermal. Conversely, it changes 22 RGB and 18 thermal male calls to non-male labels once the paired view is available. As the matched vote comes from the same three annotators, these are label changes rather than verified corrections, though their direction aligns with complementary evidence predictions.
Seasonal Complementarity: Failure modes are not identical; thermal struggles in winter (low male detection), while RGB struggles during casting/regrowth. Matched review stabilizes performance across these periods.

2. Classification Performance and Modality Ordering

Fusion Superiority: Matched (fused) features consistently yield the highest accuracy (95.4% full coverage, 98.9% covered accuracy at 75.3% coverage) across all classifier families.
Representation Dependency: The ordering of single modalities depends on the encoder.
- With frozen DINO embeddings (RGB-pretrained), RGB outperforms thermal (e.g., SVM: 93.6% vs. 84.1%).
- With task-trained YOLO, thermal outperforms RGB (92.4% vs. 86.3% accuracy).
Channel Replacement: Injecting thermal data into the Blue channel of an RGB image (Thermal $\to$ B) slightly outperforms plain RGB (94.8% vs. 93.6%), suggesting the blue channel is the least disruptive slot for thermal signal injection in RGB-pretrained backbones.

3. Selective Prediction and Abstention

Accuracy-Coverage Trade-off: Tightening the uncertainty band improves covered accuracy but reduces coverage. At 80% coverage, matched views achieve 98.6% accuracy.
Class-Specific Bias: Abstention rules disproportionately defer male samples. For the Linear SVM on matched data, the male reject rate is 45.9% versus 21.2% for females/juveniles at a fixed band. This indicates that high covered accuracy is partly achieved by discarding the most difficult (male) cases.
Uncertainty Alignment: Model uncertainty (fraction of samples in the abstention band) broadly aligns with human annotator uncertainty and the seasonal priors. July (hard antlers) is the least uncertain month; winter months show elevated uncertainty.

4. Impact of Seasonal Priors

Training Weights: Applying soft priors as sample weights during training (up-weighting "safe" periods, down-weighting "uncertain" periods) yields small but consistent gains in the expected direction (e.g., +0.2 to +0.5 accuracy points for RGB and thermal under specific boost schedules).
Operational Use: The paper concludes that while priors are useful for diagnostics (guiding annotators on when to be cautious) and provide consistent, albeit small, improvements when used as training weights, their direct impact on model performance is secondary to the benefits of matched annotation and confidence-based abstention.

Significance and Claims

The paper claims that a biologically grounded seasonal calendar is a critical, underutilized tool for wildlife monitoring. Its primary significance lies in:

Predicting Unreliability: The antler cycle successfully predicts when annotation will be difficult and when model confidence will be low, allowing for proactive protocol design.
Justifying Matched Review: It demonstrates that matched RGB+thermal review is not merely a data augmentation technique but a necessary step to resolve ambiguity that single modalities cannot, roughly doubling the share of crops that can be resolved (from ~30% to ~53%).
Refining Evaluation: The study highlights that reporting overall accuracy or even "covered accuracy" without class-specific reject rates is misleading, as high accuracy can mask the systematic deferral of minority classes (males).
Modest Role of Priors: The authors note that while seasonal priors align with uncertainty and yield consistent positive gains when used as training weights, their direct impact on model performance is secondary to the benefits of matched annotation and confidence-based abstention.

The work establishes that fine-grained wildlife classification is as much a problem of evidence sufficiency and label quality as it is of model architecture, and that ecological knowledge should be integrated into the annotation and evaluation pipeline.