Insights on back marking for the automated identification of animals

1. Problem Statement

While machine learning (ML) has advanced the automated monitoring of animal social behavior, a significant bottleneck remains: individual identification of uniform-looking species (e.g., pigs). Traditional group-level monitoring is insufficient for detailed behavioral analysis. Although physical markers like back marks are a common solution, there is a lack of empirical research and guidelines on how to design these marks to ensure they are robustly recognized by computer vision algorithms under real-world conditions. The core challenge is designing marks that remain distinguishable despite motion blur, varying viewing angles, occlusions, and the specific data augmentation strategies used during model training.

2. Methodology

The study employed a controlled experimental setup to analyze the performance of a deep learning model on specific back mark designs.

• Experimental Setup:
  • Subject: Pigs in an observational study on social behavior ("Let me out" project).
  • Hardware: Two identical pens monitored by elevated side-view fisheye cameras (HIKVISION DS 2CD5046G0-AP, 1200x780@25fps).
  • Back Marks: Ten unique symbols were applied to the pigs' backs (inspired by mouse patterns). The set included: dot dot, dot line horizontal, i, line line horizontal, o, reverse t, s, v, vertical line, x. Symbols were chosen over numbers for ease of application and reproducibility.
• Data Preparation:
  • Dataset: 11 video clips (10–30 seconds each) were annotated by a human labeller with bounding boxes and class IDs.
  • Split: 9 clips for training, 1 for validation, and 1 for testing.
  • Cropping: Raw frames were processed to extract bounding box crops. The final dataset consisted of 26,260 training crops, 500 validation crops, and 500 test crops.
• Model Architecture:
  • A ResNet-50 neural network (implemented in PyTorch) was trained for image classification.
• Training Strategy:
  • Extensive data augmentation was applied to simulate real-world variability, including:
    • Geometric: Horizontal/vertical flipping, random rotation, and cropping.
    • Photometric: Brightness, contrast, saturation, hue modifications, grayscale conversion, and blurring.

3. Key Results

The trained model demonstrated a significant disparity between validation and test performance, highlighting the difficulty of generalization in dynamic environments.

• Overall Accuracy:
  • Validation Set: 91%
  • Test Set: 69%
• Class-Specific Performance:
  • Performance varied drastically across symbols. For example, the "vertical line" achieved 88% accuracy on the test set, whereas the "reverse t" dropped to only 38%.
• Error Analysis (Failure Modes):
  The study identified three primary behavioral factors causing misclassification:
  1. Motion Blur: Fast movement caused the "reverse t" to blur into a shape resembling a "vertical line." Pixel importance analysis showed the model ignored the blurred vertical bar of the 'T', relying only on the horizontal component.
  2. Viewing Angles: Specific poses caused "reverse t" to visually resemble "v" or "x," leading to low model confidence.
  3. Occlusion: Partial occlusion by other animals made it impossible for the model to distinguish between "line line horizontal" and "o".
• Impact of Data Augmentation:
  • Color/Grayscale: Augmentations like color jitter and grayscale conversion can render colored back marks indistinguishable.
  • Cropping: Random cropping can remove critical features, making "reverse t" look like a "vertical line" or "line line horizontal" look like "o".
  • Flipping: Horizontal flipping creates symmetry issues; for instance, a mirrored "s" can look like a different class (or a non-existent class), causing confusion if the mark set is not designed to be flip-invariant or distinct under flipping.

4. Key Contributions & Design Guidelines

The paper provides actionable guidelines for designing back marks for automated livestock monitoring, moving beyond trial-and-error to data-driven design:

1. Behavioral Robustness: Marks must be designed to withstand predictable animal behaviors:
   • Avoid shapes that blur into one another (e.g., avoid "reverse t" if "vertical line" is in the set).
   • Ensure distinctiveness across diverse poses and viewing angles.
   • Maintain identifiability even when partially occluded.
2. Augmentation Compatibility: Mark design must account for standard ML training augmentations:
   • Avoid reliance on specific colors if grayscale augmentation is used.
   • Ensure marks remain unique after random cropping (critical features must be central or redundant).
   • Avoid pairs of marks that are mirror images of each other if horizontal flipping is used.

5. Significance

This study bridges the gap between computer vision algorithm development and practical animal husbandry. By analyzing error cases in a controlled setting, the authors demonstrate that mark design is as critical as model architecture for successful individual identification. The insights provided allow researchers and industry practitioners to optimize back mark sets, thereby improving the reliability of individual-level monitoring in Precision Livestock Farming (PLF). This reduces the need for retraining models due to poor data quality and facilitates the deployment of robust, automated behavioral analysis systems in real-world farming environments.