Different Paradigms from Computer Vision Align with Human Assessment of the Mouse Grimace Scale

This study bridges the gap between automated computer vision and human assessment of the Mouse Grimace Scale by evaluating three dominant paradigms, demonstrating that they achieve reliable binary classification of mouse well-being with low error rates while focusing on relevant facial features beyond the traditional scale.

The Gist

Imagine you are a caretaker for a large group of mice living in a research lab. Your most important job is to make sure they are happy and not in pain. But mice are tricky: they are nocturnal (active at night), they hide their pain well, and they often act differently when humans are watching.

To solve this, scientists developed a "Mouse Grimace Scale" (MGS). Think of it like a human pain scale, but instead of asking the mouse "How much does it hurt?", you look at its face. If a mouse is in pain, it squints its eyes, flattens its ears, and changes the shape of its nose and whiskers.

The Problem:
Currently, humans have to look at thousands of photos of mice and manually score these facial changes. It's slow, tiring, and prone to human error (we get tired, or we might miss a subtle sign). We need a robot to do this job, but we need to make sure the robot is actually looking at the right things and not just guessing.

The Solution (This Paper):
The researchers in this paper acted like a "Taste Test" for three different types of computer vision (AI) systems. They wanted to see which "brain" was best at looking at a mouse photo and saying, "This mouse is in pain" or "This mouse is fine."

They tested three different approaches, which we can imagine as three different ways a detective might solve a mystery:

1. The "Supervised Detective" (Supervised Learning)

• How it works: This AI was trained like a student in a classroom. Humans showed it thousands of photos and said, "See this squint? That's pain. See this relaxed ear? That's fine." It learned by memorizing these examples.
• The Result: It did a great job, getting about 80% accuracy. It was a reliable student.

2. The "Self-Taught Explorer" (Self-Supervised Learning)

• How it works: This AI wasn't given a textbook. Instead, it was thrown into a library of millions of images and told to find patterns on its own, without anyone telling it what "pain" looks like. It learned to understand shapes, textures, and features just by looking at the world. Then, it was tested on the mice.
• The Result: This was the winner. It performed slightly better than the supervised detective (about 83-84% accuracy). It seems that letting the AI learn the "language" of images on its own first makes it a sharper observer.

3. The "Geometry Measurer" (Landmark Locations)

• How it works: Instead of looking at the whole picture, this AI was taught to find specific dots on the mouse's face (like the tip of the nose, the corner of the eye, the tip of the ear). It then measured the distances between these dots to calculate pain.
• The Result: This one struggled the most (only about 63% accuracy). It was like trying to diagnose a headache just by measuring the distance between a person's eyebrows. It missed the big picture.

The "X-Ray Vision" Check (Qualitative Assessment)

The researchers didn't just trust the scores; they used a special tool to see what the AI was looking at when it made a decision. They turned the AI's "attention" into a heat map (like a thermal camera).

• Good News: The winning AIs were looking at the mouse's face, not the cage or the background. They focused on the eyes, ears, and whiskers—exactly what a human expert would look for.
• Bonus Discovery: The AI also noticed things humans hadn't explicitly taught it. For example, it noticed if the mouse's fur was standing up (a sign of stress) or if the shape of its upper lip looked tense. It even noticed if there was food in the mouse's face (which means the mouse was burrowing and active, a sign of good health).

The Bottom Line

The paper concludes that we can trust computers to monitor mouse welfare. The "Self-Taught Explorer" AI is the best tool for the job. It can work 24/7, it doesn't get tired, and it can spot subtle signs of pain that a tired human might miss.

Why does this matter?
If we can automate this, we can catch pain earlier, treat the mice faster, and ensure that scientific research is ethical and humane. It's like giving every mouse in the lab a 24-hour bodyguard that never blinks.

Technical Summary

1. Problem Statement

Animal welfare in research relies heavily on the Mouse Grimace Scale (MGS), a manual method where experts assess pain and distress based on five facial action units (FAUs): orbital tightening, nose bulge, cheek bulge, ear position, and whisker change.

• Limitations: Manual scoring is labor-intensive, impractical for large-scale or continuous (24/7) monitoring, and susceptible to human bias and fatigue.
• Research Gap: While automated Computer Vision (CV) methods exist, there is a lack of systematic comparison between different CV paradigms under standardized conditions. Previous studies often used different datasets, preprocessing pipelines, and evaluation protocols, making it difficult to determine which approach is most reliable or interpretable. Furthermore, the "black box" nature of many models raises concerns about whether they are actually detecting pain-related features or merely artifacts (e.g., background, lighting).

2. Methodology

The authors evaluated three dominant computer vision paradigms on a large, diverse dataset of mouse face images to perform binary classification (Well-being: Impaired vs. Unimpaired).

Dataset

• Source: Reused a dataset of ~35,000 images from Andresen et al., covering five mouse strains (BALB/c, C57BL/6J, etc.), various treatments (anesthesia, surgery, etc.), and multiple labs.
• Preprocessing:
  • Filtered to images where at least 3 of 5 FAUs were visible and the "orbital tightening" FAU was scored.
  • Converted discrete MGS scores (0–2) into binary labels using a threshold: Mean MGS $\ge$ 0.6 = Impaired; otherwise = Unimpaired.
  • Final dataset: 3,286 images (1,389 impaired, 1,897 unimpaired).

The Three Paradigms

1. Supervised Learning (SL):
   • Model: ResNet-50 pretrained on ImageNet.
   • Approach: Transfer learning where the final layer is replaced to classify directly from raw pixel features.
2. Self-Supervised Learning (SSL):
   • Model: ResNet-50 pretrained on ImageNet using the Barlow Twins criterion (self-supervised).
   • Approach: Similar to SL, but the feature encoder learns representations without human annotations before fine-tuning for the binary task.
3. Landmark Locations (Pose Estimation):
   • Model: DeepLabCut (DLC) using HRNet-W18 architecture.
   • Approach: Detected 19 specific anatomical landmarks (e.g., eye canthi, ear apex, nose tip). These coordinates were normalized (translated, rotated, scaled) and fed into a linear classifier (linear probing) to predict the binary state.

Evaluation Metrics

• Quantitative: Precision, Recall, F1-score (macro-averaged), and specifically Type II Error (False Negative Rate) for the "Impaired" class. Low Type II error is critical to ensure painful animals are not missed.
• Qualitative: Layer-wise Relevance Propagation (LRP) was used to generate heatmaps, visualizing which pixels contributed most to the model's decision.

3. Key Results

Quantitative Performance

• Self-Supervised Learning (SSL) achieved the best overall performance:
  • Precision: 0.83, Recall: 0.84, F1: 0.83.
  • Type II Error (False Negatives): 16% (0.16).
• Supervised Learning (SL) performed nearly as well:
  • Precision: 0.80, Recall: 0.80, F1: 0.80.
  • Type II Error: 20% (0.20).
• Landmark Locations performed significantly worse:
  • Precision: 0.68, Recall: 0.68, F1: 0.63.
  • Type II Error: 36% (0.36).
• Observation: All paradigms struggled more with the "Impaired" class than the "Unimpaired" class, but SSL and SL maintained acceptable error rates for automated screening.

Qualitative Analysis (Model Attention)

Using LRP heatmaps, the authors verified that the models focused on biologically relevant features:

• MGS Features: The models correctly attended to orbital tightening (eyes), ear position (outline and rotation), whisker pads, and nose bulge.
• Novel Features: The models also utilized features not explicitly defined in the MGS but relevant to welfare:
  • Fur texture: Detecting piloerection (raised fur).
  • Snout shape: The ventral outline of the snout/upper lip (tense/angular vs. round/relaxed).
  • Contextual cues: The models occasionally used background cues (e.g., surgical clips, food pellets in the face indicating burrowing behavior) but did not overfit to them, suggesting robustness.
• Comparison: SSL models focused more on foreground features (mouse body/fur), while SL models sometimes relied more on background features.

4. Key Contributions

1. Systematic Benchmarking: First direct comparison of SL, SSL, and Landmark-based approaches on a unified, diverse dataset for MGS assessment.
2. Validation of Self-Supervised Learning: Demonstrated that SSL (Barlow Twins) outperforms traditional supervised transfer learning and landmark-based methods for this specific task, achieving a low 16% false-negative rate.
3. Interpretability & Trust: Provided visual evidence (heatmaps) that automated models align with human expert reasoning, focusing on the mouse's face rather than environmental artifacts.
4. Discovery of New Indicators: Identified that the ventral outline of the snout/upper lip and fur texture (piloerection) are significant visual indicators of distress, expanding the scope of welfare assessment beyond the original MGS.
5. Accessibility: Showed that these models require minimal computational resources (can run on standard desktops/laptops), making them accessible to general animal research facilities.

5. Significance

This work bridges the gap between computer vision and veterinary medicine, validating that automated systems can reliably replace or augment manual MGS scoring.

• Animal Welfare Impact: By achieving low Type II error rates, these systems can enable continuous, 24/7 monitoring of mice in home cages, ensuring that pain and distress are detected immediately, even at night when humans are not present.
• Scientific Rigor: The study confirms that AI models do not just "cheat" by learning dataset biases but actually learn the biological markers of pain, increasing trust in adopting these tools for regulatory and ethical compliance in animal research.
• Future Directions: The identification of new visual features (snout shape, fur texture) suggests that the MGS could be refined or expanded to include these parameters for even more accurate welfare assessment.