Automatic pain face analysis in mice: Applied to a varied dataset with non-standardized conditions

This study introduces a deep learning model trained on a large, diverse dataset that outperforms human raters in automatically assessing Mouse Grimace Scale scores across varied strains and conditions, demonstrating that combining multiple subsets yields the most reliable performance for non-standardized pain assessment.

The Gist

Imagine you are a detective trying to solve a mystery, but your suspect is a tiny mouse that can't speak. The mystery? Is the mouse in pain?

For decades, scientists have used a tool called the Mouse Grimace Scale (MGS) to solve this. It's like a "pain emoji" chart for mice. If a mouse is hurting, its face changes in five specific ways: its eyes squint tight, its nose and cheeks puff out, its ears stick out, and its whiskers straighten or curl back.

Usually, a human has to look at a photo of the mouse and manually score these changes. But humans get tired, they might be biased, and they can't watch the mouse 24 hours a day. Plus, if a human walks into the room, the mouse might get scared and hide its pain, just like a kid might stop crying when a parent walks in.

The Problem: The "One-Size-Fits-All" Failure
Scientists tried to build a computer program (an AI) to do this job automatically. But here's the catch: most of these AI programs were like students who only studied for one specific test.

• If you trained an AI on black mice in a bright lab, it would fail miserably when shown a white mouse in a dim cage.
• If it learned from mice that had surgery, it might get confused by mice that just had a blood test.
• The real world is messy. Mice come in different colors (black, white, brown), live in different cages, and are filmed with different cameras. The AI was getting confused by all these variables, like a chef who only knows how to cook with a specific brand of stove and fails when given a different one.

The Solution: The "Super-Student" AI
This paper introduces a massive new project designed to create a "Super-Student" AI that can handle any situation.

1. The Massive Library: Instead of looking at a few hundred photos, the researchers gathered a library of 35,000 images from five different laboratories. These images feature mice of different breeds, different fur colors, different ages, and different medical treatments. It's like training a student not just on one textbook, but on every book in the library, written in different languages and styles.
2. The Training Method: They didn't just throw the photos at the computer. They used a clever "two-step" training method:
   • Step 1 (The Warm-up): First, they taught the AI a simple game: "Is this mouse in pain or not?" (Yes/No). This helped the AI learn to recognize the general "vibe" of a hurting mouse.
   • Step 2 (The Exam): Once the AI understood the basics, they taught it the hard math: "Give me a specific score from 0 to 2 for how much pain it's in."
3. The Result: The AI became incredibly good. When tested on new, unseen mice, it made an average error of 0.26 on a scale of 0 to 2.
   • The Analogy: Imagine a human expert guessing the temperature of a room. They might be off by a degree or two. This AI was actually more accurate than the average human expert! It correlated with human experts at a level of 85%, which is like two best friends finishing each other's sentences.

The "Cross-Training" Surprise
The researchers also tested if the AI could handle a "cross-dataset" challenge. This is like taking a student who studied in New York and testing them in Tokyo.

• The Bad News: If you trained the AI on just one type of mouse (e.g., only black mice), it struggled when shown a white mouse.
• The Good News: The AI trained on the combined mix of all five groups was the most robust. It learned to ignore the "noise" (like the color of the cage or the fur) and focus only on the "signal" (the actual pain expression).
• The Twist: Interestingly, the AI didn't get better by focusing only on the easiest feature (squinting eyes). It actually performed better when it looked at the whole face, just like a human does.

Why This Matters
This is a huge step forward for animal welfare.

• 24/7 Monitoring: Instead of a human checking the mouse once a day, this AI could watch the mouse 24 hours a day, even while the mouse is sleeping or in the dark (when mice are most active).
• No Stress: The mouse doesn't need to be taken out of its home cage and put in a scary box for a photo. The AI can analyze the mouse right in its home, meaning the mouse stays calm and shows its true feelings.
• Better Science: If we know exactly when a mouse is in pain, we can treat it faster, and the scientific data we get from experiments is more accurate because pain doesn't skew the results.

In a Nutshell
The researchers built a "universal translator" for mouse faces. By feeding a computer a massive, diverse diet of images, they taught it to understand the universal language of pain, regardless of the mouse's color, size, or where it lives. This allows for a kinder, more scientific way to care for our tiny laboratory friends.

Technical Summary

1. Problem Statement

The assessment of pain and distress in laboratory mice is critical for ethical compliance and scientific data integrity. The Mouse Grimace Scale (MGS) is the gold standard for this assessment, relying on five facial action units (FAUs): orbital tightening, nose bulge, cheek bulge, ear position, and whisker change. However, current MGS application is manual, time-consuming, and limited to short observation windows, often influenced by human presence which can alter mouse behavior.

While automated computer vision tools for MGS exist, they face significant limitations:

• Lack of Generalization: Most existing models are trained on highly standardized datasets (specific mouse strains, standardized lighting, and cage-side recording setups).
• Non-Standardized Conditions: They fail to perform reliably under "real-world" home cage conditions or across diverse mouse strains (e.g., albino vs. black fur), varying experimental treatments, and different imaging hardware.
• Data Scarcity: There is a lack of large, diverse datasets that capture the variability inherent in non-standardized laboratory environments.

2. Methodology

Dataset Construction

The authors introduced a large, diverse dataset comprising approximately 35,000 images of mouse faces, divided into five distinct subsets (AW, JW, KH, LW, MR).

• Diversity: The dataset includes five different mouse strains (C57BL/6N, BALB/c, C57BL/6J, NMRI, DBA/1) with varying fur colors (black, albino, dilute brown).
• Conditions: Data was collected across five different laboratories with varying housing conditions, enrichment items, and imaging setups (infrared video, RGB still photos, different observation cages).
• Labeling: Images were labeled by human raters using the MGS (0–2 scale for each of the 5 FAUs). The primary target variable was the average MGS score across all FAUs.
• Preprocessing:
  • Frame Selection: For video subsets, a DeepLabCut-based detector selected frames with the highest confidence for facial features (eyes, ears, nose), enforcing temporal spacing to ensure variety.
  • Face Detection: Bounding boxes were generated around faces, scaled to include the full face, and ranked by blurriness (using Fast Fourier Transform) to select the clearest images.
  • Impairment Labeling: A binary "impaired vs. unimpaired" label was derived from MGS scores to serve as a pretext task for transfer learning.

Model Architecture and Training

The study employed a Transfer Learning approach using a ResNet-50 backbone. The training pipeline consisted of three stages:

1. Pretext Task 1 (Object Recognition): Initialization with weights pre-trained on ImageNet-21k (Big Transfer BiT-M-R50x1).
2. Pretext Task 2 (Binary Classification): Fine-tuning on the dataset to classify images as "impaired well-being" vs. "unimpaired well-being" based on the derived labels.
3. Main Task (Regression): Replacing the output head with a regression layer to predict the continuous average MGS score (0.0 to 2.0).
   • Augmentation: RandAugment was used to increase robustness against visual variations.
   • Training: Models were trained for 200 epochs.

Evaluation Metrics

• Root Mean Squared Error (RMSE): To measure the average deviation from human scores.
• Pearson Correlation Coefficient (r): To measure the linear relationship between model predictions and human scores.
• Inter-Rater Reliability (IRR): Calculated using Cohen's and Fleiss' Kappa to establish a human performance baseline.
• Cross-Dataset Evaluation: Models trained on one or more subsets were tested on held-out subsets to assess generalization.

3. Key Contributions

1. Novel Dataset: Publication of a large-scale, heterogeneous dataset (~35k images) covering multiple strains, fur colors, and non-standardized experimental conditions, addressing a major gap in the field.
2. Robust Deep Learning Model: Development of a deep learning model capable of predicting average MGS scores across diverse conditions, outperforming the average human rater in error magnitude.
3. Analysis of Generalization: A systematic evaluation of how model performance degrades when moving between subsets (cross-dataset) and the impact of training on combined vs. single subsets.
4. Feature Analysis: Investigation into whether focusing solely on the most reliable feature (Orbital Tightening) improves performance compared to using all facial features.

4. Results

Performance on Training Data

• Accuracy: When trained and tested on the combined dataset, the model achieved an RMSE of 0.26 and a Pearson correlation of 0.85.
• Human Comparison: The model's error (0.26) was lower than the average inter-rater error among human experts (which ranged from 0.28 to 0.39 depending on the subset), indicating the model is at least as consistent as human raters.
• Subset Variability: Performance varied by subset. Subset LW (mostly low pain scores) showed the lowest RMSE (0.16) but also the lowest correlation (0.24), suggesting a "low-score bias" where the model simply predicted the mean. Subsets with wider score distributions (KH, MR) showed higher correlations (>0.8).

Cross-Dataset Evaluation

• Generalization Gap: Models trained on a single subset performed significantly worse when tested on other subsets (RMSE increased to 0.37–0.63).
• Combined Training: Training on a combination of all subsets improved generalization. The most reliable model for a novel dataset was the one trained on the combined subsets.
• Fine-Tuning Potential: The authors noted that performance could be further enhanced by fine-tuning the pre-trained model with a small portion of human-labeled data from the new target dataset.

Feature-Specific Analysis (Orbital Tightening Only)

• Contrary to the hypothesis that focusing on the most reliable feature (Orbital Tightening) would improve results, models trained only on Orbital Tightening generally showed higher RMSE than models trained on all features.
• However, these models showed higher correlation coefficients in some cases, suggesting they might be capturing the trend well but with a systematic bias or larger variance in absolute error.

5. Significance and Conclusion

This work represents a significant step toward fully automated, 24-hour pain monitoring in laboratory mice under non-standardized conditions (e.g., home cages).

• Ethical Impact: By enabling continuous, automated monitoring without human intervention, the system reduces stress on animals (which can mask pain) and allows for earlier detection of distress.
• Scientific Rigor: The study demonstrates that while standardized datasets are easier to model, diverse, non-standardized datasets are essential for building robust, transferable AI tools.
• Future Directions: The authors conclude that the most effective strategy for deploying these tools in new laboratories is to train on a large, diverse combined dataset and then perform fine-tuning with a small amount of local, human-labeled data. The dataset and model are made publicly available to accelerate further research in automated animal welfare assessment.