Using machine learning to automate the analysis of an olfactory habituation-dishabituation task in mice

This paper presents and validates a machine learning pipeline combining DeepLabCut and SimBA to accurately automate the analysis of mouse olfactory habituation-dishabituation tasks, achieving results comparable to manual annotation while improving research throughput.

The Gist

Imagine you are a detective trying to solve a mystery about mouse memory. Your clue? How long a mouse stops to sniff a new smell. This is called an "olfactory habituation-dishabituation" task. In simple terms: if you show a mouse the same smell three times, it gets bored and stops sniffing (habituation). But if you show it a new smell, it gets excited and sniffs again (dishabituation). This tells scientists if the mouse's brain is working correctly.

The Problem:
Traditionally, to get this data, a human had to sit in front of a screen for hours, watching black-and-white videos of mice in cages. They would manually click a stopwatch every time a mouse sniffed a cotton swab. It was slow, boring, and prone to human error (like getting tired or distracted). Plus, the videos were taken from the side, meaning the mouse often hid behind its own body or the cage equipment, making it hard to see exactly what it was doing.

The Solution:
The authors of this paper built a "robot detective" using Artificial Intelligence (AI) to do the job automatically. They created a two-step machine learning pipeline:

1. The "Eyes" (DeepLabCut): Think of this as a super-observant camera operator. It was trained to track specific body parts of the mouse (like its nose, ears, and tail) and the cotton swab, even when the mouse was moving or partially hidden. It's like teaching a computer to play "connect the dots" on a moving mouse, frame by frame.
2. The "Brain" (SimBA): Once the "Eyes" track the movement, the "Brain" steps in. This is a classifier that looks at the movement patterns and decides: "Is the mouse sniffing, or is it just walking by?" It learned this by watching thousands of examples of humans manually scoring the videos.

The Experiment:
The team tested their new robot detective on two different groups of mice:

• Group A: Mice modeling a specific type of dementia (ALS/FTD).
• Group B: Healthy mice (the control group).

They ran the test when the mice were young (15 weeks) and again when they were old (67 weeks). They compared the robot's results against the old-school human stopwatch method.

The Results:
The robot detective was amazing!

• Accuracy: The time the robot calculated for sniffing was almost identical to the time humans calculated. It's like having two detectives who agree on the facts 90% of the time.
• The Verdict: When the scientists analyzed the data to see if the "dementia" mice behaved differently than the healthy ones, the robot gave the exact same answer as the humans.
• Efficiency: The robot could process videos much faster than a human could, and it didn't get tired.

Why This Matters:

• Speed: It frees up scientists to do more research instead of staring at screens.
• Consistency: Robots don't have bad days or get distracted by coffee breaks.
• Accessibility: You don't need a supercomputer to run this; a standard high-end desktop computer can handle it.
• Real-world Application: Because the videos are taken from the side (like a home security camera), this method works well for mice living in their normal cages, not just in special lab arenas.

The Catch:
Like any new tool, it's not perfect. Sometimes, if the mouse hides completely or the video quality is poor, the robot might miss a sniff. Also, it needs to be "taught" (trained) with a good set of examples before it can work on a new type of experiment. But overall, this paper proves that AI can be a reliable partner in understanding how mouse brains work, making science faster and more accurate.

In a Nutshell:
The authors replaced the tedious job of human stopwatch-timers with a smart AI system that watches mice sniffing. The AI is just as accurate as the humans but much faster, allowing scientists to study brain diseases more efficiently.

Technical Summary

1. Problem Statement

• Bottleneck in Preclinical Research: Manual annotation of mouse behavioral data is time-consuming, low-throughput, and prone to human error and inter-rater variability. This limits the efficiency of preclinical research, particularly for longitudinal studies.
• Specific Technical Challenge: The olfactory habituation-dishabituation task is critical for assessing sensory and cognitive function in mouse models of neurological diseases (e.g., ALS/FTD). However, this task is typically recorded using a single side-view camera in home-cage environments.
  • Occlusion: Side-view projection causes significant occlusion of body parts (e.g., the nose and snout are often hidden by the body or cage structures).
  • Depth Estimation: Single-camera setups lack depth information, making 3D reconstruction difficult.
  • Visual Constraints: Videos are often recorded in black-and-white (infrared) to avoid disturbing nocturnal animals, reducing feature contrast.
• Gap: Existing machine learning (ML) pipelines often rely on top-down views or multi-camera setups. There is a lack of validated, automated tools capable of accurately quantifying specific behaviors (sniffing) in complex, occluded, side-view home-cage environments.

2. Methodology

The authors developed and validated a two-stage machine learning pipeline combining DeepLabCut (DLC) for pose estimation and SimBA (Simple Behavioural Analysis) for behavioral classification.

A. Data Acquisition & Preprocessing

• Source Data: Used previously published, manually annotated datasets from two mouse models of ALS/FTD (C9orf72GR400/+ and TardbpQ331K/Q331K) and their wildtype (WT) controls.
• Task Setup: Mice were tested in Individually Ventilated Cages (IVC) using a side-view infrared camera. Odors (water, familiar mouse, novel mouse) were presented via cotton swabs through a food hopper.
• Video Preprocessing: Videos were cropped to the region of interest (ROI) near the hopper to reduce computational load and focus on relevant behavior.
• Training Set: 60 videos (balanced by age and genotype) were selected for training; 80 videos were used for the full pipeline (including SimBA training).

B. Stage 1: Pose Estimation (DeepLabCut)

• Tool: DeepLabCut v2.3.9 using ResNet-101 architecture.
• Labels: 10 mouse body parts (snout, eyes, ears, neck, back, tail base, abdomen) plus 4 environmental landmarks (hopper corners, cotton bud tip, floor).
• Training Strategy:
  • 2,069 frames were manually labeled.
  • Iterative retraining (16 cycles) focused on frames where tracking failed.
  • Performance: Achieved a test error of 2.42 pixels (~1.1 mm spatial resolution) with a confidence cutoff of 0.6.

C. Stage 2: Behavioral Classification (SimBA)

• Tool: SimBA v2.0.7 using Random Forest classifiers.
• Feature Extraction:
  • Input: DLC pose data (body part coordinates) and manually defined ROIs (hopper, cotton bud).
  • Features: Angles, velocities, distances, and rolling time-window aggregates.
  • Crucial Step: Frames without the cotton bud (no odor presentation) were filtered out to balance the dataset and improve classification accuracy.
• Classes: Two mutually exclusive classes: "Sniffs" and "Does not sniff."
• Training Parameters: 2,000 estimators, 80/20 train/test split.
• Performance Metrics:
  • "Sniffs": Precision 0.92, Recall 0.86, F1-score 0.89.
  • "Does not sniff": Precision 0.98, Recall 0.99, F1-score 0.98.
• Thresholds: 0.5 confidence threshold and a 200 ms bout duration filter.

D. Validation & Statistical Analysis

• Correlation: Pearson correlation between manual scoring and ML output.
• Statistical Modeling: Linear Mixed-Effects Models (lmer) and Generalized Linear Mixed Models (glmer) in R (using lme4 and lmerTest packages) to compare:
  • Filtering rates (data exclusion based on engagement).
  • Biological outcomes (genotype, age, sex, odor type effects).
  • Technical validity (habituation and dishabituation patterns).

3. Key Contributions

1. Novel Pipeline for Side-View Occlusion: Successfully adapted state-of-the-art pose estimation (DLC) and behavioral classification (SimBA) to a challenging single side-view, occluded, infrared environment.
2. Open-Source Reproducibility: All code (Python/R scripts), genotyping data, and trained models are publicly available on GitHub (sboya23/ML-analysis-olfaction).
3. Hardware Accessibility: Demonstrated that the pipeline can run on a standard desktop workstation (Nvidia RTX4090 GPU), removing the barrier of requiring high-performance computing clusters.
4. Validation in Disease Models: Validated the pipeline across two distinct genetic models (C9orf72 and Tardbp) and two age groups (young and old), proving robustness across biological variables.

4. Results

• High Correlation: The ML pipeline showed strong positive correlations with manual scoring:
  • C9orf72 (15w): $r = 0.90$; (67w): $r = 0.87$.
  • Tardbp (15w): $r = 0.93$; (67w): $r = 0.88$.
  • These correlations were comparable to the correlation between two independent human scorers.
• Filtering Consistency: In the C9orf72 study, there was no significant difference in data filtering rates between manual and ML methods. In the Tardbp study, a minor interaction was found at 67 weeks, but it did not alter the overall biological conclusions.
• Biological Validity:
  • The ML pipeline successfully recapitulated key biological findings: significant effects of genotype, age, and sex on sniffing behavior.
  • Habituation/Dishabituation: Both manual and ML data detected the expected pattern of habituation to repeated odors and dishabituation to novel odors.
  • Discrepancies: Minor post-hoc discrepancies existed (e.g., ML failed to detect habituation to control odor in WT mice at 15 weeks, or missed specific dishabituation events in Tardbp at 67 weeks). However, these did not impact the primary conclusion that genotype did not significantly alter habituation/dishabituation patterns in these specific models.
• Statistical Equivalence: Linear mixed models confirmed that the "source of data" (Manual vs. ML) did not significantly interact with genotype, sex, or odor type, indicating the ML method is a valid substitute for manual scoring.

5. Significance

• Throughput & Efficiency: The pipeline automates a labor-intensive task, allowing for high-throughput analysis of longitudinal behavioral data without sacrificing accuracy.
• Welfare-Compliant Monitoring: By utilizing side-view cameras in home cages, the method supports long-term, group-housed monitoring under high-welfare conditions, which is difficult with traditional top-down arenas.
• Scalability: The approach is accessible to researchers without specialized hardware, democratizing advanced behavioral analysis.
• Future Applications: While currently optimized for sniffing, the framework can be adapted to classify other behaviors (licking, biting) or other sensory tasks in home-cage environments. The authors note that future work could explore segmentation-based tracking to reduce training data requirements.

Conclusion: The study successfully validates a machine learning pipeline that automates the scoring of olfactory habituation-dishabituation tasks in mice. It achieves high accuracy despite the technical challenges of side-view occlusion and infrared imaging, producing biological results indistinguishable from manual annotation.