Assessment of Visual Function in Mice Using Light/Dark Box and Multi-Feature Machine Learning

This study introduces a robust, automated machine learning paradigm that integrates multiple behavioral metrics to overcome the limitations of traditional single-metric light/dark box tests, thereby significantly improving the reliability and accuracy of visual function assessment in mice.

The Gist

Imagine you are trying to figure out if a friend can see the world around them. You put them in a room with a bright, sunny side and a cozy, dark side.

The Old Way: The "Guessing Game"
For years, scientists have used a simple test called the "Light/Dark Box" to check if mice can see. The idea is simple: mice naturally hate bright lights and love dark, safe spaces. So, if a mouse can see, it should spend most of its time in the dark. If it's blind, it wanders around randomly, spending equal time in both.

The problem? Scientists were only looking at one single number: How much time did the mouse spend in the dark?

Think of it like trying to judge a soccer player's skill by only counting how many times they touched the ball. It's a clue, but it's not the whole story. A mouse might spend a lot of time in the dark not because it can see, but because it's just shy, anxious, or feeling lazy. Conversely, a blind mouse might wander into the light just because it's curious. Relying on just that one number was like trying to solve a complex puzzle with only one piece—it often led to wrong answers.

The New Way: The "Detective Squad"
In this new study, the researchers (led by Gang Luo) decided to stop guessing and start investigating like a team of detectives. Instead of just asking, "How long was it in the dark?", they asked ten different questions about the mouse's behavior:

• How fast was it running?
• How many times did it cross from light to dark?
• Did it freeze in fear?
• How long did it take to make a decision?

They fed all these clues into a Machine Learning "Brain" (an AI). This AI is like a super-smart coach who watches the mouse's entire performance, not just one stat. It looks at the pattern of behavior.

The Results: A Clear Winner
The results were like night and day (pun intended!):

• The Old Way (Single Metric): The AI was barely better than flipping a coin. It couldn't reliably tell the difference between a seeing mouse and a blind one because the "time in the dark" number was too messy and influenced by too many other things (like anxiety).
• The New Way (Multi-Feature): The AI became a master detective. By combining speed, movement patterns, and hesitation, it could distinguish between sighted and blind mice with high accuracy. It was so good that even if the experiment was set up slightly differently (like changing how long the mouse got to warm up), the AI still got it right.

The "Secret Sauce": Cutting the Clutter
The researchers also found something funny. They realized that some of the clues they were feeding the AI were actually "noise"—like asking a detective, "What color was the suspect's left shoe?" when it didn't matter. By removing the useless clues and keeping only the most important ones (like how fast the mouse moved and how often it crossed the room), the AI got even smarter and faster.

Why Does This Matter?
This is a big deal for medical research.

1. Saving Time and Money: Scientists can now test new eye drugs on mice much faster and with more confidence, without needing to train the mice for weeks or use invasive surgery.
2. Better Medicine: If we can accurately tell if a mouse can see, we can better test if a new drug actually works to restore vision.
3. Less Stress: The test is "training-free," meaning the mice just walk around naturally. No stress, no forced learning, just honest behavior.

In a Nutshell
The old method was like judging a movie by looking at just one frame. The new method is like watching the whole movie, analyzing the acting, the lighting, the sound, and the plot. By using a smart computer to look at the whole picture of how a mouse moves, scientists can finally get a clear, reliable answer on whether a mouse can see, paving the way for better treatments for human eye diseases.

Technical Summary

1. Problem Statement

Traditional methods for assessing visual function in rodents, specifically the Light/Dark Box test, rely heavily on a single behavioral metric: the dwell time (percentage of time spent) in the light or dark compartment.

• Limitations: This single-metric approach suffers from low reliability because mouse behavior is influenced by confounding factors unrelated to vision, such as anxiety levels, individual variability, and exploratory motivation.
• Experimental Variability: Existing studies use inconsistent protocols regarding habituation duration (1–5 mins) and observation windows (3–10 mins), leading to unclear outcomes and poor reproducibility.
• Current Gap: There is a need for a non-invasive, training-free, and robust method that can accurately distinguish between sighted and blind mice across different experimental settings without being skewed by non-visual behavioral noise.

2. Methodology

A. Experimental Setup

• Subjects: 50 mice were tested, divided into two groups:
  • Sighted (n=25): Wild-type (WT) mice and young Rhodopsin knockout (Rho⁻/⁻) mice (4–10 weeks old) with preserved vision.
  • Blind (n=25): Older Rho⁻/⁻ mice (16–45 weeks) with degenerated vision and WT mice subjected to double optic nerve crush (ONC).
• Apparatus: An automated Light/Dark Box system was constructed, featuring:
  • A light compartment (30×40×40 cm) and a dark compartment (20×40×40 cm).
  • White LED lighting (~1500 lux) and infrared (940 nm) illumination for night-vision tracking.
  • An overhead infrared camera recording at 30 fps.
• Localization Algorithm: A custom computer vision pipeline was developed to track mouse position:
  1. Region Localization: Uses fiducial markers to define compartment boundaries.
  2. Background Subtraction: Isolates the mouse from the static background.
  3. Tail Removal: Uses distance transform and dilation to filter out tail shadows/curvature, ensuring the centroid represents the body mass.

B. Feature Extraction

Instead of relying on dwell time alone, the authors extracted 10 behavioral metrics from 20-minute video recordings:

1. Entries to dark compartment.
2. Average speed in dark/light compartments.
3. Total distance traveled in dark/light compartments.
4. Standard deviation of latency to enter dark/light compartments.
5. Immobility time in dark/light compartments.
6. Dark side percentage (the traditional single metric).

C. Machine Learning Paradigm

• Data Processing: Videos were segmented using a sliding window approach (1–10 minute windows, sliding in 0.5-minute intervals) to maximize sample size and capture temporal dynamics.
• Models Tested: Five classifiers were trained and compared:
  • Support Vector Machine (SVM)
  • Logistic Regression (LR)
  • Multilayer Perceptron (MLP)
  • Random Forest (RF)
  • Extreme Gradient Boosting (XGBoost)
• Validation: A 5-fold group-wise cross-validation was used, ensuring that data from the same mouse never appeared in both training and testing sets simultaneously to prevent data leakage.

D. Optimization

• Feature Importance Analysis: Using the RF model, the authors ranked features by importance.
• Ablation Studies: Features were systematically removed to test their impact. Surprisingly, removing certain features (specifically F8 and F9) improved model performance, suggesting redundancy in the full feature set.
• Optimized Subset: An optimized feature subset (excluding F8 and F9) was constructed to enhance generalization.

3. Key Results

A. Superiority of Multi-Feature ML over Single Metric

• Single Metric Failure: The traditional "dark side percentage" showed no statistically significant difference between sighted and blind mice across various window lengths and habituation times (p > 0.05). The distributions overlapped significantly, with blind mice showing higher individual variability.
• ML Success: The multi-feature ML approach (specifically SVM) achieved significant separation.
  • Discrimination Threshold: Significant differences emerged with observation windows as short as 2 minutes.
  • Performance: At a 7-minute window, the SVM model achieved an AUC of 0.836 and Accuracy of 0.784.
  • Comparison: The single-metric model peaked at an AUC of only 0.552 (near random chance).

B. Model Comparison

• SVM performed best overall (Avg AUC: 0.793).
• Logistic Regression followed closely (Avg AUC: 0.791).
• MLP, RF, and XGBoost showed intermediate to lower performance, likely due to the relatively small dataset size favoring simpler or kernel-based models over complex ensembles.

C. Feature Optimization

• Feature Importance: "Entries to dark," "Dark average speed," and "Light average speed" were the most critical features.
• Optimization Impact: Removing the least informative/redundant features (F8 and F9) actually improved performance.
  • For SVM, the optimized subset increased the peak AUC from 0.836 to 0.866 and accuracy from 0.784 to 0.802.
  • This improvement was consistent across all five models tested.

D. Robustness

The multi-feature method demonstrated stability across different observation window lengths (2–10 mins) and was insensitive to the duration of the habituation period, unlike the single-metric approach.

4. Key Contributions

1. Paradigm Shift: Demonstrated that the traditional single-metric (dwell time) approach is unreliable for visual assessment due to confounding non-visual factors, and proposed a multi-feature machine learning framework as a superior alternative.
2. Automated Pipeline: Developed a fully automated system combining custom hardware (Light/Dark Box), computer vision (tail-removed localization), and ML for training-free visual phenotyping.
3. Feature Engineering Insights: Identified that specific behavioral features (speed, entries) are more predictive than dwell time and showed that feature subset optimization (removing redundant features) can further boost model accuracy.
4. Protocol Standardization: Provided evidence that reliable discrimination can be achieved with observation windows as short as 2 minutes, offering a more efficient protocol for preclinical studies.

5. Significance and Future Impact

• Preclinical Efficiency: This method offers a rapid, non-invasive, and training-free tool for assessing visual function, which is crucial for drug development and genetic studies of eye diseases.
• Scalability: The automated nature of the system allows for high-throughput screening of large mouse cohorts, reducing labor costs and human error.
• Generalizability: While currently tested on specific strains and ages, the framework is designed to be adaptable to different genetic backgrounds, sexes, and pathologies once standardized.
• Research Quality: By mitigating the influence of anxiety and exploratory behavior through multi-feature analysis, this approach significantly reduces false positives/negatives in visual function studies, leading to more reproducible scientific results.