Millisecond-scale detection of mouse ultrasonic vocalizations enables closed-loop experiments

Mice are far more vocal than their squeaks suggest. When they interact socially, whether with a mother, a potential mate, or a stranger, they emit a rapid stream of sounds that are far too high-pitched for human ears to hear. These ultrasonic calls, which can reach frequencies well beyond what we can perceive, serve as a rich language for their social world. For scientists studying behavior, these sounds offer a direct window into the mouse's mind, revealing moments of courtship, distress, or curiosity. However, for decades, listening to this conversation has been a slow, backward-looking process. Researchers would record hours of audio, then spend days or weeks using computers to sift through the silence, looking for the brief, fleeting calls hidden within the noise. By the time the calls were identified, the moment had long passed, making it impossible to react to the mouse's communication as it happened.

This limitation has kept a powerful tool out of reach for real-time science. If a researcher could hear the call the instant it was made, they could instantly respond with a light, a sound, or a stimulus, turning the mouse's own voice into a trigger for the experiment. The challenge has been technical: these calls are incredibly short, often lasting only a few milliseconds, and they occur in rapid bursts. Existing computer programs were too slow to catch them as they happened, usually requiring a long stretch of audio to be recorded before they could even begin to analyze it. A new tool, developed by researchers at the University of Bonn, changes this dynamic entirely. They have created a system that listens to the raw sound waves of a mouse's voice and decides, within milliseconds, whether a call is occurring, fast enough to trigger an immediate response while the mouse is still speaking.

The researchers, led by Martin K. Schwarz, built a system they call DeepFisFis. Instead of trying to translate the sound into a visual picture first, a step that takes time and computing power, their system looks directly at the shape of the sound wave itself. Imagine the sound wave as a jagged line moving across a screen. The system chops this line into tiny, five-millisecond slices. For each slice, it asks a simple question: does this tiny piece of sound look like part of a mouse call? Because the slices are so small, the computer can answer this question almost instantly. In fact, the system processes each slice in about 2.5 milliseconds, which is faster than the time it takes for the microphone to even record that slice of sound. This speed means the computer is never behind the audio stream; it is always listening in real time, ready to act the moment a call begins.

To prove this worked, the team tested their system against established methods that had been used for years. They fed the system recordings of mice interacting and asked it to find the calls. The new system found nearly every single call that human experts had marked, matching or even beating the accuracy of the older, slower tools. But accuracy was only half the story. The true test was speed. When the researchers measured how long it took to analyze a chunk of audio, their new system was four to five times faster than the previous best tools. More importantly, because it works on such tiny slices, it does not need to wait for a whole call to finish before making a decision. It can identify that a call has started just a few milliseconds after the sound begins.

This speed unlocked a new way to run experiments. The team set up a closed-loop system, a setup where the computer listens to the mouse and immediately triggers a response. In their test, they played back recorded mouse calls to the system. As soon as the system detected a call, it triggered a tone to play through a speaker. The entire process, from the sound arriving at the microphone to the tone playing, took about 128 milliseconds. While this delay is mostly due to the hardware and not the software, the critical finding was that the software itself added almost no extra time. The system was fast enough to keep up with the flow of sound, proving that the computer could act as a real-time partner in the experiment rather than just a passive recorder.

Beyond triggering responses, this speed offers a practical solution to a massive problem in modern science: data storage. Recording high-quality audio at the speeds needed to catch these calls generates enormous amounts of data. If a researcher records for an hour, they might end up with gigabytes of mostly silence, with only a tiny fraction containing actual calls. The new system allows researchers to record only what matters. By using the detector to watch the audio stream live, the computer can decide to save only the moments when a call is detected, along with a small buffer of sound before and after. In their tests, this method reduced the amount of data stored by more than 75 percent while still keeping almost all of the actual vocalizations. This means scientists can run longer experiments or record more complex behaviors without being overwhelmed by empty data.

The system also showed remarkable flexibility. The researchers trained the computer on recordings made at one specific speed and then tested it on recordings made at a different, faster speed. Without needing to collect new data or retrain the system from scratch, they were able to adapt the tool to the new conditions and maintain high accuracy. This suggests that the system learns the fundamental shape of the mouse voice rather than just memorizing specific recordings, making it robust enough to work in different labs with different equipment.

The work represents a shift in how scientists interact with animal behavior. For years, the study of mouse communication has been a retrospective activity, analyzing what happened after the fact. This new approach turns the mouse's voice into a live signal, available the instant it is produced. It does not replace the detailed analysis that happens later, but it opens the door to experiments where the animal's own voice drives the course of the study. By listening and reacting in the blink of an eye, researchers can now ask questions about the neural circuits behind communication in a way that was previously impossible, moving from observing a conversation to participating in it.