Leveraging unlabelled data for generalizable neural population decoding
The paper introduces MOJO, a training framework that combines self-supervised masked autoencoding with supervised learning for spike-tokenizing neural models, demonstrating that leveraging unlabelled data significantly improves decoding performance, generalizability across species and modalities, and interpretability, particularly in scenarios with limited labeled data.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where we can read the mind's electrical whispers to help paralyzed people move their limbs or to understand how we make decisions. This is the realm of brain-computer interfaces (BCIs), a field where scientists try to translate the chaotic, flickering signals of neurons into clear commands for computers. For a long time, the best way to do this was like teaching a dog tricks: you show it a signal, tell it exactly what behavior happened (like "the arm moved left"), and it learns to match the two. But this method has a huge flaw: it requires a teacher to be present for every single lesson. In the real world, we have mountains of recorded brain data where the "teacher" is missing—we have the neural chatter, but no label saying what the animal was doing at that exact moment. Until now, most advanced brain-reading models could only learn from the "labeled" data, leaving vast libraries of "unlabeled" brain recordings gathering dust, unable to help improve the decoder.
Enter a new approach that treats the brain like a language. Just as large language models (like the ones that write stories or answer questions) learn by reading billions of books without needing a human to explain every sentence, this new research suggests we can teach brain-decoding models to understand the "grammar" of neural activity on their own. The paper introduces a method called MOJO (Masked autOencoder-based JOint training). Think of it as a student who studies both a textbook with answers (labeled data) and a massive library of mystery novels without answer keys (unlabeled data). By trying to fill in the missing pages of the mystery novels, the student learns the underlying structure of the story so well that when they finally take the test with the textbook, they ace it. The researchers found that by combining these two learning styles, their model became a much better decoder, especially when there were very few labeled examples to start with, and it even worked on different species and types of brain signals.
The Problem: The "Label" Bottleneck
Imagine you are trying to learn a new language. You have a dictionary that tells you that the word "apple" means a red fruit. If you only have this dictionary (labeled data), you can learn the word "apple." But if you want to become fluent, you need to read thousands of books, articles, and stories (unlabeled data) to understand how words fit together, how sentences flow, and what context changes a word's meaning.
For years, the most advanced brain-decoding models, known as "spike-tokenizing" models (like the POYO family), were like students who only had the dictionary. They were incredibly good at matching specific brain signals to specific movements, but they could only learn from data where scientists had carefully recorded exactly what the animal was doing at every moment. This is a huge limitation because collecting labeled data is expensive, time-consuming, and often impossible for certain experiments. Meanwhile, neuroscientists have collected terabytes of "unlabeled" brain data—recordings of neurons firing without knowing what the animal was thinking or doing at that exact second. These models were effectively ignoring this treasure trove.
The Solution: MOJO's "Fill-in-the-Blanks" Trick
The authors of this paper, Ximeng Mao, Nanda H Krishna, and their team, created MOJO to solve this. They wanted to teach these models to learn from the unlabeled data, just like a language model learns from a book.
To do this, they used a clever trick called masked autoencoding. Imagine you have a sentence: "The cat sat on the mat." If you cover up the word "sat," can you guess what it was? If you've read enough stories about cats, you can probably guess "sat" or "stood" or "jumped" based on the context. MOJO does this with brain signals. It takes a sequence of neural spikes (the "sentence"), randomly hides (masks) some of the signals, and asks the model to predict what those missing signals were.
Here is the magic: The model has to learn the hidden rules of how neurons talk to each other to make these guesses. It learns the "grammar" of the brain. But the model doesn't just guess; it also keeps its original job of predicting behavior (like arm movement) using the labeled data. It does both at the same time. This is called joint training. The model learns the deep structure of the brain from the unlabeled data, which helps it understand the labeled data much better.
The Results: Smarter, Faster, and More Flexible
The team tested MOJO on three very different types of brain data:
- Monkey Reaching: Monkeys reaching for targets on a screen.
- Mouse Vision: Mice looking at different visual patterns.
- Human Speech: Humans speaking syllables, recorded from electrodes on the brain surface (ECoG).
In every single test, MOJO outperformed models that only used labeled data. But the most exciting part was what happened when they had very little labeled data.
The "Few-Shot" Superpower
Usually, if you have very few examples to teach a model (a "few-shot" scenario), it struggles. But MOJO was a champion here. In one experiment with monkeys, the team tried to teach the model to decode movements using only two labeled trials (calibration attempts). A standard model failed miserably. MOJO, however, used the unlabeled data to understand the general "feel" of the brain's activity. When they gave it just two labeled trials, it achieved over 60% of the performance of a model trained on a full dataset. With just four labeled trials, it hit 75%. It was like a student who, after reading a thousand mystery novels, could solve a new mystery with only two clues.
Cross-Species and Cross-Modality
The researchers also tested if MOJO could learn from one species and apply it to another. They pre-trained the model on monkey reaching data and then fine-tuned it on mouse vision data. The model transferred its knowledge successfully, showing that the "grammar" of the brain it learned was universal enough to help it understand a different animal's brain.
Even more surprisingly, they tried MOJO on human speech data (ECoG), which is a continuous signal, not the discrete "spikes" of neurons. Even though MOJO was designed for spikes, it adapted and performed just as well as models specifically built for continuous signals. This suggests that the core idea of learning from unlabeled data is a powerful tool that works across different types of brain recordings.
The "Hidden" Insights
One of the coolest side effects of MOJO was that the model learned to understand the brain's structure without being explicitly told to. The researchers asked the model to guess which part of the brain a neuron came from (e.g., the visual cortex vs. the hippocampus) just by looking at its internal "embedding" (its digital fingerprint). The model got it right 94% of the time for some tasks, even though it was never trained to do this classification. It also learned to predict the firing rates of neurons with high accuracy. This means the model wasn't just memorizing answers; it was actually learning a rich, interpretable map of how the brain works.
Why This Matters
The paper suggests that by combining supervised learning (learning with answers) and self-supervised learning (learning from patterns in the data), we can build much more flexible and powerful brain-computer interfaces. We don't need to throw away the mountains of unlabeled data we already have; we can use them to teach our models the fundamental rules of the brain.
This approach makes it easier to train models for new tasks or new patients with very little calibration time. It also opens the door to "neuro-foundation models"—massive, general-purpose brain models that can be adapted to many different jobs, from helping paralyzed people move to understanding how we make decisions. The authors note that while this is a significant step forward, the models still need a lot of data to learn the complex dynamics of the brain, and there are still challenges in how to handle different types of signals perfectly. But the path is clear: the future of decoding the brain lies in teaching these models to read the "unlabeled" stories of our minds.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.