BioMod: Biological Module Tokenization for Single-Cell Foundation Modeling

Cells are the fundamental units of life, each carrying a unique instruction manual written in a language of genes. When scientists want to understand how a body works, or why a disease takes hold, they often look at these instructions in individual cells. This field, known as single-cell biology, allows researchers to see the distinct identity of every cell in a tissue, from a beating heart muscle to a fighting immune cell. However, the sheer volume of data is overwhelming. A single cell contains thousands of active genes, and modern experiments can profile millions of cells at once. To make sense of this ocean of information, researchers have turned to artificial intelligence, specifically "foundation models." These are powerful computer programs trained on massive datasets to learn the underlying rules of biology, much like a language model learns the rules of grammar by reading vast libraries of text. The goal is to create a system that can recognize a cell's type, predict how it will react to a drug, or understand how it changes when a gene is altered, all without needing to be retrained for every new question.

The challenge has been how to feed this information into the computer. Most existing models treat every single gene in a cell as a separate word in a sentence. If a cell has 20,000 active genes, the computer must process a sequence of 20,000 items. This is computationally expensive and forces the AI to figure out complex relationships between genes from scratch, often struggling with the fact that many genes are silent or barely active in any given cell. A team of researchers at the Shenzhen Loop Area Institute and the Chinese University of Hong Kong, Shenzhen, proposed a different approach. They asked whether cells could be understood more efficiently by grouping genes into functional teams before the computer even begins its analysis. In nature, genes rarely work alone; they operate in coordinated groups called pathways or modules to perform specific tasks, such as fighting infection or repairing DNA. The researchers hypothesized that if they could represent these biological teams as single units, the computer could learn faster, use less power, and perhaps understand the biology more deeply.

To test this idea, the team developed a new model called BioMod. Instead of feeding the computer a list of thousands of individual genes, they first organized the genes into 660 overlapping biological modules. These modules were built from a mix of known biological pathways found in scientific databases and new gene groups discovered by analyzing patterns in single-cell data. A single gene could belong to multiple modules, reflecting the reality that a gene might be involved in several different biological processes at once. For every cell, the researchers created a compact representation consisting of just 660 tokens, one for each module, rather than thousands of tokens for individual genes. This process is similar to how a human might summarize a long story not by listing every word, but by identifying the main themes and characters. The model was then trained on a massive dataset of approximately 73 million human cells, covering 25 different organs and tissues, including the brain, blood, and lungs. During training, the computer was shown most of these module summaries and had to guess the missing parts, forcing it to learn the deep connections between different biological programs.

The results showed that this method of grouping genes worked remarkably well. BioMod, which contains about 50 million parameters—a measure of its complexity—was able to perform as well as, and often better than, much larger models that use billions of parameters and process individual genes. In tests involving the identification of cell types, the model correctly classified cells across diverse tissues with high accuracy, often outperforming previous state-of-the-art systems. It also excelled at predicting how cells would respond to genetic changes, such as when a specific gene is turned off, and how cancer cells would react to different drugs. One of the most significant findings was the model's efficiency. Because it processes a sequence of only 660 items instead of thousands, it runs significantly faster on computer hardware. On a standard high-performance graphics card, BioMod could process hundreds of cells per second, while other models struggled to handle even a few dozen without running out of memory. This speed makes it possible to analyze massive datasets on a single machine, a task that previously required expensive, large-scale computing clusters.

The researchers also wanted to know if the specific way they grouped the genes actually mattered, or if the success was just due to the model's architecture. To find out, they created a control version of the model where the genes were randomly shuffled into the modules, destroying the natural biological relationships while keeping the number of modules and the model's structure exactly the same. When they tested this shuffled version, its performance dropped significantly. It became worse at identifying cell types, integrating data from different experiments, and predicting drug responses. This confirmed that the biological organization of the genes was not just a convenient shortcut; the specific, meaningful connections between genes in the modules were essential for the model's success. The analysis further revealed that the model had learned to pay attention to the right things. When the researchers looked at which modules were most important for identifying a specific cell type, they found that the model relied heavily on modules containing known marker genes for that cell. If they hid these key modules, the model's ability to identify the cell collapsed, proving that it had learned the correct biological logic rather than just memorizing patterns.

This work suggests that the future of biological AI may not lie in building ever-larger models that try to process every single detail of a cell, but in designing systems that respect the natural organization of biology. By translating the complex language of genes into the functional language of biological modules, BioMod demonstrates that a smaller, more efficient model can achieve superior results. It offers a new way to compress the vast information of the cell into a form that computers can handle with ease, opening the door to faster and more accurate analyses of human health and disease. The study does not claim to have solved all problems in single-cell biology, but it provides strong evidence that incorporating biological knowledge directly into the structure of AI models can lead to more powerful and efficient tools for understanding life at its most fundamental level.