🤖 AI

bioMoR: Biology-Guided Mixture-of-Recursions for Effective Genomic Learning

bioMoR is the first framework to integrate structured biological knowledge into a Mixture-of-Recursions architecture for genomic learning, achieving superior performance and efficiency over existing baselines by using graph-based mechanisms to refine embeddings, guide attention, and dynamically allocate computational depth to specific genes or pathways.

Original authors: Koushik Howlader, Tirtho Roy, Md Tauhidul Islam, Wei Le

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Koushik Howlader, Tirtho Roy, Md Tauhidul Islam, Wei Le

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a massive crime scene, but instead of a few clues, you have a room filled with thousands of scattered notes, photos, and fingerprints. In the world of biology, this "crime scene" is a single cell from your body, and the "notes" are the thousands of genes inside it. Scientists have recently started using a type of computer brain called a Transformer to read these notes. Think of a Transformer like a super-smart librarian who reads every single book in a library to understand a story. It's great at finding connections, but it's also incredibly slow and expensive because it treats every single book with the same amount of attention, even if most of them are just blank pages or irrelevant advertisements.

The big question is: How do we make this librarian smarter and faster? We know that in any cell, only a tiny handful of genes are actually doing the heavy lifting to tell us if a person is healthy or sick. The rest are just background noise. If the computer could learn to ignore the boring books and spend extra time reading the important ones, it would be much more efficient. This is where a new idea called Mixture-of-Recursions (MoR) comes in. Imagine MoR as a system where the librarian can choose to read a book once, twice, or even ten times, depending on how interesting it seems. But here's the catch: without any outside help, the librarian doesn't actually know which books are important; it just guesses based on the text alone.

This is exactly what the paper bioMoR tackles. The researchers, Koushik Howlader and his team, asked: "What if we gave the librarian a map of the library?" They built a new system that doesn't just guess which genes are important; it uses a pre-existing map of how genes actually work together in the real world. They found that by feeding this biological "map" directly into the computer's decision-making process, the model becomes a much better detective. It not only gets better at predicting diseases but also uses significantly less computer power, proving that knowing the rules of biology helps the computer learn faster and smarter.

The Problem: The Overworked Librarian

In the past, scientists built computer models to analyze genes, but they treated every gene the same. It's like a teacher grading a 1,000-page essay by spending exactly the same amount of time reading every single word, even the ones that say "the sky is blue" or "and then." In biology, we know that only a few specific genes (called "marker genes") or groups of genes (called "pathways") are the real stars of the show. They are the ones that decide if a cell is a healthy lung cell or a cancerous one.

The standard computer models, however, waste a huge amount of energy reading the boring parts. They also miss the fact that genes don't work in isolation; they work in teams. If Gene A is talking to Gene B, the computer should know that they are connected. But the old models didn't have this "social network" information built-in. They were biology-agnostic, meaning they were blind to the actual relationships between the genes.

The Solution: bioMoR, the Guided Detective

The authors introduced bioMoR (Biology-Guided Mixture-of-Recursions). To understand how it works, let's go back to our librarian analogy.

The Map (Biological Knowledge): Before the librarian even starts reading, bioMoR gives them a map. This map shows which genes are friends, which ones work in the same team (pathways), and which ones talk to each other. This map is based on real scientific data about how genes interact.
Smoothing the Notes (Embedding Smoothing): When the librarian picks up a note (a gene), they don't just read it in isolation. They look at the map and say, "Oh, this gene is friends with these three others. Let me blend their ideas together to understand this note better." This helps clean up the noise and makes the gene's "voice" clearer.
The Smart Router (Adaptive Depth): This is the magic part. As the librarian reads, they have to decide: "Do I need to read this note again? Do I need to read it three more times?" In the old models, this decision was a random guess. In bioMoR, the librarian looks at the map again. If a gene is part of a critical team (like a pathway known to fight cancer), the router says, "This one is important! Let's give it extra attention and read it deeper." If a gene is just background noise, the router says, "We've read this enough; let's move on."

This process happens in three specific places in the computer's brain:

At the start: To clean up the gene's initial description.
During reading: To make sure related genes pay attention to each other.
At the decision point: To decide how many times to re-read a gene.

The Results: Smarter, Faster, and Cheaper

The team tested bioMoR on eight different datasets, ranging from single-cell data (looking at individual cells) to complex cancer data from thousands of patients. They compared their new system against the best existing models.

The results were impressive. The bioMoR model improved the average accuracy of predictions by 8.2 percentage points in one key measure (macro-F1) and 7.1 percentage points in another (balanced accuracy) compared to the strongest non-biological model. But the real win was efficiency.

Fewer Parameters: The bioMoR model used 75% fewer parameters (the internal "knobs" the computer adjusts to learn) than a standard deep model.
Less Computing Power: It used up to 58% fewer FLOPs (a measure of the math operations the computer has to perform) than a standard Transformer that doesn't skip steps.

This means bioMoR isn't just more accurate; it's also much cheaper to run. It achieves this by not wasting time on the genes that don't matter.

The "Why" Behind the "What"

One of the most exciting parts of the study is that the computer didn't just get lucky; it actually learned the right things. The researchers looked at which genes and pathways the model decided to "read deeper." They found that the model consistently chose pathways known to be involved in cancer, such as Wnt signaling and PI3K-Akt signaling, which are famous for helping cancer cells spread and invade other tissues.

This suggests that the model isn't just memorizing patterns; it's using the biological map to figure out which parts of the cell are actually driving the disease. It's like a detective who, instead of guessing, uses a police database to instantly know which suspects are most likely to be guilty.

What the Paper Says (and Doesn't Say)

The authors are very clear about what they have proven. They showed that adding biological knowledge to this specific type of computer architecture (MoR) works better than ignoring it. They demonstrated this across many different types of data, from single cells to whole tumors.

However, they don't claim to have solved all of biology. They didn't say this model can cure cancer or replace doctors. They also didn't claim that this is the only way to do it, but rather that it is a highly effective new way. The paper explicitly argues against the idea that we should treat every gene the same or that we should rely on computer models that don't know anything about how genes interact. They showed that ignoring biology leads to wasted effort and lower accuracy.

In short, bioMoR suggests that when we teach computers to understand life, we shouldn't just give them a stack of papers and say "figure it out." We should give them the textbook, the map, and the rules of the game. When we do that, they become much better students, learning faster and making fewer mistakes.

Technical Summary: bioMoR – Biology-Guided Mixture-of-Recursions for Effective Genomic Learning

Problem Statement

High-dimensional omics data (e.g., single-cell gene expression, multi-omics cancer cohorts) present a computational challenge for Transformer models. While Transformers can effectively model long-range molecular dependencies, standard architectures apply uniform deep computation to every input token (gene or pathway). This is inefficient because genomic signals are typically sparse; specific phenotypes (e.g., cell types, tumor states) are often driven by a small subset of marker genes or pathways, while the majority of features are weakly informative. Existing efficient attention mechanisms (e.g., Linformer, Performer) reduce sequence costs but do not adapt computation based on biological relevance. Conversely, existing biology-aware models utilize known structures for feature engineering or post-hoc interpretation but do not use this knowledge to dynamically select which tokens require deeper computation.

Methodology: bioMoR

The authors propose bioMoR, a framework that integrates structured biological knowledge into a Mixture-of-Recursions (MoR) backbone. MoR improves efficiency by routing tokens to an adaptive number of recursive steps using a shared Transformer block, rather than stacking independent layers. bioMoR distinguishes itself by injecting biological knowledge at three specific sites to guide token selection and recursion depth:

Genomic Tokenization:
- Single-cell data: Thousands of genes are compressed into $M \ll N$ "marker tokens." Each token is formed by a learnable query that softly selects and sums relevant gene embeddings, collapsing to a specific top gene at test time.
- Multi-omics data: Genes are grouped into Reactome pathways. Tokens are created by pooling member gene features (mean pooling for expression/CNV, sum pooling for mutations).
Biological Knowledge Construction:
- For single-cell data, a gene co-expression matrix is derived from partial correlations of the inverse covariance of expression data.
- For multi-omics, curated Reactome pathway relationships are used.
- These are normalized into a fixed adjacency matrix $A$ used across all three integration sites.
Three Integration Sites:
- Site 1: Embedding Smoothing: Before recursion, token embeddings $T$ are smoothed using the biological graph: $eT = (1-\lambda)T + \lambda AT$ . This integrates information from biologically related neighbors into each token's initial representation.
- Site 2: Attention Bias: Within the shared recursive Transformer block, self-attention is biased toward biologically related tokens: $Attn(Q, K, V) = \text{softmax}(\frac{QK^T}{\sqrt{d}} + \lambda_{attn}\mathbb{1}[A > 0])V$ . This encourages information exchange between related tokens during refinement.
- Site 3: Biology-Aware Routing: The router determines whether a token proceeds to the next recursion step. The routing score combines the standard MoR score with a correction term derived from the token's biological neighborhood ($AH(t)$ vs. $H(t) - AH(t)$). This correction is learned; it starts at zero and activates only if biological information reduces task loss.
Training and Efficiency:
- The model minimizes a class-weighted cross-entropy loss plus auxiliary losses for marker sufficiency, diversity, compression, and router stability.
- Computational cost is measured in FLOPs, calculated based on the mean number of active tokens at each recursion step.

Key Contributions

First Application of MoR to Genomics: bioMoR is the first framework to apply Mixture-of-Recursions to gene-level and pathway-level learning.
Structured Biological Guidance: The paper introduces a novel method to integrate biological knowledge (gene co-expression, pathway relations) not just as static priors, but as dynamic control signals for adaptive computation (embedding refinement, attention bias, and routing).
Efficiency and Performance: The framework achieves significant gains in predictive performance while drastically reducing computational cost and parameter count compared to non-recursive Transformers and biology-agnostic MoR baselines.

Experimental Results

The authors evaluated bioMoR across eight diverse benchmarks, including single-cell cell-type classification (e.g., Segerstolpe, Lung, T-cell) and multi-omics cancer tasks (e.g., TCGA cohorts for survival prediction and classification).

Predictive Performance: Compared to the strongest biology-agnostic MoR baseline, bioMoR improved average macro-F1 by 8.2 percentage points and balanced accuracy by 7.1 percentage points. It also outperformed state-of-the-art external baselines (e.g., scBiGNN, Geneformer, Pathformer) on representative datasets.
Efficiency: bioMoR achieved these gains while using 75% fewer parameters than a standard Transformer and up to 58% fewer FLOPs than a non-recursive Transformer.
Ablation Studies: Integrating knowledge at all three sites (embedding, attention, router) yielded the best performance. Embedding injection alone provided the most significant single-site improvement.
Interpretability: The model successfully prioritized biologically relevant pathways (e.g., Wnt signaling, EMT, PI3K-Akt) for deeper computation, aligning with known literature on cancer metastasis. Frozen embedding visualizations showed cleaner cell-type separation and higher linear probe accuracy compared to baselines.

Significance and Claims

The paper claims that bioMoR addresses a central mismatch in omics modeling: the assumption that all features require equal computational depth. By demonstrating that biological structure can serve as a control signal for adaptive computation, the authors argue that genomic models should not only learn representations from data but also leverage known molecular structures to decide where to spend computational resources.

The work establishes that:

Selective computation based on biological semantics improves both accuracy and efficiency.
Biological knowledge is most effective when used to refine representations, guide attention, and dynamically route tokens, rather than just for static feature engineering.
The proposed framework is generally applicable across single-cell and multi-omics domains without requiring dataset-specific hyperparameter tuning.

The authors conclude that bioMoR represents a step toward more efficient and interpretable foundation models for genomics, where the "mixture-of-recursions" is guided by the underlying biology of the data.