📄 bioengineering

RNASeek: A Cross-Phyla Generative Foundation Model for Multipurpose RNA Modeling and Reinforcement Learning-Based Design

RNASeek is a 1.6-billion-parameter generative foundation model that leverages reinforcement learning to unify RNA sequence representation, functional prediction, and de novo design, successfully generating experimentally validated ribozymes and 3' UTRs with enhanced performance.

Original authors: Chen, S., Fernandes, N., Li, W. V., Wang, L., Jia, Z., Xiang, J. S.

Published 2026-09-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chen, S., Fernandes, N., Li, W. V., Wang, L., Jia, Z., Xiang, J. S.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Inside every living cell, a vast and intricate conversation takes place, not with words, but with molecules. At the heart of this dialogue is RNA, a versatile molecule that acts as both a messenger and a regulator, carrying instructions from the genetic blueprint and deciding how those instructions are executed. For decades, scientists have understood that the specific order of letters in an RNA strand determines its shape and function, much like the arrangement of letters in a sentence determines its meaning. However, predicting exactly how a new sequence of letters will behave has been a formidable challenge. The rules are complex, involving long-range interactions and subtle structural nuances that are difficult to map by hand. Now, researchers have developed a new tool that learns these rules by reading the biological literature of life itself, allowing them to not only predict how RNA will behave but to design entirely new molecules with specific, desired properties.

The team behind this work, led by researchers at the University of California, Riverside and Columbia University, created a massive computer program they call RNASeek. Think of this program as a student that has read every book in a library containing the genetic instructions of over one hundred different species, from plants and animals to fungi and viruses. Unlike previous tools that were designed to solve only one specific puzzle, RNASeek was built to understand the general language of RNA. It was trained on a dataset comprising roughly 150 billion pieces of genetic information, learning to recognize patterns in how different organisms construct their RNA. The program also learned to read natural language instructions, meaning a scientist can simply tell it what they want, such as "create a stable RNA sequence" or "design a molecule that cuts itself," and the model will attempt to generate a solution.

To test if this digital student had truly learned the language, the researchers first asked it to predict the behavior of ribozymes. These are RNA molecules that act as enzymes, capable of cutting themselves or other molecules. The team provided the model with thousands of known ribozyme sequences and their measured cutting speeds. RNASeek successfully learned to predict which sequences would cut quickly and which would be slow, identifying subtle features that contribute to speed, such as the flexibility of certain loops in the molecule's structure. The model performed as well as, or better than, many specialized tools designed specifically for this task, proving that it had grasped the underlying principles of how RNA structure dictates function.

Encouraged by this success, the researchers pushed the model further, asking it to design new RNA sequences from scratch. They focused on a different type of RNA found in the 3' untranslated region, a section of the molecule that helps determine how long the RNA survives inside a cell. A longer lifespan means the cell produces more of the protein the RNA codes for, which is crucial for therapies like mRNA vaccines. The researchers used a technique called reinforcement learning, a method where the computer program plays a game of trial and error. It generated thousands of candidate sequences, and a separate part of the model acted as a judge, scoring them based on how stable they were predicted to be. The program then adjusted its strategy to generate better candidates, gradually learning to produce sequences that were highly stable.

The results were striking. The sequences designed by RNASeek were not just random combinations of letters; they contained specific patterns, such as an abundance of adenine and uracil, that are known to help stabilize RNA. When the researchers tested these computer-generated designs in a laboratory setting, they found that the new sequences performed exceptionally well. Some of the designed molecules were even more stable than the best sequences found in nature or those created by other artificial intelligence tools. Crucially, when the researchers shuffled the letters of these successful designs, scrambling their order while keeping the same ingredients, the stability vanished. This confirmed that the success came from the specific arrangement of the letters, not just the chemical composition, proving the model had learned the true syntax of the language.

The study also demonstrated that the model could follow complex instructions. Scientists could ask the program to generate a stable RNA sequence that included a specific motif for cloning or excluded a particular pattern that might cause problems. The model successfully adhered to these constraints while still optimizing for stability. This ability to follow natural language commands and produce functional biological parts suggests a new way of engineering life. Instead of relying on slow, expensive cycles of trial and error in the lab, scientists can now use a unified system to predict how RNA will behave and design new molecules to meet precise needs. The work establishes that a single, large-scale model can bridge the gap between understanding biological data and creating new, functional biological tools, opening the door to more efficient design of therapies and research tools.

Technical Summary: RNASeek

Problem Statement
While large language models (LLMs) have revolutionized natural language processing and protein design, a general framework connecting RNA foundation models to functional sequence design remains limited. Existing RNA models are primarily designed for representation learning and downstream prediction rather than instruction-conditioned sequence generation guided by quantitative experimental rewards. Furthermore, effective reinforcement learning (RL) for RNA design requires dense, quantitative reward signals mapping sequence variation to functional outcomes across diverse sequence spaces, a capability often lacking in current regulatory RNA design workflows.

Methodology
The authors present RNASeek, a 1.6-billion-parameter generative foundation model built on a DeepSeek architecture (specifically a Qwen2.5-based decoder-only transformer) and trained on a cross-phyla transcriptomic corpus.

Architecture and Tokenization: RNASeek utilizes a unified token vocabulary comprising:
1. Nucleotide tokens (A, U, C, G).
2. Predicted RNA secondary structure tokens (dot-bracket notation).
3. Genomic markup tokens (e.g., |u5u|, |u3u|, |s|, |t|, |i|) to define functional regions.
4. Natural language tokens (12,000 tokens derived from PubMed/PMC) for task instructions.
  This hybrid strategy allows the model to accept natural-language-style task commands (e.g., `$predict_efficiency`) alongside sequence data, enabling flexible conditional prediction and design without retraining the tokenizer for each task.
Pretraining: The model was pretrained on approximately $1.5 \times 10^{11}$ mRNA tokens from over 100 diverse species (plants, animals, fungi, viruses) and 2 billion natural language tokens. The training context window was 32,768 tokens. Approximately 50% of sequences were augmented with predicted secondary structure (RNAfold) to learn structure-sequence dependencies.
Supervised Fine-Tuning (Prediction): The pretrained backbone was fine-tuned with a regression head to predict:
- Ribozyme self-cleavage activity: Using massively parallel reporter assay (MPRA) data from hammerhead ribozyme libraries (c-di-GMP and randomized Loop I/II variants).
- Viral mRNA stability: Using MPRA data where viral RNA fragments were inserted into 3′ UTRs.
  Target values were transformed using log-plus-one z-scores to stabilize variance.
Reinforcement Learning (Design): To move from prediction to design, the authors employed Group Relative Policy Optimization (GRPO).
- Reward Model: The fine-tuned regression models served as reward functions (predicting ribozyme efficiency or 3′ UTR stability).
- Policy Optimization: The generative backbone was updated to maximize rewards while adhering to user-specified constraints (e.g., IUPAC motif inclusion/exclusion, length constraints) encoded in natural language prompts.
- Composite Reward: The final reward combined the regression model score (40%) with a syntax/structure score (60%) ensuring valid nucleotide composition, correct markers, and adherence to IUPAC constraints.

Key Contributions

Unified Framework: RNASeek establishes a single autoregressive backbone capable of both quantitative functional prediction and controllable de novo sequence generation, bridging the gap between representation learning and generative design.
Cross-Phyla Generalization: By training on a diverse corpus spanning multiple kingdoms and viruses, the model captures species-specific transcript features and intron-exon boundaries in a zero-shot setting.
Instruction-Conditioned Generation: The integration of natural language tokens allows users to specify complex design goals (e.g., "exclude motif X, include motif Y") without architectural changes.
RL-Driven Optimization: The application of GRPO to RNA design demonstrates a practical pathway for optimizing sequences toward desired biological properties (stability, catalytic activity) while satisfying strict compositional constraints.

Results

Zero-Shot Capabilities: Pretrained embeddings showed clear species separation in UMAP projections and achieved zero-shot intron boundary prediction with a perplexity of ~323, representing a ~37-fold improvement over a uniform token selector.
Predictive Performance:
- Ribozymes: RNASeek achieved a validation Pearson correlation of $r = 0.628$ on ribozyme efficiency, outperforming most benchmarked foundation models (including RNA-FM, Nucleotide Transformer, and various BERT-style models) and surpassing a custom LSTM baseline.
- Stability: For viral 3′ UTR stability, RNASeek achieved a Spearman correlation of $r = 0.342$ , performing competitively with top-tier models like UTR-BERT-6mer and RiNALMo-micro.
Generative Design:
- 3′ UTRs: GRPO-guided generation produced sequences with significantly higher predicted stability than random sequences, natural human 3′ UTRs, and sequences generated by ChatGPT-5 or GEMORNA.
- Experimental Validation (Stability): Four RNASeek-generated 3′ UTR candidates were experimentally validated using a dual-luciferase reporter assay. All candidates showed increased reporter expression compared to negative controls. Notably, RNASeek527 exhibited significantly higher activity than all tested GEMORNA sequences and exceeded the performance of the top-performing sequence (K4) from the original training dataset.
- Ribozymes: GRPO-generated ribozymes showed higher predicted activity than shuffled controls and wet-lab training data. Experimental validation confirmed that selected RNASeek-generated ribozymes achieved wild-type levels of self-cleavage activity in cells.
Mechanistic Insights: Analysis of generated sequences revealed that high-efficiency ribozymes favored flexible, low-GC loops with high unpaired probability. For 3′ UTRs, the model learned to deploy AU-rich syntax and specific positional nucleotide biases to enhance stability, distinct from simple compositional shuffling.

Significance
The paper claims that RNASeek provides a general strategy for engineering regulatory RNAs with desired properties by connecting learned RNA function to controllable de novo sequence design. It demonstrates that large language model architectures and policy-optimization methods (GRPO) developed for natural language can be successfully adapted to biological sequence design. The work establishes a unified pretrain–predict–optimize framework that reduces reliance on repeated wet-lab experimentation for sequence screening. While the authors note limitations regarding cell-type specificity and the potential for reward-model bias, they posit that this approach offers a scalable foundation for future tasks including enzyme activity prediction, splicing modeling, and therapeutic RNA design.

More like this