🧬 biology

Reconstructing sequence-grammar trajectories enables interpretable and tunable cis-regulatory element design

The paper presents GO-CRE, an interpretable deep learning framework that combines a hybrid Transformer-Mamba2 model with reinforcement learning and sequence-grammar trajectory reconstruction to design and experimentally validate diverse, cell-type-specific cis-regulatory elements with enhanced activity and specificity.

Original authors: Mingqian Ma, Wanjuan Bu, Guoqing Liu, Yuxuan Liu, Sizhen Liu, Zhen Zhao, Shijie Yao, Qingru Hua, Yujie Zhang, Cuiting Zhong, Haitao Huang, Pan Deng, Peiran Jin, Qijin Yin, Chuan Cao, Haiguang Liu, Mo

Published 2026-09-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mingqian Ma, Wanjuan Bu, Guoqing Liu, Yuxuan Liu, Sizhen Liu, Zhen Zhao, Shijie Yao, Qingru Hua, Yujie Zhang, Cuiting Zhong, Haitao Huang, Pan Deng, Peiran Jin, Qijin Yin, Chuan Cao, Haiguang Liu, Mo Xu, Yuan He, Tao Qin, Zeyu Chen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Inside every cell, a vast library of instructions dictates how life functions, but the most critical parts of this library are not the genes themselves. Instead, the true conductors of cellular life are short, switch-like segments of DNA known as cis-regulatory elements. These segments act as dimmer switches and on-off buttons, telling specific genes when to turn on, how loudly to sing, and in which types of cells to operate. Without them, a liver cell could not know to filter toxins, and a blood cell could not know to carry oxygen. For decades, scientists have been able to read these switches, but designing new ones from scratch to control cells for therapies or research has remained a profound challenge. It is like trying to write a new language where the grammar is invisible and the rules change depending on the neighborhood the word lives in.

A team of researchers has now developed a new way to design these genetic switches, turning a process that was once a black box into a transparent, steerable journey. By building a system that watches how DNA sequences evolve as they are being designed, they can see exactly how the "grammar" of life emerges. This approach allowed them to create synthetic switches that work with high precision in specific human cell types, offering a clearer path toward engineering the genetic circuits that control our health.

The researchers, led by Mingqian Ma and colleagues, created a framework they call GO-CRE, which stands for Guided Optimization of Cis-Regulatory Elements. To understand how it works, imagine a computer program trying to write a sentence that will make a specific cell type react in a desired way. In the past, scientists would let an artificial intelligence generate thousands of DNA sequences, test them, and hope the best ones worked. This was often a trial-and-error process where the computer might find a solution that worked by accident, using simple, repetitive patterns that fooled the test but didn't truly understand the biology. The new system changes this by treating the design process not as a race to a finish line, but as a map of a journey.

The core of their method relies on a powerful computer model called HybriDNA. This model was trained on a massive collection of DNA from hundreds of species, learning the subtle patterns that govern how genetic information is stored and read. The researchers then used this model to generate new DNA sequences, but they did not stop there. They added a layer of observation that tracks every small change the computer makes as it tries to improve a sequence. They broke these changes down into tiny building blocks, looking at how the frequency of specific letter combinations and the presence of known biological tags shifted over time. This allowed them to reconstruct the "trajectory," or the path, the sequence took as it evolved from a random string of DNA into a functional switch.

When the team watched these paths unfold in a liver cell type known as HepG2, they discovered something unexpected. The sequences were getting stuck in a trap. The computer was finding a shortcut by creating long, repetitive stretches of a single letter, specifically a string of Gs, which the model mistakenly thought was a good solution. This was a low-quality answer that satisfied the computer's scoring system but failed to create a real, working biological switch. Because the researchers could see this happening in real-time, they were able to intervene. They adjusted the rules of the game, adding a penalty for these repetitive strings. This simple correction redirected the computer's path, steering it away from the trap and toward a more complex, biologically meaningful solution.

This ability to diagnose and fix the design process in the middle of the journey revealed that successful creation of these switches happens in three distinct phases. First, the computer explores widely, trying many different combinations. Next, it commits to a specific direction, locking onto a set of biological features that belong to the target cell type. Finally, it fine-tunes the result, polishing the sequence while keeping its core identity. In the liver cells, the system learned to assemble a specific team of regulatory tags in a precise order, creating a compact and efficient switch. In blood cells, known as K562, it assembled a different team of tags suited for that environment.

To prove that these computer-generated switches actually worked, the researchers tested them in the lab. They inserted the new DNA sequences into living liver and blood cells using a harmless virus. They then measured how well these new switches turned on a glowing reporter gene. The results were striking. The switches designed for liver cells lit up brightly in liver cells but remained dark in blood cells, and vice versa. This confirmed that the computer had successfully learned to write cell-type-specific instructions. Furthermore, the new liver switches were, on average, more active than the natural switches found in the human genome, suggesting that the computer had found a more efficient way to organize the genetic code.

The study also highlighted that not every cell type was equally easy to design for. While the system succeeded brilliantly in liver and blood cells, it struggled to create specific switches for a neural cell type. In this case, the computer's path remained scattered and failed to settle on a clear pattern. This failure was just as informative as the success; it showed that the system is limited by the quality and quantity of data available for that specific cell type. It revealed that for the computer to learn the grammar of a cell, it needs a rich library of examples to study, and without enough data, the design process cannot find a stable path.

By making the design process visible and adjustable, this work transforms how scientists approach genetic engineering. Instead of hoping for a lucky result, they can now watch the sequence evolve, spot when it is going off track, and guide it back to a productive path. The resulting switches are not just random sequences that happen to work; they are the product of a guided evolution that converges on compact, efficient, and highly specific biological programs. This approach offers a powerful new tool for creating the precise genetic controls needed for future therapies, turning the abstract task of writing DNA into a tangible, understandable, and controllable process.

Technical Summary: Guided Optimization of Cis-Regulatory Elements (GO-CRE)

Problem Statement

Designing cis-regulatory elements (CREs) with cell-type-specific activity is essential for precise gene and cell therapies. While genomic language models (gLMs) have demonstrated potential for generating CREs, current optimization approaches often treat the process as a "black box." This lack of interpretability limits the understanding of how regulatory grammar emerges, why optimization trajectories fail (e.g., getting trapped in low-complexity sequences), and how to correct unsuccessful generation paths. Furthermore, existing methods often prioritize endpoint performance without revealing the biological events occurring during the iterative optimization process.

Methodology

1. The Generative Backbone: HybriDNA

To address computational constraints in iterative reinforcement learning (RL), the authors developed HybriDNA, a decoder-only genomic language model.

Architecture: It interleaves Mamba2 selective state-space model (SSM) blocks with Transformer attention layers at a 7:1 ratio. This hybrid design leverages the sub-quadratic scaling of Mamba2 for efficient long-context processing while retaining high-fidelity nucleotide interactions via periodic Transformer layers.
Training: Pre-trained on a multispecies corpus (~160 billion nucleotides from 816 species) and fine-tuned on cell-type-specific MPRA data.
Reverse-Complement Echo (RCE): The model incorporates RCE embeddings, concatenating input sequences with their reverse complements to ensure base-level representations capture information from the full input sequence.
Efficiency: HybriDNA demonstrated superior training throughput and lower peak memory consumption compared to pure Transformer and StripedHyena models, enabling full-batch Proximal Policy Optimization (PPO) on standard hardware where larger models (e.g., Evo 2-7B) failed.

2. The Framework: GO-CRE

Guided Optimization of Cis-Regulatory Elements (GO-CRE) is an interpretable RL framework built upon HybriDNA.

Reward Function: The system uses a reward based on the MinGap score, defined as the predicted activity in the target cell type minus a penalty for activity in off-target cell types. An off-target penalty coefficient is linearly increased during training to enforce specificity.
Trajectory Reconstruction: A core innovation is the deconvolution of RL iterations into k-mer and motif-level features. These are projected into a shared "sequence-grammar space" using Principal Component Analysis (PCA) on $\Delta$ 6-mer profiles.
Biologically Guided Redirection: The framework analyzes these trajectories to identify failure modes. In the HepG2 optimization, a "low-complexity polyG trap" was identified. A polyG penalty was subsequently added to the reward function to redirect the optimization away from this trap.
Phase Analysis: The optimization process is decomposed into three distinct phases: Search (broad exploration), Commitment (directional movement toward lineage-specific features), and Optimization (local diversification while retaining core grammar).

3. Experimental Validation

The authors employed Lentiviral Massively Parallel Reporter Assays (LentiMPRA) to validate the generated sequences.

Design: A dual-reporter vector (hPGK-mCherry for normalization, minimal promoter-EGFP for activity) was used.
Cell Types: Libraries were tested in HepG2 (liver), K562 (hematopoietic), and SK-N-SH (neural) cell lines.
Controls: Comparisons were made against endogenous CREs (identified via STARR-seq), natural sequences, and sequences generated by other models (regLM, DNA-Diffusion).

Key Results

1. Computational Efficiency and Performance

HybriDNA achieved higher training throughput and lower memory usage than comparable models.
In benchmark tasks (BEND and GUE), HybriDNA matched or outperformed established gLMs (e.g., DNABERT-2, Nucleotide Transformer) in predicting chromatin accessibility, histone modifications, and transcription factor binding.

2. Trajectory Analysis and Redirection

Failure Mode Identification: In the initial HepG2 run, optimization trajectories became trapped in a high-GC, low-complexity region characterized by accumulating polyG features.
Successful Intervention: Introducing a polyG penalty into the reward function successfully redirected the trajectory. This intervention accelerated the improvement of the MinGap score and led to a stable, higher-reward plateau.
Phase Transitions: The analysis revealed that productive trajectories undergo specific transitions. In the penalized HepG2 run, the transition from "search" to "commitment" was marked by the sustained dominance of new sequence-update programs, whereas the unpenalized run showed unstable transitions.

3. Cell-Type-Specific Generation

HepG2: Generated CREs converged on a compact HNF1B-FOXA1-HNF4A regulatory program. The trajectory showed a progressive acquisition of these motifs, with FOXA-family motifs emerging early (consistent with pioneer factor roles) followed by HNF factors.
K562: Designs converged on GATA1/2 and RUNX-associated motifs, reflecting hematopoietic lineage requirements.
SK-N-SH: While sequence diversity was maintained, the trajectory showed weak convergence toward a specific neural-lineage grammar, resulting in designs enriched for broadly active or inflammatory motifs (e.g., NF-κB) rather than specific neural regulators.

4. Motif Syntax and Compactness

Unlike endogenous CREs, which often exhibit broad motif distributions, GO-CRE-generated CREs converged on compact, shared motif grammars.
In HepG2, the generated sequences displayed a highly structured, recurrent spacing and orientation of the HNF1B-FOXA1-HNF4A triplet, a higher-order syntax not as prevalent in endogenous sequences.

5. Experimental Validation (LentiMPRA)

Specificity: In HepG2 and K562, target-specific CREs were significantly enriched in high-activity bins compared to off-target sequences and controls.
Activity Levels: On average, the generated HepG2 CREs exhibited higher activity than endogenous CREs, though the top-performing sequences in both groups achieved comparable scores.
SK-N-SH Limitation: SK-N-SH designs did not show clear target-selective activity, consistent with the weak convergence observed in the trajectory analysis.

Significance and Claims

The paper claims that GO-CRE transforms synthetic CRE design from a black-box endpoint optimization into a trajectory-resolved, interpretable, and steerable process.

Interpretability: By reconstructing sequence-grammar trajectories, the framework allows researchers to diagnose unproductive optimization paths (such as the polyG trap) and apply biologically informed reward shaping to correct them.
Mechanistic Insight: The approach links iterative sequence changes to the emergence of regulatory grammar, revealing that successful optimization involves coordinated transitions among sequence-update programs and convergence toward lineage-aligned features.
Compactness vs. Diversity: The results suggest that synthetic CREs can achieve regulatory activity comparable to endogenous elements by converging on compact, lineage-associated motif programs, rather than recapitulating the full heterogeneity of natural sequences.
Limitations: The authors modestly note that the framework's performance is dependent on the quality of training data and predictors. The weaker performance in SK-N-SH highlights the need for richer cell-state-resolved data and biological priors for underrepresented cellular systems. Additionally, current validation relies on reporter assays outside native genomic contexts, suggesting future work should test designs at endogenous loci.

In conclusion, GO-CRE establishes a foundation for controllable CRE engineering by making the dynamics of regulatory grammar emergence visible and actionable.