🧬 biology

Biological Reasoning-Informed Regression for Interpretable Regulatory DNA Activity Prediction

The paper introduces R3LM, a novel framework that leverages structured biological knowledge and a two-stage training process to enable large language models to perform interpretable, state-of-the-art regression for predicting regulatory DNA activity while providing mechanistic explanations.

Original authors: Yi Duan, Zhao Yang, Jiwei Zhu, Ying Ba, Chuan Cao, Bing Su

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yi Duan, Zhao Yang, Jiwei Zhu, Ying Ba, Chuan Cao, Bing Su

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

The Big Problem: The "Black Box" vs. The "Translator"

Imagine DNA as a massive library of instruction manuals written in a secret code (A, C, G, T). Scientists want to know which parts of these manuals act as "on/off switches" (called enhancers) to control how genes work.

For a long time, computers have been good at guessing which DNA sequence makes a switch turn "on" or "off." However, they do this like a black box: you put the DNA in, and a number comes out. The computer says, "This sequence is 85% active," but it can't explain why. It's like a magic 8-ball that gives the right answer but refuses to show its work.

Biologists hate this because they need to know the reasoning to trust the answer and to design new switches. They want the computer to say, "This switch is active because it has a specific key (motif) that fits a lock, and the spacing between them is just right."

The Failed Attempt: Asking a Genius to Read a Secret Code

The researchers tried using Large Language Models (LLMs)—the same kind of AI that writes essays and solves math problems—to read the DNA. They thought, "Since LLMs are great at reasoning, they should be able to figure out the DNA logic."

The Result: It failed miserably.

The Analogy: Imagine handing a brilliant translator a book written in a language they've never seen, with no dictionary, just a long string of random letters. Even if the translator is a genius, they can't guess the meaning.
The Paper's Finding: When the AI was just given raw DNA letters (e.g., "ACGT..."), it performed no better than random guessing. It didn't understand the "grammar" of life.

The Solution: R3LM (The "Translator + Detective" Team)

The team realized the AI wasn't stupid; it just needed the DNA translated into a format it could understand before it tried to reason. They built a framework called R3LM (Reasoning Regression Reward Language Model).

Here is how it works, step-by-step:

1. The "Regulatory Context Card" (RCC) – The Translator

Instead of feeding the AI raw DNA letters, they first run the DNA through a specialized tool that acts like a translator.

What it does: It scans the DNA and creates a structured "report card" (the RCC).
The Analogy: Instead of giving the detective a pile of unsorted evidence, the translator organizes it into a neat file: "Here are the fingerprints (motifs), here is the weather report (GC content), and here is the timeline of events (grammar/spacing)."
The Result: Now, the AI can actually "read" the evidence because it's presented in a structured, logical format.

2. The "Reasoning Trace" – The Detective's Notebook

The researchers created a new dataset called CRE-ReasonBench. This isn't just a list of DNA and scores; it includes a step-by-step story explaining why a specific DNA sequence has a certain activity level.

The Analogy: It's like a math teacher not just giving the answer key, but writing out the full solution: "Step 1: Identify the variable. Step 2: Apply the formula. Step 3: Calculate the result."
The Goal: They taught the AI to write these "detective stories" (reasoning traces) before it gives the final score.

3. The Two-Stage Training – Learning to Think, Then Predicting

They trained the AI in two distinct phases to make sure it actually learned to reason:

Stage 1 (The Reasoning Teacher): They taught the AI to look at the "Report Card" (RCC) and write the "Detective Story" (the reasoning steps). It learned to say, "Because there is a strong 'MYPOP' motif here, and the spacing is correct, the activity should be high."
Stage 2 (The Predictor): Once the AI could write the story, they trained it to use that story to predict the final number (the activity score). Crucially, the AI had to predict the number based on the story it just wrote, not just memorize the DNA.

The Results: Why It Matters

The paper tested this system on three different types of human cells. Here is what they found:

It's Smarter: R3LM predicted DNA activity better than previous "black box" models (like Enformer) and much better than just asking an AI to read raw DNA.
It's Transparent: Unlike other models, R3LM can show its work. If a biologist asks, "Why did you give this a high score?", R3LM can point to the specific "fingerprints" (motifs) and "spacing" it used to make that decision.
It's a "Reward Model": In the world of designing new DNA, scientists use computers to try millions of designs. R3LM acts as a judge that says, "This design is good because of X, Y, and Z," helping scientists design better biological switches faster.

Summary in One Sentence

The paper introduces a new AI system that first translates raw DNA into a structured "report card," then teaches the AI to write a step-by-step detective story explaining the biology, and finally uses that story to accurately predict how active a DNA sequence will be, making the AI's decisions understandable to human scientists.

Technical Summary: Biological Reasoning-Informed Regression for Interpretable Regulatory DNA Activity Prediction

Problem Statement
Predicting the regulatory activity of cis-regulatory elements (CREs), such as enhancers, from DNA sequences is a critical task for metabolic engineering, personalized medicine, and agriculture. While deep learning models (e.g., Enformer) have achieved success by learning end-to-end mappings from sequence to activity, they operate as black boxes. This lack of interpretability makes them vulnerable to "reward hacking" when used as reward models for sequence optimization and limits biologist trust. Conversely, Large Language Models (LLMs) excel at step-by-step reasoning (Chain-of-Thought) but fail when applied directly to raw DNA sequences ("ACGT...") due to a representation mismatch; they cannot inherently extract biologically meaningful features like transcription factor binding sites (TFBSs) or regulatory grammar from unstructured nucleotide strings.

Methodology: The R3LM Framework
The authors propose R3LM (Reasoning Regression Reward Language Model), a framework designed to bridge the gap between LLM reasoning capabilities and biological sequence analysis through a two-stage training pipeline.

Regulatory Context Card (RCC):
To address the representation mismatch, the authors introduce RCC, a structured input format that compiles raw DNA sequences into interpretable, biologically grounded representations. Instead of feeding raw nucleotides, the RCC transforms a sequence $S$ and context $c$ (e.g., cell type) into a deterministic schema containing:
- Motif Evidence ( $M$ ): TFBSs identified via scanning against JASPAR databases (family, score, position, strand).
- Sequence Statistics ( $G$ ): Global priors such as GC content.
- Grammar Tags ( $T$ ): Higher-level heuristics derived from motif arrangements (e.g., spacing, co-occurrence).
- Context: Explicit experimental conditions.
  This transformation allows LLMs to "read" the regulatory logic rather than arbitrary strings.
CRE-ReasonBench Dataset:
The authors construct the first dataset pairing DNA sequences with mechanistic reasoning traces. Using a frontier LLM (Claude Sonnet 4.5), they synthesize step-by-step rationales that explain how specific motif configurations, spacing, and cellular context determine activity levels. Each sample includes an RCC prompt, a structured reasoning trace (CoT), and a discrete activity level label (0–3).
Two-Stage Training Strategy:
To avoid gradient conflicts and memory issues associated with joint optimization of generation and regression, R3LM employs a decoupled two-stage approach:
- Stage 1 (Reasoning Generator): A Chain-of-Thought generator ( $g_\theta$ ) is trained to produce mechanistic rationales and a discrete activity level ( $\ell$ ) based on the RCC prompt. The model is optimized using standard language modeling loss.
- Stage 2 (Reason-Conditioned Regressor): A regressor ( $f_\Theta$ ) is trained to predict the continuous activity score ( $y$ ). The input consists of the RCC prompt concatenated with the self-generated rationale from Stage 1 (excluding the final discrete conclusion). The model extracts a hidden state at an anchor position and projects it to a continuous score using Mean Squared Error (MSE) loss. Crucially, the regressor is trained on self-generated rationales (filtered only for syntactic validity) to align training with the inference-time distribution, preventing the "oracle gap."

Key Contributions

RCC: A standardized, programmatic schema that transforms raw DNA into structured, auditable biological evidence accessible to LLMs.
CRE-ReasonBench: The first dataset linking DNA sequences and activity scores with explicit, step-by-step mechanistic reasoning traces.
R3LM Framework: A two-stage training pipeline that enables LLMs to perform reasoning-informed regression, achieving state-of-the-art performance while providing interpretable explanations.
Empirical Validation: Demonstration that biological reasoning-informed LLMs outperform specialized black-box models (like Enformer) and raw-sequence LLMs across three cell types (HepG2, K562, SK-N-SH).

Results

Regression Performance: R3LM achieves state-of-the-art Pearson correlation ( $\rho$ ) and lowest RMSE on enhancer prediction tasks. For example, on the K562 cell type, R3LM (Generated-CoT) achieved $\rho = 0.8246$ , outperforming Enformer ( $\rho = 0.7990$ ) and raw-sequence LLMs ( $\rho = 0.3919$ ).
Interpretability: The model provides mechanistic rationales grounded in RCC evidence (motifs, grammar, GC content). Ablation studies confirm that the regressor relies on these rationales; shuffling the rationale text at test time causes performance to collapse (from $\rho = 0.8246$ to $0.0198$), proving the reasoning trace is an informative intermediate factor.
Reasoning Quality: The Stage 1 generator achieves high syntactic validity (~97–99%) and reasonable accuracy in predicting discrete activity levels (Acc@1 $\approx$ 78–81%), even though exact level prediction is imperfect. The regressor remains robust despite these imperfections.

Significance and Claims
The paper claims that R3LM addresses the fundamental limitation of current CRE prediction models: the trade-off between performance and interpretability. By teaching LLMs to reason over structured biological knowledge rather than raw sequences, the framework serves as an interpretable reward model. This allows biologists to not only receive a predicted activity score but also inspect the mechanistic evidence (e.g., specific TF binding and spacing logic) supporting that prediction. The authors position R3LM as a tool to assist in the design of tailored CREs, offering a transparent alternative to black-box neural networks while maintaining competitive predictive accuracy. The work emphasizes that the generated rationales are "grounded and auditable" proxies based on programmatic evidence, rather than claims of discovering new ground-truth causal mechanisms.