Technical Summary: ProMORNA for Full-Length mRNA Design

Problem Statement

Designing therapeutic messenger RNA (mRNA) requires generating full-length transcripts that simultaneously optimize multiple, often conflicting, biological objectives: structural stability, translation efficiency, and immune safety. The design space is exponentially large due to the redundancy of the genetic code (synonymous codons) and the variability of untranslated regions (UTRs).

Existing approaches face significant limitations:

• Heuristic Methods: Tools like LinearDesign optimize proxy metrics (e.g., Minimum Free Energy) but often fail to achieve maximal in vivo expression due to complex, competing biological constraints.
• Supervised Generative Models: Models trained solely on natural sequences learn patterns from existing data but do not directly optimize for downstream biological objectives and struggle to explore candidates beyond the training distribution.
• Reinforcement Learning (RL) Challenges: Applying RL to full-length mRNA generation is difficult because biological efficacy is governed by multiple metrics with different units, scales, and noise levels. Standard approaches that scalarize these metrics into a single reward are often unstable and sensitive to weighting choices. Furthermore, many current generative models (e.g., decoder-only transformers) condition the 5' UTR generation only on partial context, failing to jointly model the interaction between the 5' UTR, Coding Sequence (CDS), and 3' UTR based on the complete target protein.

Methodology: ProMORNA

The authors propose ProMORNA (Protein-conditioned Multi-Objective mRNA generation), an end-to-end framework that generates complete mRNA transcripts directly from a target protein sequence.

1. Model Architecture and Training

• Architecture: The system utilizes a BART-style encoder–decoder model (BART-small configuration, ~45M parameters).
  • Encoder: Processes the full amino acid sequence of the target protein.
  • Decoder: Autoregressively generates the full-length mRNA (5' UTR, CDS, 3' UTR).
  • Conditioning: Unlike decoder-only models that generate the 5' UTR before seeing the CDS context, ProMORNA conditions every generated token (including the 5' UTR) on the complete target protein sequence via the encoder.
• Tokenization: Uses a hybrid vocabulary: per-nucleotide tokenization for UTRs and per-codon tokenization for the CDS, with special separators to manage region boundaries.
• Pre-training & Fine-tuning:
  • Supervised Pre-training: Trained on 6.09 million natural protein–mRNA pairs from NCBI RefSeq to learn general codon usage and UTR compositions.
  • Supervised Fine-tuning (SFT): Further trained on a high-confidence subset (54,573 records) to ensure transcript validity.

2. Multi-Objective Group Relative Policy Optimization (MO-GRPO)

To align the model with biological goals without unstable scalarization, the authors introduce MO-GRPO.

• Biological Objectives: The framework optimizes five distinct metrics:
  1. Predicted Half-life: Maximized (using a frozen mRNABERT-based ridge regression predictor).
  2. Predicted Translation Efficiency: Maximized (using a similar mRNABERT-based predictor).
  3. Normalized Minimum Free Energy (MFE): Minimized (proxy for thermodynamic stability).
  4. U-content: Minimized (to reduce innate immune activation).
  5. UTR Length Plausibility: Maximized (using Gaussian priors based on empirical distributions).
• Optimization Mechanism:
  • Instead of summing raw scores, MO-GRPO samples a group of $G$ sequences for a given prompt.
  • It computes the empirical mean ($\mu_k$) and standard deviation ($\sigma_k$) for each objective $k$ within the group.
  • It calculates a standardized advantage for each objective: $(r_{i,k} - \mu_k) / \sigma_k$.
  • These standardized advantages are aggregated using weighted sums (where weights indicate direction and importance) to form a single advantage signal for policy updates.
  • Stability Mechanisms: Includes dynamic damping factors (to handle low-variance metrics), extreme-value clipping, and post-centralization to ensure training stability without re-dividing by standard deviation, which preserves adaptive magnitude.

3. Constraints and Validation

• Constrained Decoding: The decoder is restricted to synonymous codons for the CDS and valid nucleotides for UTRs.
• Post-hoc Checks: Generated sequences are validated for region boundaries, frame consistency, start/stop codons, and the absence of long double-stranded regions (>33 bp) to ensure biological plausibility. Invalid sequences receive maximal penalties.

Key Contributions

1. Protein-Conditioned Full-Length Generation: An encoder-decoder framework that generates the entire transcript (5' UTR, CDS, 3' UTR) conditioned on the full target protein sequence, enabling joint optimization of interacting regions without a wild-type template.
2. MO-GRPO Algorithm: A novel RL optimization method that standardizes relative advantages at the metric level before aggregation. This avoids the instability of scalarizing raw biological scores with different scales and allows for flexible balancing of multiple objectives.
3. Empirical Validation: A comprehensive evaluation of generated candidates against supervised baselines and the state-of-the-art GEMORNA framework, focusing on held-out target generalization.

Results

The framework was evaluated on firefly luciferase, a target protein excluded from all training data and the prompt pool.

• Functional Performance: ProMORNA achieved the highest scores for predicted half-life and predicted translation efficiency compared to supervised baselines (BASE, SFT) and the GEMORNA baseline.
• Pareto Frontier: Analysis of the Pareto frontier for half-life and translation efficiency showed that ProMORNA's frontier shifted significantly to the upper-right, indicating simultaneous improvement in both objectives rather than a trade-off.
• Sequence Properties:
  • ProMORNA generated sequences with moderate GC content increases and significantly reduced accessible TLR7/8 sensing motifs (improving immune safety) compared to supervised baselines.
  • It achieved a balanced distribution of UTR lengths tailored to the specific target, rather than defaulting to fixed lengths.
  • While GEMORNA showed slightly better raw MFE and U-content scores, ProMORNA demonstrated a more balanced trade-off across all metrics, prioritizing functional output.
• Learning Dynamics: UMAP visualization of 5-mer features indicated that MO-GRPO successfully moved the generator away from the supervised distribution into new sequence-composition regimes while improving reward metrics.

Significance and Claims

The paper claims that ProMORNA demonstrates the feasibility of using multi-objective reinforcement learning for full-length mRNA design on unseen targets.

• Scientific Impact: The study suggests that on-policy reinforcement learning can effectively expand the search capability of generative models beyond what is possible with supervised sequence modeling alone.
• Methodological Advance: By addressing the instability of multi-objective optimization in biological sequence design through MO-GRPO, the framework offers a robust method for balancing competing biological constraints (stability vs. efficiency vs. safety).
• Limitations and Modesty: The authors explicitly state that the results are computational findings (in silico). The evaluation relies on proxy predictors trained on existing data, and the study uses a single held-out reporter protein. Consequently, the paper does not claim proven in vivo biological activity or therapeutic efficacy, noting that experimental validation in wet-lab systems is required to confirm improved biological activity. Future work is identified as necessary to improve predictor accuracy and integrate experimental feedback for closed-loop optimization.