💻 bioinformatics

PARNET: A CLIP-SEQ-BASED FOUNDATION MODEL FOR RNA SEQUENCE REPRESENTATION LEARNING

PARNET is a novel RNA foundation model trained exclusively on experimental CLIP-seq data to predict base-resolution RBP binding profiles, demonstrating superior performance and mechanistic interpretability across diverse downstream tasks compared to traditional sequence-based language models.

Original authors: Moyon, L., Tirabassi, A., Baranowskii, A., Capitanchik, C., Kuret Hodnik, K., Wilkinson, L., Londhe, S., Dumbovic, G., Gagneur, J., Ule, J., Horlacher, M., Marsico, A.

Published 2026-08-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Moyon, L., Tirabassi, A., Baranowskii, A., Capitanchik, C., Kuret Hodnik, K., Wilkinson, L., Londhe, S., Dumbovic, G., Gagneur, J., Ule, J., Horlacher, M., Marsico, A.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine your body is a bustling, high-tech city. In this city, DNA is the master blueprint stored safely in the city hall, but the actual work happens on the streets. To get the blueprints to the construction sites, the city sends out copies called RNA. These RNA messages are like delivery trucks carrying instructions to build proteins, the workers that keep the city running. But here's the catch: these RNA trucks are fragile, and the instructions on them can be messy or ambiguous. If a truck breaks down, gets lost, or delivers the wrong cargo, the whole city could face chaos, leading to diseases.

To keep everything running smoothly, the city employs a massive team of specialized traffic controllers called RNA-binding proteins (RBPs). These controllers don't just drive the trucks; they read the RNA messages, decide where they should go, how long they should stay, and whether they should be translated into proteins or thrown away. They act like a complex, invisible code that determines the fate of every RNA message. For years, scientists have tried to crack this code using computers, but it's been like trying to guess the rules of a game just by looking at the board pieces. The big question is: can we build a smart computer that understands not just the letters of the RNA message, but also how the traffic controllers interact with them to control the city's life?

Enter Parnet, a new kind of "super-smart" computer model that claims to have found a shortcut to understanding this biological traffic system. Instead of trying to guess the rules by reading millions of RNA messages in the dark (a method called self-supervised learning, which is like a student trying to learn a language by reading a book with no pictures), Parnet learned by watching the traffic controllers in action. The researchers trained Parnet on a massive dataset of 223 different experiments where scientists actually watched 150 different types of RNA-binding proteins grab onto RNA strands. Think of it as teaching a student by showing them thousands of videos of the traffic controllers doing their jobs, rather than just giving them a dictionary.

The results are surprisingly powerful. Parnet learned to predict exactly where these proteins would bind to an RNA strand, down to the single letter, with much higher accuracy than previous models. But the real magic happened when the researchers tested what Parnet had actually learned. They "froze" the model's brain and used its internal knowledge to solve other puzzles it had never seen before. Without any extra training, Parnet could successfully predict:

Which RNA messages would stick to the cell's "walls" (chromatin) and which would float freely.
How efficiently a message would be turned into a protein (translation efficiency).
Whether a specific RNA cut-and-paste event (splicing) would happen correctly or go wrong.
Even the impact of tiny genetic typos (mutations) that might cause disease.

What makes Parnet special is that it didn't just get the right answers; it explained why. Because it learned directly from the interactions between proteins and RNA, it can point to the specific "traffic controllers" and the specific patterns on the RNA that caused a prediction. For example, if it predicts a mutation will cause a disease, it can show you that the mutation broke the binding site for a specific protein, effectively jamming the traffic.

The paper suggests that this approach—learning directly from the physical interactions between proteins and RNA—is a much more efficient and understandable way to build AI for biology than the current trend of just reading sequences. While the model was trained on data from just two types of human cells, it showed it could generalize to unseen cell types and even predict how proteins bind in diseases like ALS, where a protein called TDP-43 goes rogue. The authors are careful to note that while Parnet is a huge step forward, it's not perfect; it currently only knows about the proteins it was trained on, and it doesn't yet account for the 3D shape of RNA or chemical modifications. However, by grounding its knowledge in real, measured biological interactions, Parnet offers a clear, interpretable window into the complex code that governs how our cells function, potentially helping scientists prioritize which genetic mutations to study for new treatments.

Technical Summary: PARNET – A CLIP-Seq-Based Foundation Model for RNA Sequence Representation Learning

Problem Statement

RNA-binding proteins (RBPs) orchestrate a complex combinatorial regulatory "code" governing RNA splicing, stability, localization, and translation. While deep learning has transformed genomics, existing RNA foundation models primarily rely on self-supervised objectives (e.g., masked language modeling) trained on large unlabeled sequence corpora. These models learn general-purpose representations based on sequence statistics and conservation but lack direct connections to the regulatory principles governing RNA function. Consequently, they often require extensive fine-tuning for downstream tasks and may not efficiently capture sequence features most relevant to post-transcriptional regulation. The authors posit that knowing an RNA's binding partners provides direct insight into its processing and function, suggesting that a model densely representing an RNA's interactome could serve as a superior foundation for diverse RNA tasks.

Methodology

Data and Preprocessing

The model, Parnet, is trained exclusively on experimental eCLIP-seq data from the ENCODE database. The dataset comprises 223 eCLIP experiments spanning 150 unique RBPs across two cell lines (K562 and HepG2).

Input: Raw RNA sequences are extracted from GENCODE (v48), tiled into 600-nucleotide sequences with 150nt overlaps, and one-hot encoded.
Signal Processing: For each experiment, biological replicates are merged, and 5' read-start counts are extracted at each nucleotide position. Size-Matched Input (SMI) controls are used to estimate locus-specific background noise.

Model Architecture

Parnet extends the single-task RBPNet architecture into a multi-task framework:

Backbone: A 1D convolutional neural network with residual blocks (9 blocks total) processes the input sequence. The model size was scaled up from 0.5M (RBPNet) to 21 million parameters to accommodate the multi-task nature.
Output Heads: The shared backbone branches into 223 task-specific output heads, one for each eCLIP experiment.
Signal Decomposition: Each head predicts a "total track" (mimicking the eCLIP signal) modeled as a mixture of two components:
1. Target Track: The protein-specific binding signal.
2. Control Track: The experimental background (SMI) signal.
- The total signal is defined as $p_{total} = \pi \times p_{target} + (1 - \pi) \times p_{control}$ , where $\pi$ is a learned mixing coefficient.
Loss Function: The model minimizes the negative log-likelihood for both target and control tracks. Crucially, a penalty term ( $\lambda \pi$ ) is added to the loss to incentivize the model to maximize the use of the control track for background noise, thereby "denoising" the target track to isolate genuine protein-RNA interactions.

Training Strategy

Objective: End-to-end supervised training to predict base-resolution RBP binding profiles directly from raw RNA sequences.
Optimization: AdamW optimizer with early stopping based on validation performance.
Hyperparameters: A penalty factor of $\lambda = 10$ was identified as the optimal trade-off for denoising while preserving motif signal.

Key Contributions

Novel Pretraining Strategy: Parnet introduces a foundation model pretraining strategy grounded in experimentally measured protein-RNA interactions (CLIP-seq) rather than self-supervised sequence reconstruction. This provides stronger, biologically motivated inductive biases.
Multi-Task Learning: By jointly modeling 223 eCLIP experiments, Parnet captures complex, combinatorial RBP-binding patterns and leverages correlations between related processes (e.g., co-binding patterns), improving performance on low-quality datasets.
Mechanistic Interpretability: Unlike black-box language models, Parnet's predictions can be traced back to specific RBPs and their sequence motifs via Integrated Gradients and embedding space interrogation, directly linking computational predictions to molecular machinery.
Efficiency: Despite having significantly fewer parameters (21M) than many existing genomic foundation models (e.g., Nucleotide Transformer, Evo2), Parnet achieves state-of-the-art performance across diverse downstream tasks with minimal or no fine-tuning.

Results

1. Improved Binding Profile Prediction

Accuracy: Parnet outperforms single-task RBPNet models, achieving an average 35% increase in Pearson correlation and 19% increase in Spearman correlation between predicted and observed read-start counts.
Generalization: The model generalizes to unseen cell types and assay types. For example, it successfully recovered TDP-43 binding at the cryptic exon of STMN2 in SH-SY5Y cells, despite being trained only on K562 eCLIP data where STMN2 is not expressed.
Denoising: Increasing the $\lambda$ penalty effectively separates genuine binding motifs (e.g., RBFOX2, QKI) from background noise (e.g., G-rich or U-rich patterns), resulting in target tracks that show higher concordance with in vitro motifs (RNAcompete, RNA Bind-n-Seq).

2. Recovery of Regulatory Logic

Splicing Regulation: Parnet recapitulates position-dependent splicing regulation. RNA splicing maps generated from Parnet predictions correctly identify distinct binding patterns for splicing enhancers (e.g., SR proteins at exonic regions) and silencers (e.g., PTBP1 at 3' splice sites), matching patterns derived from experimental RNA-seq knockdown data.
Zero-Shot Performance: Frozen Parnet embeddings (without fine-tuning) match or exceed the performance of larger self-supervised RNA and genomic language models (RiNALMo, Nucleotide Transformer) on tasks including:
- RNA Biotype Classification: Distinguishing mRNAs, lncRNAs, miRNAs, etc.
- Genomic Region Classification: Identifying CDS, UTRs, and introns.
- lncRNA Chromatin Localization: Predicting chromatin dissociation half-lives (Pearson $r=0.54$ with a neural network), separating chromatin-retained from nucleoplasmic lncRNAs.
- Translational Efficiency (TE): Predicting TE across multiple cell types (HEK293, muscle, PC3), outperforming specialized tools like Optimus 5-Prime and FramePool.
- Splice Site Recognition: Achieving >90% AUROC/AUPRC in ternary classification (donor, acceptor, non-splice) with frozen embeddings, outperforming Spliceator and RiNALMo.
- Intron Retention: Accurately predicting intron retention events (AUROC 0.90) by capturing splice silencing and enhancing elements.

3. Variant Effect Prediction

Zero-Shot Variant Scoring: Parnet embeddings effectively discriminate functional splice-altering variants from benign controls in the MutSpliceDB dataset, achieving an AUPRC of 0.801, surpassing SpliceAI (0.794) and RiNALMo.
Mechanistic Insight: The model identifies which specific RBPs are disrupted by a mutation. For example, it correctly identified that a pathogenic NF1 variant (missed by SpliceAI) disrupts U2AF1/U2AF2 binding, and that an RB1 variant disrupts PRPF8/AQR binding.

Significance and Claims

The paper claims that Parnet establishes the RBP interactome as a compact, functionally sufficient, and interpretable basis for foundation model pretraining in RNA biology.

Superiority of Supervised Functional Pretraining: The authors argue that anchoring representations in experimental functional data (RBP binding) is more effective for regulatory tasks than self-supervised reconstruction objectives, which optimize for sequence likelihood rather than function. This is evidenced by Parnet's ability to generalize to complex downstream tasks (splicing, translation, localization) with minimal fine-tuning, whereas self-supervised models often require task-specific tuning and are more susceptible to confounding biases like sequence length and GC content.
Interpretability: A critical claim is that Parnet offers a unique interpretability advantage. Because it is trained to predict RBP binding, its internal representations can be directly interrogated to reveal which specific proteins and motifs drive predictions for RNA properties or variant effects.
Efficiency: The model demonstrates that a relatively small, functionally grounded model (21M parameters) can outperform much larger, self-supervised models trained on orders of magnitude more data.

The authors conclude that Parnet bridges the gap between single-task binding prediction and integrated modeling of post-transcriptional regulation, providing a versatile foundation for computational RNA biology, from variant prioritization to the design of RNAs with tailored regulatory properties.