GRASP: Gene-relation adaptive soft prompt for scalable and generalizable gene network inference with large language models

The paper introduces GRASP, a parameter-efficient framework that utilizes gene- and relation-specific soft prompts to enhance the scalability and generalizability of large language models for inferring diverse gene network interactions.

The Gist

Imagine the human body as a massive, bustling city. Inside this city, every gene is a unique building or a specific worker. To keep the city running, these workers need to talk to each other, form teams, and pass instructions. This complex web of conversations is called a Gene Network.

Scientists have long tried to map out these conversations to understand diseases and how our bodies work. However, the "language" of these interactions is tricky. Some workers shake hands (physical interactions), some give orders (regulatory networks), and some modify each other's tools (phosphorylation).

For a long time, computers used different "languages" or tools to map each type of interaction. But recently, we have Large Language Models (LLMs)—super-smart AI that has read almost every scientific paper ever written. You might think, "Great! Let's just ask the AI, 'Do Gene A and Gene B talk to each other?'"

The Problem: The "One-Size-Fits-All" Prompt

The problem is that asking an AI a question is like asking a librarian for a book.

• If you just say, "Do these two people know each other?" the librarian might give a generic answer.
• If you give the librarian a massive, 50-page biography of both people, they might get overwhelmed by the details and miss the point.
• If you give them a generic template for everyone, they can't account for the unique chemistry between specific pairs.

In technical terms, standard "prompts" (the instructions we give the AI) are either too rigid (fixed text) or too generic (one set of instructions for everyone). They fail to capture the unique "vibe" of every specific gene pair.

The Solution: GRASP (The Chameleon Prompt)

The authors of this paper introduced GRASP (Gene-Relation Adaptive Soft Prompt). Think of GRASP as a chameleon or a custom-tailored suit for the AI.

Instead of giving the AI a static instruction, GRASP creates a tiny, custom-made "mental note" for every single pair of genes it is asked about.

Here is how it works, using a simple analogy:

1. The Briefing (Gene Vector Encoding):
   Before the AI tries to guess if two genes interact, GRASP asks the AI to write a one-sentence "elevator pitch" for each gene. It turns this sentence into a compact, digital ID card.

   • Analogy: Imagine you are introducing two strangers at a party. Instead of reading their whole resumes, you quickly summarize their hobbies and jobs in your head.

2. The Custom Mix (Factorized Soft Prompt):
   GRASP takes the ID cards of Gene A and Gene B, mixes them together, and adds a third card that represents "how they might interact."

   • It uses a special "recipe" (math) to create three tiny, invisible tokens (virtual words). These aren't real words you can read; they are like secret codes that tell the AI exactly how to think about this specific pair.
   • Analogy: Instead of giving the AI a generic "How do people get along?" question, GRASP whispers a secret code: "Remember, Gene A is a shy introvert and Gene B is a loud extrovert; they might clash, or they might balance each other out."

3. The Prediction:
   The AI reads these three secret codes along with the gene names and makes a highly informed guess about whether they interact.

Why is this a Big Deal?

The researchers tested GRASP on three different "cities" of gene interactions:

1. Physical Handshakes (Protein-Protein Interactions): Like checking if two people actually touched hands.
2. Boss-Employee Orders (Regulatory Networks): Like checking if a boss told an employee to do something.
3. Tool Modifications (Phosphorylation): Like checking if one worker handed a tool to another to fix it.

The Results:

• Better Accuracy: GRASP was much better at finding the right connections than other methods. It didn't just guess; it understood the context.
• Cross-Species Magic: They trained the AI on human data and then asked it about chickens, cows, and dogs. Even though it had never seen those specific animals before, GRASP could still figure out the relationships because it learned the logic of the interactions, not just memorized names.
• The "Detective" Superpower: The most exciting part? GRASP found interactions that weren't in any database yet.
  • Example: The AI predicted that a gene called INSR (the insulin receptor) and PTPRF (a phosphatase) interact. When the researchers checked the science, they found that PTPRF actually turns off the insulin signal. The AI had "deduced" a biological truth that wasn't explicitly written in its training data, acting like a detective connecting clues to solve a mystery.

The Takeaway

GRASP is like giving a super-intelligent librarian a customized, instant cheat sheet for every single question they are asked. It doesn't just rely on what it has read; it adapts its thinking style to the specific characters involved.

This means we can now use AI to map the complex wiring of our cells more accurately, potentially discovering new drug targets and understanding diseases faster than ever before. It turns the AI from a "search engine" into a "biological detective."

Technical Summary

1. Problem Statement

Gene networks (GNs) encode diverse molecular relationships (e.g., protein-protein interactions, gene regulatory networks, phosphorylation) essential for understanding cellular function and disease. While Large Language Models (LLMs) offer a unified, text-based approach to infer these networks by leveraging internalized biological knowledge, they face two critical challenges:

• Prompt Sensitivity: LLM performance in biomedicine is highly sensitive to prompt design. Fixed prompts fail to capture the heterogeneity of gene functions and interaction types.
• Scalability vs. Specificity: Naively appending verbose gene descriptions to prompts can introduce noise and obscure relational signals. Conversely, standard "soft prompts" (learnable virtual tokens) typically learn a single shared embedding for an entire task, failing to capture instance-level biological variability between specific gene pairs.

The core challenge is to design a parameter-efficient, instance-adaptive prompting framework that conditions inference on the specific context of each gene pair without requiring full model fine-tuning or verbose text inputs.

2. Methodology: GRASP Framework

The authors propose GRASP (Gene-Relation Adaptive Soft Prompt), a framework that generates a small set of adaptive virtual tokens (specifically three) conditioned on each queried gene pair.

A. Pre-training Strategy

• Domain Adaptation: The backbone LLMs (Gemma-3-4B-IT and Llama-3.1-8B-Instruct) undergo Continual Pre-training (CPT) on a corpus of 6.3 million gene-related PubMed titles and abstracts to internalize domain-specific knowledge before fine-tuning.

B. GRASP Architecture

GRASP operates in a two-stage process for every gene pair $(A, B)$:

1. Gene Vector Encoding:

   • The backbone LLM generates a concise textual summary for each gene.
   • These summaries are encoded into fixed-length vectors ($s_A, s_B$) via mean pooling of the final-layer hidden states.
   • A third context vector ($u_{A,B}$) is constructed as the element-wise absolute difference $|s_A - s_B|$ to capture contrastive relational features.

2. Factorized Soft Prompt Synthesis:

   • Instead of a single shared embedding, GRASP uses a factorized formulation to generate soft prompts: $P(z) = C(z)B(z)$.
   • $C(z)$ (Gene-specific Coefficient): A linear projection of the context vector $z$ that captures pair-specific signals.
   • $B(z)$ (Shared Prototype): A convex combination of $K$ global basis matrices, representing common interaction patterns.
   • Output: This process yields exactly three adaptive virtual tokens per gene pair: two gene-specific tokens and one relation-specific token. These are appended as a suffix to the input prompt.

3. Training Objective:

   • The LLM backbone remains frozen.
   • Only the classification head and the GRASP soft prompt parameters (coefficient matrices and prototype bases) are fine-tuned.
   • A lightweight MLP aggregates token-level logits into a sequence-level interaction probability using Binary Cross-Entropy (BCE) loss.

3. Key Contributions

• Instance-Adaptive Prompting: Unlike standard soft prompts that apply a static embedding to all inputs, GRASP dynamically synthesizes prompts based on the specific biological context of each gene pair.
• Parameter Efficiency: The method introduces only three virtual tokens per pair and uses a factorized low-rank structure, making it highly scalable for large-scale network inference.
• Noise Reduction: By compressing gene summaries into latent vectors rather than appending raw text, GRASP avoids the noise introduced by verbose or irrelevant gene descriptions, which was observed to degrade performance in fixed-prompt baselines.
• Discovery Capability: The framework demonstrates the ability to distinguish genuine biological interactions from synthetic negatives, suggesting it captures underlying biological signals beyond curated database annotations.

4. Experimental Results

GRASP was evaluated across three distinct gene network inference tasks using Gemma-3-4B and Llama-3.1-8B backbones:

A. Large-Scale Protein-Protein Interaction (PPI) Inference

• Dataset: 2.1M human gene pairs (BioGRID, MIPS, IntAct, STRING).
• Performance: GRASP achieved the highest Precision-Recall trade-off and ROC-AUC (0.931 for Gemma, 0.937 for Llama) across all configurations.
• Improvement: Outperformed baselines by ~6% in precision and ~10% in recall.
• Robustness: Showed superior performance on high-degree genes and reduced false positives in specific subnetworks (e.g., COG4, ZIC1).

B. Cross-Species Transferability

• Task: Applied human-trained models to Chicken, Cow, and Dog PPI datasets (excluding overlapping pairs).
• Result: While all methods struggled with species-specific interactions, GRASP maintained the highest ROC-AUC, indicating better generalization of relational patterns beyond simple gene-name recognition.

C. Single-Cell Perturbation Benchmarks (CausalBench)

• Task: Inferring regulatory relationships from Perturb-seq data (K562 and RPE1 cell lines) in a zero-shot setting (no expression data used).
• Result: GRASP achieved the highest biological F1 score, outperforming expression-based baselines (GRNBoost) in recovering curated regulatory interactions, highlighting the value of language-based representations.

D. Phosphorylation Network Inference

• Task: Kinase-substrate prediction (8,750 pairs).
• Result: GRASP significantly outperformed all baselines. Notably, adding gene descriptions to fixed prompts reduced performance, confirming GRASP's advantage in filtering relevant signals from noisy text.

E. Recovery of Unannotated Interactions

• Test: Evaluated the model's ability to score "hidden positives" (interactions present in external databases like IID but labeled as negatives in the training set).
• Result: GRASP achieved a Cohen's d of ~1.88, indicating near-complete distributional separation between hidden positives and true negatives.
• Case Study: The model successfully generated mechanistic hypotheses for recovered interactions (e.g., INSR-PTPRF in insulin signaling) via zero-shot chain-of-thought reasoning.

5. Significance and Conclusion

GRASP establishes instance-adaptive prompt conditioning as a superior strategy for biological prediction with LLMs. Its key significance lies in:

1. Scalability: It enables the inference of millions of interactions with minimal parameter overhead.
2. Generalizability: It effectively transfers across different interaction types (PPI, regulatory, phosphorylation) and species without task-specific feature engineering.
3. Biological Discovery: By capturing signals beyond curated databases, GRASP serves as a powerful prioritization tool for experimental validation, potentially identifying novel biological relationships that current databases miss.

The work suggests that the future of biological network inference lies in frameworks that can efficiently and adaptively query the vast, implicit knowledge internalized within large language models.