Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference

Technical Summary: Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference

Problem Statement
Reconstructing Gene Regulatory Networks (GRNs) from paired single-cell transcriptomic (scRNA-seq) and chromatin accessibility (scATAC-seq) data is a critical challenge in computational biology. While integrating these modalities offers complementary evidence, current methods face significant hurdles:

1. Data Sparsity and Noise: scATAC-seq data is extremely sparse and noisy, making it difficult to distinguish genuine regulatory signals from stochastic fluctuations.
2. Ambiguous Linking: Linking chromatin accessibility peaks to their target genes is fundamentally ambiguous. Existing approaches often rely on fixed, heuristic peak-to-gene mappings (e.g., based on genomic distance), which can assign accessibility evidence to incorrect targets and fail to capture context-specific enhancer-promoter interactions.
3. Label Scarcity: Accurate GRN inference requires learning directed regulatory relationships from observational data with limited reliable ground-truth labels.

Methodology: EpiAwareNet
The authors propose EpiAwareNet, a prior-guided multi-omic Transformer framework designed to reconstruct GRNs using only lightweight biological priors. The framework operates in two distinct stages:

Stage 1: Prior-Guided Representation Learning
This stage focuses on learning joint gene-peak representations without hard-coded peak-to-gene assignments.

• Architecture: A Transformer-based backbone processes paired scRNA-seq and scATAC-seq data.
• Candidate-Constrained Cross-Attention: Instead of fixed mappings, the model defines a candidate set of ATAC peaks ($C_g$) for each gene $g$ using genomic proximity as a lightweight prior. It employs a sparse gene-peak cross-attention mechanism where gene tokens attend only to peaks within their candidate set. This allows for data-driven, soft aggregation of accessibility signals, enabling the model to learn which candidate peaks best explain gene expression variation.
• Sparse Gene-Gene Attention: To manage computational complexity, gene-gene self-attention is restricted to $k$-nearest neighbors in the latent space.
• Pretraining Objective: The backbone is pretrained via a masked gene reconstruction task using a Negative Binomial (NB) loss, which better models the overdispersed nature of scRNA-seq data.

Stage 2: Weakly Supervised GRN Refinement
This stage introduces directionality and edge scoring using weak supervision.

• Weak Priors: The model incorporates a bulk-derived GRN as a set of "noisy positive" edges ($P_{bulk}$). These edges are not treated as ground truth but as soft guides.
• Positive-Unlabeled (PU) Learning: The remaining regulator-target pairs are treated as unlabeled. The model is trained to distinguish the noisy positives from the unlabeled set.
• Objectives: The authors evaluate two objectives:
  1. Binary Cross-Entropy (BCE): Treats sampled unlabeled pairs as negatives, optimizing for global ranking quality.
  2. Non-negative PU (nnPU): Explicitly accounts for hidden positives among unlabeled pairs, resulting in a more conservative scoring function that concentrates probability mass on a smaller set of high-confidence edges, optimizing for early retrieval precision.
• Prediction: A lightweight prediction head scores ordered regulator-target pairs based on aggregated cell-level representations derived from the frozen Stage 1 backbone.

Key Contributions

1. Prior-Guided Cross-Modal Representation Learning: The paper formulates GRN inference as a representation learning problem that uses sparse gene-peak cross-attention within weak genomic candidate sets, avoiding rigid peak-to-gene mapping rules.
2. Weakly Supervised Refinement with Noisy Priors: It introduces a formulation where bulk-derived edges serve as noisy positives in a PU learning setting. The authors analyze the trade-off between global ranking (BCE) and very-top precision (nnPU).
3. Comprehensive Evaluation: The framework is evaluated across mammalian (human PBMC, mouse brain) and plant (tomato root) systems, demonstrating improved reconstruction robustness and biological plausibility compared to baselines.

Experimental Results
The authors evaluated EpiAwareNet on four paired multi-omic datasets against classical RNA-only methods (GRNBoost2, WGCNA), deep representation baselines (scGPT, scGLUE with matched heads), and recent multi-omic pipelines (SCENIC+, Pando).

• Global Ranking: EpiAwareNet consistently achieved superior AUPRC and AUROC scores across all datasets compared to baselines. For instance, on the tomato mN dataset, it improved AUPRC from 0.045 (GRNBoost2) to 0.08458.
• Early Retrieval: The EpiAwareNet-nnPU variant demonstrated superior performance in "early retrieval" metrics (Precision@K, Hit@K) on plant datasets, achieving higher precision in the top-ranked edges compared to the standard BCE variant and other baselines.
• Ablation Studies:
  • Removing ATAC data (RNA-only) generally degraded performance, confirming the value of chromatin accessibility as complementary evidence, though the benefit was dataset-dependent.
  • Randomizing the gene-peak candidate sets significantly degraded performance, validating the importance of the candidate-constrained attention mechanism.
• Robustness: The model showed robustness to noise in the bulk-derived prior; performance plateaued rather than collapsed when a significant fraction of prior edges were dropped, suggesting the model learns signal beyond simply reproducing the prior.

Significance and Claims
The paper claims that EpiAwareNet effectively addresses the challenges of sparse, noisy multi-omic data by combining adaptive cross-modal representation learning with weak biological supervision. The authors argue that:

• Lightweight biological priors from bulk data can effectively guide single-cell GRN inference when paired with adaptive Transformers.
• Moving away from fixed peak-to-gene heuristics toward data-driven, candidate-constrained attention improves the robustness and biological plausibility of inferred networks.
• The framework offers a flexible trade-off between global ranking quality and high-precision top-edge retrieval depending on the chosen objective (BCE vs. nnPU).

The work is positioned not as a claim that every component is novel in isolation, but as a specific, effective formulation for refining GRNs from paired scRNA-seq/scATAC-seq data under conditions of label scarcity and data sparsity.