🤖 machine learning

OgBench: A Framework for Evaluating Graph Neural Networks on Omics Data

This paper introduces OgBench, the first benchmarking framework for evaluating Graph Neural Networks on omics data in the low-sample, high-node ( $n \ll p$ ) regime, revealing that standard GNNs often fail to outperform simple baselines and challenging the assumption that graph structure inherently improves performance in biological domains.

Original authors: Louisa Cornelis, Johan Mathe, Louis Van Langendonck, Guillermo Bernárdez, Nina Miolane

Published 2026-05-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Louisa Cornelis, Johan Mathe, Louis Van Langendonck, Guillermo Bernárdez, Nina Miolane

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The Wrong Kind of Map

Imagine you are trying to teach a student how to navigate a city.

The Old Way (Current Benchmarks): Most AI researchers have been training students on thousands of tiny maps (like a map of a single street corner), but each map only has a few landmarks. The student learns by seeing many different small maps.
The Real World (Omics Data): In biology (studying genes, proteins, and diseases), the situation is the opposite. You have very few patients (maybe just a few hundred), but for each patient, you have a massive map containing tens of thousands of landmarks (genes and proteins).

The paper argues that the "students" (Graph Neural Networks, or GNNs) have been trained on the wrong kind of maps. They are experts at navigating many small streets, but they are failing when asked to navigate one giant, complex city with very few guides.

What is OgBench?

The authors built OgBench (Omics-Graph Bench). Think of this as a new, specialized driving test designed specifically for the "few drivers, giant city" scenario.

Before this test, there was no standard way to see if AI could actually handle biological data. Different scientists used different messy methods to prepare their data, making it impossible to know if a model was smart or just lucky. OgBench fixes this by providing:

Clean, standardized maps: They took raw biological data and processed it the same way for everyone.
A modular toolkit: Researchers can swap out different parts of the "map-making" process (like how they draw the roads between landmarks) to see what works best.
A fair leaderboard: A place to see which AI models actually perform well on these specific biological tasks.

The Shocking Discovery: The "Simple Student" Wins

The authors ran a massive experiment, testing complex AI models (GNNs) against simple, old-school math models (like MLPs and SVMs).

The Result: The complex "super-intelligent" graph models did not consistently beat the simple models. In fact, on several datasets, the simple models performed just as well, or even better.

The Analogy:
Imagine you are trying to predict the weather.

The Complex Model (GNN): You build a giant, expensive machine that analyzes wind patterns, ocean currents, satellite images, and bird migration to draw a complex map of how air moves.
The Simple Model (MLP): You just look at the temperature and humidity numbers directly.

The paper found that for these specific biological "weather" problems, the giant machine often didn't give a better forecast than just looking at the numbers. The "structure" of the map (the roads connecting the genes) didn't seem to add much value over just looking at the individual genes themselves.

Why Did This Happen?

The paper suggests a few reasons, using the "few drivers, giant city" analogy:

Too Many Landmarks, Too Few Drivers: With thousands of genes (landmarks) but only a few hundred patients (drivers), the complex AI gets confused. It tries to memorize the noise rather than learning the real rules.
The Map Might Be Wrong: The AI assumes the roads (connections between genes) are important. But if the roads were drawn based on a guess (like "these two genes are often active at the same time"), the AI might be learning from a fake map.
The "Readout" Issue: The paper found that sometimes the AI was doing all the heavy lifting before it even looked at the map connections. It was just learning to read the individual gene numbers, ignoring the "graph" part entirely.

What Should We Do Now?

The paper isn't saying "stop using AI for biology." Instead, it's a call for honesty and better tools.

Stop assuming structure helps: Just because you can draw a graph doesn't mean the graph is useful.
Use the new test: If you want to build a new AI for biology, you should test it on OgBench, not on the old, easy benchmarks.
Focus on the right questions: Instead of just making bigger, more complex models, we need to figure out when and how the connections between genes actually help us understand disease.

Summary

OgBench is a new, fair testing ground that reveals a hard truth: Complex AI models are not automatically better at understanding biology. In the world of "few patients, many genes," simple models often hold their own, and we need to stop blindly applying complex graph tools without checking if they actually work. The paper provides the tools to fix this and find the models that truly understand the biology.

Technical Summary: OgBench – A Framework for Evaluating Graph Neural Networks on Omics Data

Problem Statement

Graph Neural Networks (GNNs) have established themselves as the dominant framework for inductive graph-level learning. However, existing benchmarks (e.g., OGB, TUDataset, LRGB) predominantly operate in the $n \gg p$ regime, where the number of graphs ( $n$ ) significantly exceeds the number of nodes per graph ( $p$ ). This contrasts sharply with the biological domain of omics (genomics, transcriptomics, proteomics, epigenomics), which operates in the $n \ll p$ regime. In omics, datasets are characterized by massive graphs (tens of thousands of genes or proteins per sample) but a scarcity of labeled samples (patients or subjects) due to cost, privacy, and clinical constraints.

Current GNN research risks stagnation because standard benchmarks do not reflect this high-dimensional, low-sample reality. In statistical learning, the $n \ll p$ regime is prone to overfitting and instability. While GNNs leverage structural priors that could theoretically benefit this setting, recent studies suggest they often fail to outperform simpler machine learning baselines in omics tasks. Furthermore, the field lacks a standardized pipeline; performance gains are often confounded by heterogeneous upstream preprocessing (e.g., probe-to-gene aggregation, normalization, covariate adjustment), making it difficult to isolate whether improvements stem from architectural innovation or data processing artifacts.

Methodology: The OgBench Framework

To address these gaps, the authors introduce OgBench, the first benchmarking platform explicitly designed for graph-level prediction in the $n \ll p$ omics regime. OgBench provides a modular, end-to-end infrastructure that decouples data preprocessing, graph construction, and model training.

1. Standardized Datasets

The framework integrates four cleaned, model-ready datasets spanning three omics modalities:

HERITAGE (Proteomics): Binary classification of exercise responders ( $n=654$ , $p=4977$ proteins).
Parkinson's (Transcriptomics): Binary classification of cognitive status based on MoCA scores ( $n=535$ , $p=21,755$ genes).
AddNeuroMed (Transcriptomics): Three-class clinical diagnosis (AD, MCI, Control) ( $n=711$ , $p=17,198$ genes).
BRCA (Epigenomics): Four-class breast cancer subtype classification ( $n=640$ , $p=19,049$ methylation regions mapped to genes).

All datasets undergo a consistent preprocessing pipeline including probe-to-gene aggregation, normalization, and covariate adjustment, with data splits fixed at 70/15/15 (train/val/test).

2. Modular Graph Construction

OgBench treats node selection and edge construction as interchangeable modules to allow controlled ablation studies:

Node Selection: Strategies include Variance filtering, Pearson Correlation, Distance Correlation (for non-linear dependencies), and Random subsampling. This allows researchers to study the impact of feature selection priors on model performance.
Edge Construction: Two primary strategies are supported:
- Data-driven: Co-expression graphs built via pairwise Pearson correlation with soft-thresholding to approximate scale-free topology.
- Biological Prior: Protein-Protein Interaction (PPI) networks derived from the STRING database (medium-confidence interactions).
- Crucially, the adjacency matrix is fixed across all samples (global topology), while node features (expression values) remain sample-specific.

3. Model Suite and Baselines

The framework benchmarks a diverse array of models against classical baselines:

Baselines: Support Vector Machines (SVM) and Elastic Net (regularized linear models), which are often competitive in high-dimensional settings.
Deep Learning Baseline: Multi-Layer Perceptrons (MLPs) to isolate the contribution of graph topology from non-linear feature transformation.
GNNs:
- Classical: GCN, GraphSAGE, GATv2, GIN.
- Scalable: SAGN (designed for large graphs).
- Omics-specific: ChebNet (spectral convolutions), MLA-GNN (multi-level attention).
- Transformers: GPS (General Graph Transformer).

4. Evaluation Protocol

Experiments involve rigorous hyperparameter optimization (grid search over learning rates, layer counts, etc.) and the use of Top-K ensemble aggregation to mitigate the instability of validation-based selection in small-sample regimes. Metrics include Macro F1, Weighted F1, Accuracy, and AUROC, alongside computational efficiency measures.

Key Results

The experimental evaluation across the four datasets yields several critical insights:

GNNs Do Not Consistently Outperform Simple Baselines: In the $n \ll p$ regime, widely used GNNs (GCN, GAT, GIN) often fail to convincingly outperform simple MLPs or classical regularized linear models (SVM, Elastic Net). On datasets like Parkinson's and BRCA, linear baselines remain competitive or superior.
Task-Dependent Utility of Structure: The value of graph structure is highly dataset-dependent. On the HERITAGE dataset, graph-based methods showed advantages, whereas on others, they did not. This suggests that structural inductive biases are not universally beneficial in omics.
Readout Mechanisms Matter: In datasets where GNNs underperform baselines (Parkinson's, BRCA), using an MLP readout (learning weights on node embeddings without message passing) significantly outperforms vanilla pooling (mean/sum aggregation). This implies that global feature integration drives performance rather than local message passing, indicating that the graph topology adds limited signal beyond the node features themselves in these cases.
Node Selection is Critical: The choice of node selection strategy interacts complexly with model performance. Interestingly, on HERITAGE, random node selection sometimes yielded the best graph-based performance, suggesting that preserving network connectivity (even with less predictive nodes) is crucial for message-passing mechanisms, whereas correlation-based selection (which picks individually predictive nodes) may disrupt the graph structure.
Edge Source Sensitivity: No single edge source (PPI vs. Co-expression) consistently outperforms the other. The optimal choice depends on the specific GNN backbone and dataset, suggesting that the interaction between architecture and topology is more significant than the edge source itself.
Validation Instability: In the $n \ll p$ regime, selecting a single best model based on a small validation set is unreliable. Ensemble aggregation (averaging predictions from top-K configurations) reduces variance and reveals true model advantages that single-config selection often obscures.

Significance and Claims

The paper claims that OgBench provides a necessary "reality check" for the application of GNNs in biomedicine. By exposing that standard GNNs often struggle to outperform simpler methods in data-scarce, high-dimensional omics settings, the framework challenges the prevailing assumption that graph structure inherently adds value.

The authors argue that the field must move beyond incremental performance gains on standard benchmarks and engage in a critical reassessment of learning paradigms. The results suggest that blindly applying graph methods to biological data without questioning the validity of the underlying structure or the suitability of the architecture for the $n \ll p$ regime may lead to stagnation. OgBench aims to foster the development of novel architectures explicitly tailored for biological graphs and to provide the community with a reproducible, open-source ecosystem to validate such innovations. The code and datasets are released to facilitate this transition.