💻 bioinformatics

RKMR: A Rapid Kernel Machine Regression Framework for Optimal Marker Detection in Spatial Omics Data

RKMR is a scalable, uncertainty-aware framework that integrates nonlinear kernel modeling, spike-and-slab variable selection, and spatial dependence to identify robust, predictive marker panels from high-dimensional spatial omics data, outperforming existing methods in accuracy and interpretability.

Original authors: Seal, S., Chakraborty, A., Mattila, C., Rubinstein, M., Angel, P., Ghosh, D., Chung, D., Neelon, B.

Published 2026-08-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seal, S., Chakraborty, A., Mattila, C., Rubinstein, M., Angel, P., Ghosh, D., Chung, D., Neelon, B.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a detective trying to solve a mystery inside a bustling, three-dimensional city. This city isn't made of brick and mortar, but of living cells, each with its own unique personality and job. In the past, scientists could only take a picture of the whole city and count the people, or maybe zoom in on a single person and ask them what they were doing. But now, we have super-powered microscopes that let us see the entire city at once, mapping out exactly where every single cell is standing and what chemicals they are shouting. This is the world of "spatial omics."

The big challenge in this world is finding the "ID badges" for different groups of people. Just like a detective needs a short list of reliable clues to tell a police officer from a baker, biologists need a short, punchy list of genes (the cell's instruction manual) to tell one type of cell from another. The problem is, the city is noisy. Cells that are neighbors often whisper the same secrets, and some cells are very good at hiding their true identity. Old methods of finding these ID badges were like asking every single person in the city one by one, "Are you a baker?" It's slow, it misses the fact that neighbors talk to each other, and it often picks up the wrong clues because it doesn't look at the whole picture at once.

This is where a new tool called RKMR (Rapid Kernel Machine Regression) comes in. Think of RKMR as a super-smart, high-speed detective who doesn't just ask questions one by one. Instead, it looks at the entire city map simultaneously. It uses a special kind of math that understands two things at once: the complex, non-linear relationships between genes (like how a baker might also be a musician, but only when the weather is rainy) and the fact that neighbors influence each other (spatial dependence).

The researchers behind RKMR built a framework that acts like a sieve with a very specific filter. They used a "spike-and-slab" approach, which is a fancy way of saying they have a mechanism to instantly shrink the importance of useless clues down to zero (the spike) while keeping the important ones strong (the slab). They also added a "spatial" layer to their math, acknowledging that cells standing next to each other are likely to be the same type, just like houses on the same street often look similar.

To make sure this detective tool was fast enough for the massive cities of modern biology (which can have hundreds of thousands of cells), they invented a way to speed up the calculations using "low-rank approximations." Imagine trying to count every grain of sand on a beach; instead, you count a few handfuls and use a smart formula to estimate the rest without losing accuracy. This allowed RKMR to process huge datasets in a fraction of the time it would take older methods.

When the team tested RKMR, they first created fake cities in a computer simulation. They planted specific "treasure" genes and then tried to see if different detective methods could find them. In these simulations, RKMR consistently found the correct genes more often than other popular methods like Random Forest or XGBoost, especially when the city was noisy or the clues were tricky. It didn't just guess; it gave a confidence score (called a Posterior Inclusion Probability, or PIP) for every gene, telling the user exactly how sure it was that a gene was a real ID badge.

The team then took RKMR out into the real world, testing it on actual data from human pancreatic cells, mouse brain cells, and even the regenerating brain of an axolotl (a type of salamander). In every case, RKMR successfully identified the famous, well-known ID badges that scientists had already discovered, but it did so by creating a much shorter, cleaner list of genes. For example, in the mouse brain data, it found the right markers for rare cell types that only made up 1% of the total population. When applied to human brain tissue, it found consistent markers across different sections of the brain, proving it could handle the messy, real-world noise of biological data.

The paper concludes that RKMR is a powerful, scalable, and uncertainty-aware framework. It doesn't just find markers; it finds the right markers with a clear measure of confidence, all while respecting the fact that cells live in a neighborhood where location matters. It's a new way to turn the chaotic, high-dimensional noise of spatial biology into a clear, actionable list of clues for future medical discoveries.

Technical Summary: RKMR – A Rapid Kernel Machine Regression Framework for Optimal Marker Detection in Spatial Omics Data

Problem Statement

High-throughput spatial omics technologies enable molecular profiling within intact tissue architectures, yet identifying concise, predictive, and biologically interpretable marker panels for cell types, tissue domains, and disease classes remains a significant challenge. Current workflows largely rely on univariate differential-expression (DE) analyses, which ignore joint molecular structure, redundancy, and spatial dependence, offering limited predictive insight. While multivariate machine-learning methods (e.g., Random Forest, XGBoost, Elastic Net) and specialized single-cell panel-selection tools (e.g., COMET, scGeneFit, COSG) exist, they often lack explicit spatial modeling, probabilistic feature selection, and adaptive criteria for panel size. Furthermore, many existing methods were developed for single-cell RNA sequencing (scRNA-seq) data and may not scale or perform optimally when applied to spatial transcriptomics (ST) and spatial proteomics (SP), where spatial autocorrelation is a critical feature. There is a need for a scalable framework that integrates nonlinear modeling, spatial dependence, and uncertainty quantification to translate high-dimensional spatial data into robust, actionable marker panels.

Methodology

The authors propose Rapid Kernel Machine Regression (RKMR), a scalable framework that extends Bayesian Logistic Kernel Machine Regression (BLKMR) to handle spatial data and high-dimensional feature selection.

1. Model Formulation:
RKMR employs a one-versus-rest classification framework where the outcome $y_i$ (binary label for a target cell type or tissue class) is modeled using a logit link:
$\eta_i = \beta_0 + x_i^\top \beta_1 + h_1(z_i) + h_2(s_i)$

Linear Covariates ( $x_i$ ): Captures adjustment factors (e.g., batch effects).
Nonlinear Molecular Effects ( $h_1(z_i)$ ): Modeled as a Gaussian Process (GP) with an Automatic Relevance Determination (ARD) kernel. The ARD kernel allows feature-specific relevance parameters ( $r_m$ ) to control the contribution of each molecular feature (e.g., gene expression) to the kernel similarity. This enables the model to capture nonlinear relationships and feature interactions.
Spatial Dependence ( $h_2(s_i)$ ): Modeled as a latent spatial GP over location coordinates $s_i$ , capturing residual spatial autocorrelation inherent in tissue architecture.

2. Sparsity and Variable Selection:
To identify relevant markers, RKMR places a continuous spike-and-slab prior on the ARD relevance parameters ( $r_m$ ).

The "spike" component shrinks uninformative features toward zero.
The "slab" component allows informative features to have larger effects.
This formulation yields Posterior Inclusion Probabilities (PIPs) for each feature, quantifying the model-based uncertainty regarding feature inclusion.

3. Scalable Inference Algorithm:
Direct Bayesian inference via Markov Chain Monte Carlo (MCMC) is computationally prohibitive for large spatial datasets ( $N \approx 10^5$ , $p \approx 10^4$ ). RKMR addresses this by combining:

Penalized Quasi-Likelihood (PQL) and Iteratively Reweighted Least Squares (IRLS): To approximate the Bernoulli likelihood with a weighted Gaussian working model.
Variational Inference (VI): To approximate the posterior distributions of the ARD parameters and inclusion indicators.
Low-Rank Approximations: Utilizing Nyström kernel approximation to reduce the computational complexity of kernel matrix operations.
Stochastic Optimization: Using mini-batches for gradient updates to scale with sample size.

The algorithm iteratively updates the linear predictor, working response, and weights, optimizing the Evidence Lower Bound (ELBO) to estimate PIPs efficiently.

Key Contributions

Unified Framework: RKMR provides a flexible framework that jointly models nonlinear feature-outcome relationships, spatial dependence, and probabilistic feature selection for spatial omics data.
Probabilistic Selection: Unlike methods relying on user-specified panel sizes or deterministic importance scores, RKMR provides Posterior Inclusion Probabilities (PIPs), offering a principled, uncertainty-aware criterion for marker selection (e.g., selecting features with PIP > 0.5).
Scalability: By integrating Nyström approximations and stochastic variational inference, RKMR achieves near-linear scaling in both the number of observations ( $N$ ) and features ( $p$ ), making it applicable to modern large-scale spatial datasets.
Flexibility: The framework is applicable to both spatial (ST/SP) and non-spatial (scRNA-seq) data; for non-spatial data, the spatial kernel component is simply omitted.

Results

Simulation Studies:

RKMR was evaluated against general-purpose algorithms (Random Forest, XGBoost, Elastic Net) and specialized marker-selection tools (scGeneFit, gpsFISH, COSG, MAGNETO).
In both linear and nonlinear (ARD RBF kernel) simulation scenarios, RKMR consistently achieved the highest Area Under the Precision-Recall Curve (AUPRC), with improvements of up to 50% over competing methods in certain settings.
RKMR demonstrated robustness to varying levels of spatial autocorrelation and signal strength, outperforming specialized methods that often struggled with nonlinear effects or spatial noise.

Real-Data Applications:

Pancreatic Islet Cells (scRNA-seq): RKMR successfully recovered the canonical marker for every cell type (e.g., INS for Beta, GCG for Alpha) with PIP > 0.5, selecting exactly one gene per type in most cases.
Mouse Cortex (scRNA-seq): RKMR recovered reported markers for all cell types, including rare populations (e.g., Ependymal cells). In cases where the reported marker was highly correlated with another gene, RKMR selected the correlated alternative (e.g., Lyz2 for Microglia instead of Aif1), which is biologically plausible.
Axolotl Brain (Stereo-seq): RKMR identified reported top markers for 16 out of 17 cell types, often selecting sparse panels (1–4 genes) and correctly handling correlated alternatives.
Human DLPFC (Visium): Applied to 12 tissue sections, RKMR identified consistent, reproducible marker sets for cortical layers and white matter across sections. It recovered known layer-enriched markers (e.g., GFAP for Layer 1, PCP4 for Layer 5, CCK for Layer 6) despite noisy expression patterns and spatial complexity.

Significance and Claims

The paper positions RKMR as a scalable and uncertainty-aware framework for translating high-dimensional spatial omics data into robust, experimentally actionable marker panels. The authors claim that RKMR addresses critical gaps in current workflows by:

Moving beyond univariate DE to model joint predictive value and redundancy.
Explicitly accounting for spatial autocorrelation, which is often ignored in standard marker discovery.
Providing probabilistic selection criteria (PIPs) rather than arbitrary thresholds or user-defined panel sizes.
Offering a computationally efficient solution (via a Python implementation) that scales to the size of modern spatial datasets.

The authors acknowledge limitations, such as the challenge of multicollinearity where correlated features may be selected as proxies for known markers, and note that future work could explore structured priors or multinomial formulations for multi-class selection. However, they assert that RKMR currently establishes a practical standard for targeted marker-panel discovery in spatial biology.