🧬 biology

A Robust Nonparametric Framework for Detecting Repeated Spatial Patterns

This paper proposes a robust nonparametric framework that combines constrained clustering with a post-clustering reassessment step based on the maximum mean discrepancy (MMD) statistic to effectively identify repeated spatial patterns characterized by similar distributions across non-contiguous regions.

Original authors: Rajitha Senanayake, Pratheepa Jeganathan

Published 2026-04-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rajitha Senanayake, Pratheepa Jeganathan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a giant, sprawling city. Your job is to group people into neighborhoods based on what they have in common.

The Problem: The "Look-Alike" Mystery
Usually, when we look for neighborhoods, we assume that people who live next door to each other are similar. If you see a row of houses with red doors, you assume they are all one group. This is how most computer programs work: they look for spatially contiguous clusters (things that are right next to each other).

But here's the twist: In this city, there are "twin neighborhoods." Imagine a quiet, red-door neighborhood in the North, and an exact copy of it in the South, miles away. The people in the South have the same jobs, hobbies, and lifestyles as the North, but they are separated by a massive park.

Old detective tools (standard clustering methods) would fail here. They would say, "The North is one group, and the South is a totally different group because they aren't touching." They miss the fact that these two distant places are actually Repeated Spatial Patterns (RSP). They are the same story, just told in two different locations.

The Solution: The "repSpat" Detective
The authors of this paper built a new tool called repSpat. Think of it as a two-step detective process:

Step 1: The Rough Draft (Constrained Clustering)

First, the tool does the easy job. It looks at the map and groups people who are physically close to each other. It's like drawing lines on a map to create initial neighborhoods.

The Flaw: Because it only looks at proximity, it might accidentally split that "Red Door Neighborhood" into two separate groups just because a river runs through it. It creates too many small, fragmented groups.

Step 2: The "Twin Detector" (The Magic Reassignment)

This is where the magic happens. The tool now takes those initial groups and asks a new question: "Do these two distant groups actually have the same 'vibe'?"

To answer this, it uses a mathematical ruler called MMD (Maximum Mean Discrepancy).

The Analogy: Imagine you have two bags of marbles. One bag is from the North neighborhood, the other from the South. You don't just count the marbles; you look at the distribution of colors, sizes, and weights.
If the bags look statistically identical, the tool says, "Aha! These are twins!"
If they look different, it says, "Nope, these are strangers."

The "Block Permutation" Trick
Here is the tricky part: In a city, neighbors influence each other. If you randomly swap a person from the North with someone from the South to test them, you might break the natural "flow" of the neighborhood (like swapping a quiet librarian with a loud rock star in the middle of a library).

To fix this, the tool uses Block Permutation.

The Analogy: Instead of swapping individual people, it swaps entire "blocks" of the neighborhood. It keeps the local neighborhood feel intact while testing if the two distant areas are truly the same. This ensures the test is fair and accurate.

The Result

Once the tool confirms that two distant groups are "twins," it reassigns their labels. It erases the boundary between them and says, "You are now one big, unified group, even though you are miles apart."

Why Does This Matter? (The Real-World Example)

The authors tested this on Triple-Negative Breast Cancer data.

The Scene: Imagine a tumor is a messy city. Inside, there are different "districts" of cells. Some districts are full of immune cells (the body's police), and others are full of cancer cells.
The Discovery: Sometimes, the "Cancer District" appears in three different spots inside the same tumor, separated by healthy tissue.
The Impact: Old tools would see three separate cancer spots. repSpat sees them as one repeating pattern. This helps doctors understand that the cancer is behaving the same way in multiple places, which is crucial for treatment.

Summary

Think of repSpat as a smart map-maker that doesn't just draw lines based on who is standing next to whom. It looks deeper, realizing that familiarity doesn't always mean proximity. It finds the "twins" in the crowd, groups them together, and gives us a clearer, more accurate picture of complex patterns in our world, from cancer cells to climate zones.

1. Problem Statement

In modern spatial statistics, particularly in fields like spatial omics (e.g., spatial proteomics and transcriptomics), a critical challenge is identifying Repeated Spatial Patterns (RSP).

The Limitation of Existing Methods: Traditional spatial clustering methods (e.g., constrained hierarchical clustering, scan statistics) enforce spatial contiguity, meaning they group only neighboring locations with similar attributes. While effective for contiguous regions, these methods fail to detect disjoint regions that share similar underlying statistical distributions.
The Gap: Many biological and environmental processes (e.g., tumor microenvironments, climate zones) manifest as similar patterns in geographically distant locations. Existing methods often over-segment these patterns into distinct clusters because they are spatially separated, missing the underlying biological or physical similarity.
Data Complexity: Spatial omics data is heterogeneous, containing continuous, categorical, and binary data types, often with high dimensionality and complex spatial dependencies. Parametric methods (e.g., Gaussian assumptions) often lack the flexibility to handle this diversity.

2. Methodology: The repSpat Framework

The authors propose repSpat, a two-stage, nonparametric framework designed to detect RSP by combining constrained clustering with a post-clustering reassessment. The framework consists of four main steps:

Step 1: Constrained Agglomerative Hierarchical Clustering (CAHC)

Goal: Generate an initial set of spatially contiguous clusters.
Mechanism: The method uses CAHC, which merges observations based on attribute dissimilarity (e.g., Euclidean distance for continuous data, Jaccard for binary) but strictly enforces a spatial contiguity constraint. Merges only occur if there is a spatial link (e.g., $m$ -nearest neighbors) between the clusters.
Parameter Selection: The neighborhood size ( $m$ ) and the number of clusters ( $G$ ) are tuned using a spatially-informed silhouette score, which evaluates cluster compactness and separation while respecting spatial adjacency.

Step 2: Testing Spatial Invariance (MMD²)

Goal: Identify if spatially disjoint clusters share the same underlying distribution.
Statistic: The framework employs the Maximum Mean Discrepancy squared ( $MMD^2$ ) statistic. This nonparametric metric measures the distance between the embeddings of two distributions in a Reproducing Kernel Hilbert Space (RKHS).
Kernel Choice: The authors utilize the Inverse Multiquadratic (IMQ) kernel, which they demonstrate provides superior sensitivity in high-dimensional settings compared to Gaussian or Laplacian kernels.
Hypothesis: For any pair of clusters $(g, h)$ , the null hypothesis $H_0: P(g) = P(h)$ is tested against the alternative that their distributions differ.

Step 3: Block Permutation for Null Distribution

Challenge: Standard permutation tests assume exchangeability, which is violated in spatial data due to local dependence (spatial autocorrelation). Randomly shuffling individual points destroys local attribute structures.
Solution: The authors propose a block permutation strategy.
1. Each cluster is partitioned into "blocks" based on attribute similarity (using $k$ -means or dendrogram cutting).
2. Entire blocks are permuted between the two clusters being compared.
3. This preserves the local spatial structure within the blocks while generating a valid null distribution for the $MMD^2$ statistic under spatial dependence.
Inference: P-values are calculated and adjusted for multiple comparisons using the Benjamini–Hochberg procedure to control the False Discovery Rate (FDR).

Step 4: Cluster Reassignment

Mechanism: A graph is constructed where nodes represent the initial CAHC clusters. An edge exists between two nodes if the hypothesis test fails to reject the null hypothesis (i.e., the distributions are statistically similar).
Outcome: Clusters forming cliques (fully connected subgraphs) are merged and assigned a common label. This effectively re-identifies spatially distant regions as a single "Repeated Spatial Pattern."

3. Key Contributions

Novel Framework (repSpat): A flexible, nonparametric pipeline that decouples spatial contiguity from distributional similarity, specifically designed to detect RSP.
Theoretical Consistency: The authors prove that the $MMD^2$ statistic is asymptotically consistent under model-free spatial mixing conditions, ensuring validity even with complex spatial dependencies.
Robust Resampling: The introduction of the block permutation strategy allows for valid statistical inference in spatial data where standard permutations fail, preserving local attribute structures.
Versatility: The framework is agnostic to data types (continuous, binary, categorical) by adapting the dissimilarity metric and kernel function, making it highly applicable to spatial omics.

4. Results

The framework was evaluated through extensive simulation studies and a real-world application:

Simulation Studies:
- Setup: Data was generated using Gaussian Markov Random Fields (GMRF) with varying sample sizes ( $n=2500, 4900$ ), dimensions ( $p=5, 10$ ), and spatial autocorrelation strengths ( $\eta=0.3, 0.8$ ).
- Performance: repSpat achieved a median Adjusted Rand Index (ARI) of 1.0, indicating perfect recovery of true clusters.
- Comparison: Compared against Banksy (a state-of-the-art spatial omics clustering tool), repSpat significantly outperformed it. Banksy achieved ARI values between 0.75 and 0.83, often misclassifying cluster boundaries due to its reliance on spatial smoothing without explicit distributional re-evaluation.
- Robustness: repSpat maintained high accuracy even with added noise features and under strong spatial autocorrelation.
Real Data Application (Triple-Negative Breast Cancer):
- Data: Spatial proteomics data (MIBI-TOF) from 39 patients, containing 36 protein markers.
- Findings: When applied to CAHC outputs, repSpat successfully identified repeated tumor microenvironments (TMEs).
- Insight: The method revealed that distinct spatial patches in different parts of a tissue sample shared similar protein expression profiles (enriched in mixed tumor cell populations), which were missed by standard contiguous clustering. The use of binary markers (thresholded intensities) combined with the IMQ kernel yielded the most distinct RSP detection.

5. Significance and Future Directions

Methodological Advancement: repSpat addresses a fundamental gap in spatial statistics by bridging the divide between spatial contiguity and distributional similarity. It offers a rigorous statistical solution for "over-segmentation" issues common in constrained clustering.
Biological Impact: In spatial omics, identifying RSP is crucial for understanding disease heterogeneity. The ability to find similar biological processes in disjoint tissue regions can lead to better biomarker discovery and a deeper understanding of tumor evolution.
Future Work: The authors suggest extending the framework to include:
- Integration with other constrained clustering algorithms.
- Variable selection to improve interpretability.
- Bayesian extensions for uncertainty quantification.
- Adaptation for lattice/areal data (e.g., spot-based transcriptomics).

In conclusion, repSpat provides a robust, flexible, and statistically rigorous tool for uncovering hidden repeated patterns in complex spatial datasets, with immediate applicability in the rapidly growing field of spatial biology.