Stable but confounded: brain region, not diagnosis, drives a validation-passing molecular subtype in postmortem psychiatric brain

Technical Summary: "Stable but confounded: brain region, not diagnosis, drives a validation-passing molecular subtype in postmortem psychiatric brain"

Problem Statement
Unsupervised molecular subtyping of brain transcriptomes is increasingly used to resolve the heterogeneity of psychiatric diagnoses (e.g., autism, schizophrenia). However, clustering algorithms will always return a partition, regardless of whether that partition reflects reproducible biology or merely technical/anatomical artifacts. The paper identifies three acute hazards in postmortem psychiatric studies:

1. Small sample sizes (tens to low hundreds) which limit statistical power.
2. Subject overlap between nominally independent cohorts, leading to false replication claims.
3. Anatomical dominance, where transcriptomic variation by brain region vastly exceeds variation by diagnosis. Consequently, clustering pooled multi-region data often recovers anatomy rather than disease subtypes.

The authors argue that current validation practices often fail to distinguish between a "stable" cluster (reproducible structure) and a "valid" cluster (biologically relevant to the disease), particularly when confounders like brain region are not explicitly tested.

Methodology
The authors re-analyzed three public postmortem cohorts (384 samples total) using a "validation-first" discipline that assembles established controls and adds explicit confound testing. The datasets included:

• Autism Spectrum Disorder (ASD): GSE28521 (microarray, frontal/temporal cortex/cerebellum) and GSE64018 (RNA-seq, superior temporal gyrus).
• Schizophrenia, Bipolar Disorder, Major Depression: GSE80655 (RNA-seq, anterior cingulate, DLPFC, nucleus accumbens).

Analytical Pipeline:

1. Feature Reduction: Gene expression was summarized into pathway-level scores using single-sample enrichment (ssGSEA or mean-of-per-gene-z-scores).
2. Clustering: Samples were clustered using Gaussian Mixture Modeling (GMM) with component selection via Bayesian Information Criterion (BIC).
3. Validation Controls: Every candidate partition was subjected to four gates:
   • Label-permutation null: Testing if clusters are recoverable from permuted labels.
   • Random-gene-set null: Testing if the partition is specific to curated pathways or arises from random gene sets.
   • Bootstrap stability: Measuring the Adjusted Rand Index (ARI) between original and resampled assignments (threshold $\ge$ 0.80).
   • Ancestry independence: Checking for association with ancestry principal components.
4. Confound Testing: Explicit contingency-table association ($\chi^2$, Cramér's V) was performed between partitions and brain region, donor overlap, and clinical diagnosis.
5. Non-Circular Analysis: To avoid "double dipping," disease signals were assessed via diagnosis-based case-control contrasts within regions, using a fixed panel of canonical cell-type marker genes (e.g., PVALB, SST, GAD1) rather than features derived from the clustering itself.

Key Results

1. Failure of the Autism "GABAergic" Subtype:
   A putative ASD subgroup characterized by GABAergic pathway downregulation failed bootstrap stability validation (ARI = 0.71, below the 0.80 threshold). Furthermore, apparent replication in a second cohort (GSE64018) was invalidated because the two cohorts shared 10 of 24 subjects, meaning the agreement was cross-platform corroboration on the same brains, not independent replication.

2. The "Validation-Passing" Schizophrenia Partition is a Region Classifier:
   In the schizophrenia cohort (GSE80655), a $k=3$ partition passed all statistical gates (Bootstrap ARI = 0.923; passed random-gene-set and ancestry checks). However, confound testing revealed this partition was driven entirely by brain region, not diagnosis:

   • Subtype 0 and 2 were predominantly nucleus accumbens (nAcc).
   • Subtype 1 was exclusively cortical (AnCg and DLPFC).
   • The association between subtype and region was overwhelming ($p = 4.3 \times 10^{-26}$, Cramér's V = 0.67), while the association with diagnosis was non-significant ($p = 0.41$).
   • Restricting the analysis to a single region (DLPFC) caused the stability to drop below the validation threshold (ARI = 0.68), confirming the "stability" was an artifact of the region axis.

3. Systematic Nature of the Artifact:
   This region-driven artifact was systematic across all diagnoses in the cohort (Bipolar, Major Depression) and invariant to the number of clusters ($k$). The pooled four-diagnosis cohort showed the strongest region structure but the weakest stability.

4. Recovery of Known Biology via Non-Circular Analysis:
   While the clustering framework missed disease signals, a conventional, non-circular case-control analysis recovered established interneuron biology:

   • ASD: Significant depletion of parvalbumin (PVALB) markers in the cortex.
   • Schizophrenia: Significant depletion of somatostatin (SST) and GABA-synthetic enzymes (GAD1/GAD2) in cortical regions.
   • Double Dissociation: A significant, oppositely-signed interaction confirmed that PVALB depletion is more pronounced in ASD, while SST depletion is more pronounced in schizophrenia.

5. Benchmark Failure:
   A companion calibration benchmark (an adaptive-threshold model claiming $R^2 = 0.889$) failed to reproduce. The authors traced this to a metric defect: the Adjusted Rand Index (ARI) returned a spurious value of 1.0 on degenerate inputs (datasets with a single true cluster), artificially inflating the correlation.

Significance and Claims
The paper does not claim to discover new molecular subtypes; rather, it offers a "worked case study" demonstrating how a subtype claim can pass rigorous statistical stability checks yet be entirely driven by a known confound (brain region).

The authors conclude that:

• Stability $\neq$ Validity: High bootstrap stability certifies reproducibility but does not ensure the partition reflects the intended biological signal (disease vs. anatomy).
• Confound Testing is Mandatory: Testing partitions against brain region (and other confounds) must be a routine component of the validation battery for any subtyping analysis that pools regions or cohorts.
• Discipline over Discovery: In small, pooled, multi-region cohorts, exploratory discovery is prone to failure. The authors advocate for a specific checklist: reporting null controls, testing against confounds, ensuring subject-disjoint replication, avoiding circular inference, and publishing partitions that fail controls.

The study emphasizes that while molecular subtyping can succeed (citing Bowen et al., 2019, as an example of region-controlled success), the specific failure modes documented here—driven by anatomical dominance and unexamined pipeline defaults—are critical pitfalls in current psychiatric genomics.