Large-scale automated detection reveals pervasive sex imbalance in biomedical research

Technical Summary: Large-scale Automated Detection of Sex Imbalance in Biomedical Research

Problem Statement
Decades of biomedical research have relied disproportionately on male study subjects, creating significant gaps in the understanding of female-specific disease biology. While sex is a critical biological variable affecting disease risk, progression, and treatment response, existing efforts to quantify this imbalance have been limited by scale and methodology. Previous approaches relied on:

1. Explicitly reported sex counts, which are often incomplete or inconsistent (e.g., missing in ~50% of ClinicalTrials.gov records).
2. Repository-provided sex annotations, which are absent for the majority of publicly available samples.
3. Direct prediction from expression data using exact text matching, which fails to capture synonyms and related terminology in metadata.

Consequently, there is a lack of automated, large-scale frameworks capable of identifying research areas with sex imbalances across thousands of disease terms and millions of biological samples.

Methodology
The authors present a multimodal computational framework to infer biological sex and quantify research-derived sex imbalance at scale. The workflow consists of four primary stages:

1. Data Collection and Preprocessing:

   • The study analyzed 229,528 human transcriptomic samples (124,748 microarray and 104,780 RNA-Seq) from the Gene Expression Omnibus (GEO) and refine.bio.
   • A "gold-standard" subset of 30,339 samples with manually curated sex and age labels was used to train and validate the inference model.

2. Sex Inference from Expression Profiles:

   • Instead of relying on metadata, sex was inferred directly from gene expression data.
   • The authors identified nine genes with the highest balanced accuracy (>0.82) across 11 age groups for separating male and female samples. These genes are primarily located on the Y chromosome (e.g., RPS4Y1, KDM5D, EIF1AY) and one on the X chromosome (XIST).
   • Sex labels were assigned based on a majority vote of these nine genes against expression thresholds. The method achieved 98% accuracy for microarray and 95% for RNA-Seq samples in the gold-standard subset.
   • Sample quality was assessed to ensure low-quality samples (which might lead to misclassification due to low Y-chromosome expression) did not skew results; only a small fraction of samples were deemed "poor quality."

3. Disease Term Extraction and Mapping:

   • Disease terms were extracted from two metadata sources: ~9,000 study descriptions and ~5,000 publication abstracts.
   • Natural Language Processing (NLP) tools (spaCy with the "en_core_sci_md" model and PubTator3) were used to extract terms.
   • Extracted terms were mapped to the Mondo Disease Ontology using a custom Retrieval-Augmented Generation (RAG) pipeline combined with OpenAI API calls to handle synonyms, abbreviations, and free-text variations.

4. Quantifying Imbalance:

   • Research-Derived Imbalance: Calculated as the Spearman rank correlation between the TF-IDF score of each disease term and the female-to-male sample ratio within each study or publication.
   • Biological Sex Imbalance: To distinguish research bias from biologically justified differences (e.g., prostate cancer), the research-derived values were adjusted using global sex-specific disease prevalence data from the Global Health Data Exchange (GHDx). This adjustment reduced the magnitude of imbalance for diseases with known sex-specific prevalence, isolating areas with unjustified underrepresentation.
   • Clustering: Disease terms were embedded and clustered using node2vec and the Louvain algorithm to identify groups of diseases with similar imbalance patterns.

Key Results

• Validation: The sex inference method proved robust, with 97% of sample labels correctly inferred in the gold-standard set. The full compendium showed a bimodal distribution of inferred sex, with high confidence in the majority of samples.
• Prevalence of Male Skew: Across all metadata corpora, terms skewed toward male representation outnumbered those skewed toward female representation.
  • In publication metadata, 54–61% of the top 100 terms with the highest imbalance magnitude were male-skewed without biological justification.
  • After adjusting for global disease prevalence, up to 58% of disease terms in RNA-Seq associated publications and up to 57% in microarray-associated publications showed a male-leaning association.
• Specific Understudied Areas:
  • Male-Skewed (Biologically Unjustified): Diseases such as glioblastoma, cirrhosis, idiopathic pulmonary fibrosis, schizophrenia, myocardial infarction, and hepatocellular carcinoma emerged as critically understudied in females despite affecting both sexes comparably.
  • Female-Skewed (Biologically Unjustified): Terms including pancreatic cancer, colorectal cancer, and uveal melanoma showed female overrepresentation.
• Clustering: The analysis revealed that sex imbalances are not random but concentrated in specific biological domains. Clusters with male skew included heart diseases, liver diseases, and mental health disorders, while female-skewed clusters included breast/ovarian cancers and autoimmune disorders.

Significance and Claims
The paper claims to provide a principled, data-driven basis for prioritizing compensatory research efforts. By shifting the assessment of sex imbalance from manual monitoring of a few journals to a scalable, repeatable computational audit of hundreds of thousands of samples, the framework offers a reusable tool for ongoing monitoring of sex representation in biomedical literature.

The authors emphasize that their findings highlight specific disease areas where a lack of rigorous research using female models has serious public health consequences, particularly where sex plays a critical role in treatment response (e.g., estrogen's protective effect in glioblastoma or differential side effects in schizophrenia). The framework enables researchers, funders, and journals to identify where compensatory studies are most urgently needed and to track whether investments are closing the gap over time.

Limitations Acknowledged by the Authors

• Inference Bias: The method inferred a slightly higher proportion of female samples in the full compendium compared to the gold standard. While poor sample quality was ruled out as the primary cause, the authors note that cell lines losing Y-chromosomes or a genuine excess of female-derived samples in public repositories could contribute.
• Term Extraction: The NLP pipeline extracted non-disease terms (e.g., "adult," "damage") and acronyms that required manual correction or AI-assisted mapping, which could introduce minor errors in ontology mapping.
• Monotonicity Assumption: The correlation approach assumes a monotonic relationship between disease term relevance and sex ratio; if a disease term appears in both male- and female-dominated studies for different biological reasons, the correlation might underestimate the true imbalance.