Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets

Technical Summary: Mental Health Disorder Detection Beyond Social Media: A Systematic Review of Available Datasets

Problem Statement
The timely detection of mental health disorders is a critical societal challenge. While Natural Language Processing (NLP) and Machine Learning (ML) have advanced the identification of mental health conditions, current research relies predominantly on data scraped from social media platforms (e.g., Twitter, Reddit, Facebook). The authors identify significant limitations in this reliance: social media datasets often suffer from sampling biases (reflecting only active, willing users), lack demographic representativeness, and raise substantial ethical and privacy concerns regarding consent and algorithmic misuse. Conversely, non-social media sources, such as Electronic Health Records (EHRs), clinical interview transcripts, and standardized questionnaire responses, offer rich, clinically documented insights. However, these resources remain underexplored due to privacy constraints, data access challenges, and annotation complexities, creating a barrier to developing robust, generalizable NLP models for clinical integration.

Methodology
This study presents the first comprehensive systematic review of non-social media, free-text datasets for mental health research. Adhering to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, the authors conducted a literature search through June 2025 using databases including Google Scholar and PubMed.

The search strategy involved:

1. Keyword Selection: Terms related to specific disorders (depression, anxiety, suicidal ideation, suicide, mental disorder, crisis) combined with detection/analysis terms.
2. Exclusion Criteria: Papers explicitly referencing social media platforms or scraped data were excluded to isolate clinical and non-social media sources.
3. Screening Process: A multi-phase screening removed duplicate studies, non-textual data (audio/visual only), and studies based solely on pre-existing datasets without new analysis.
4. Backtracking: The authors traced citations in included studies to identify original dataset publications that might not have matched initial search keywords.
5. Final Corpus: The process yielded 45 relevant papers proposing or analyzing newly collected free-text based mental health datasets.

Key Contributions
The paper provides a structured taxonomy and analysis of 45 non-social media free-text datasets, addressing three primary research questions:

• RQ1 (Availability & Variation): Cataloging datasets by source, structure, and population.
• RQ2 (Labeling & Clinical Implications): Analyzing how conditions are defined and labeled.
• RQ3 (Adoption Factors): Identifying drivers of dataset popularity and research adoption.

The review details the landscape of these resources across languages, disorders, platforms, data types, and availability status.

Results
The analysis reveals several critical trends and gaps in the current landscape:

• Linguistic Imbalance: English (EN) dominates the field, comprising 62.2% of datasets. Chinese (ZH) follows at 17.8%, with other languages (Korean, Polish, Japanese, Arabic, Thai, Greek) and code-mixed datasets representing the remainder. Notably, while Chinese datasets are more frequently publicly available, English datasets often require Data Use Agreements (DUA) or are restricted.
• Disorder Concentration: Depression (including Major, Minor, and Postnatal) is the most studied condition, appearing in 33 datasets. Anxiety and Suicidal Ideation/Suicide follow, while disorders such as PTSD, Schizophrenia, Bipolar Disorder, and Dementia are significantly underrepresented.
• Data Sources and Types: Clinical datasets primarily consist of interview transcripts (48%) and Electronic Health Records/Discharge Summaries (39%). Forum-based posts and mixed-source datasets (including apps and virtual agents) make up the rest.
• Annotation and Labeling: Labeling strategies vary widely:
  • Clinical Diagnostic Tools: The "gold standard" includes DSM-IV/V and ICD-9/10 codes, often used in conjunction with structured interviews.
  • Screening Questionnaires: Tools like PHQ-9, GAD-7, BDI, and HAMD are frequently used for self-reported or clinician-assessed labels.
  • Manual/Contextual: Some datasets rely on manual annotation by trained annotators or clinicians, sometimes using inter-rater agreement metrics (e.g., Cohen's Kappa), while others use keyword searches or NLP-assisted labeling.
• Demographics: Most datasets report gender and age, but attributes like income, race, and ethnicity are less common. Some studies actively balance control groups to match demographic profiles, a practice less common in social media research.
• Modeling Trends: Computational approaches have evolved from traditional ML (SVM, Logistic Regression) and feature engineering (LIWC, tf-idf) to Deep Learning (CNN, LSTM, GRU) and, more recently, Transformer-based models (Clinical-BigBird, MentalBERT) and Large Language Models (LLMs).

Significance and Claims
The authors position this work as the first systematic review of its kind, aiming to guide future efforts in dataset development and application. The paper claims that the current scarcity of diverse, reliable, and clinically relevant non-social media datasets hinders the advancement of robust NLP models for clinical practice.

Key implications highlighted include:

• Need for Diversity: There is an urgent need to expand dataset creation to include low-resource languages and a broader spectrum of mental health disorders beyond depression.
• Standardization: The lack of consistent reporting on why specific labeling tools were chosen suggests a need for better reporting standards. The authors suggest that advances in NLP, particularly instruction-tuned LLMs and techniques like Chain-of-Thought prompting, could help standardize labeling based on formal diagnostic criteria (DSM/ICD) to improve consistency and interpretability.
• Accessibility: While privacy concerns are valid, the authors argue that releasing more public or agreement-governed clinical datasets is essential to expand research opportunities and improve reproducibility.

The paper concludes modestly, acknowledging limitations such as the potential for missed studies due to English-only search queries and the exclusion of certain clinical databases (e.g., CINAHL). It emphasizes that while the review identifies gaps, the ultimate goal is to facilitate the development of more inclusive and clinically effective resources for mental health detection.