Reading Between the Signs: Predicting Future Suicidal Ideation from Adolescent Social Media Texts

Technical Summary: Predicting Future Suicidal Ideation from Adolescent Social Media Texts

Problem Statement
Suicide remains a leading cause of death among adolescents, yet prediction methods have stagnated over the past 50 years. Traditional clinical approaches rely on "risk factors" (e.g., depression, substance use) which often perform only slightly better than chance and suffer from low specificity. Furthermore, a significant portion of at-risk individuals never seek professional help, leaving them undetected by traditional healthcare systems. While social media offers a rich source of real-time self-disclosure, existing Natural Language Processing (NLP) methods typically focus on detecting suicidal ideation and behavior (SIB) after a user has explicitly expressed it. This limits their utility for early intervention. The authors propose a novel challenge: predicting whether an adolescent will future express SIB based solely on their prior social media interaction history, without using any SIB-related posts as input.

Methodology
The authors introduce EARLY-SIB, a transformer-based model designed to process sequential user interactions to predict future SIB. The methodology involves a three-stage pipeline:

1. Dataset Construction:

   • Source: Data was scraped from De Kindertelefoon, a Dutch forum for adolescents (12–18 years) focused on mental health.
   • Post-Level Labeling: A small dataset of posts was manually annotated to distinguish between SIB and non-SIB content. Annotators used C-SSRS guidelines, achieving perfect agreement ($\kappa=1.00$) after resolution. The dataset included "hard" negative instances (posts with suicide-related language but not actual SIB).
   • User-Level Dataset: A classifier (fine-tuned LLaMA-3-8B) was applied to the entire forum to label posts. A user-level dataset was then constructed where the input consists of a user's interaction history (posts and replies) preceding their first SIB post (if any). The target label is binary: whether the user will eventually post SIB.

2. EARLY-SIB Architecture:

   • Input: A sequence of $N$ interactions (posts or replies) for a user.
   • Body Transformer: Each interaction's text (title + body for posts; reply text for replies) is processed by a pre-trained Dutch BERT model (BERTje) to obtain CLS representations.
   • Sequential Modeling: These representations are passed through a bidirectional LSTM followed by an attention layer to capture sequential dependencies and weigh informative interactions.
   • Title+Tag Transformer: Titles and tags are concatenated and processed by a separate BERTje instance, as they contain distinct predictive signals.
   • Fusion: The mean-pooled LSTM output and the Title+Tag CLS embedding are concatenated and fed into a fully connected layer for binary classification.

3. Training Strategy:

   • Due to severe class imbalance (<4% positive cases), the majority class was down-sampled to a 50-50 distribution in the training set, which yielded better performance than weighted loss or other resampling ratios.
   • Input length was capped at $N=30$ interactions, prioritizing posts over replies.

Key Results

• SIB Detection: The model used to generate user labels (fine-tuned LLaMA-3-8B) achieved an F1-score of 0.96 in detecting SIB posts, significantly outperforming zero-shot prompting baselines.
• Early Prediction: On the user-level task, EARLY-SIB achieved a balanced accuracy of 0.73 and a recall of 0.71.
  • This significantly outperformed the baseline architecture by Zhang et al. (2024) (0.67 balanced accuracy) and zero-shot LLaMA-3 baselines (which failed to predict future SIB effectively, with balanced accuracies around 0.55–0.57).
  • Precision was low (0.10), which the authors note is expected given the extreme class imbalance but acceptable for a life-critical application where missing a positive case (false negative) is costlier than a false alarm.
• Ablation & Analysis:
  • Removing the LSTM or the Body Transformer resulted in performance drops, confirming the necessity of sequential modeling and body text analysis.
  • The Title+Tag Transformer performed well independently, suggesting metadata is highly informative.
  • Performance improved as the context window ($N$) increased, plateauing around $N=15$.
• Explainability: Using SHAP (Shapley Additive Explanations), the authors demonstrated that predictions rely on a diverse set of interactions (mean complexity 0.90) rather than a single dominant post. The most predictive interactions often occurred within 10 days of the eventual SIB disclosure, with 23% occurring less than one day prior.

Key Contributions

1. Novel Task Definition: The paper defines and addresses the task of early SIB prediction, moving the timeline one step earlier than existing literature by predicting SIB before any explicit expression occurs.
2. EARLY-SIB Model: A specialized architecture combining body text and metadata (titles/tags) with sequential modeling (LSTM) to handle long user histories and capture temporal dependencies.
3. Empirical Validation: Demonstration that social media data contains detectable signals of future risk, achieving a balanced accuracy of 0.73, which the authors argue is a meaningful addition to traditional methods.
4. Interpretability: Application of SHAP to reveal that risk prediction is a complex interplay of multiple interactions, not just isolated "red flag" posts.

Significance and Claims
The authors claim that their work demonstrates the feasibility of using supervised machine learning on social media to identify at-risk adolescents before they explicitly disclose suicidal thoughts. They position EARLY-SIB not as a replacement for clinical assessment, but as a complementary tool to support human moderators by providing early warning signals and explainable rationales.

The paper maintains a modest tone regarding its claims:

• It acknowledges that the ground truth is based on self-disclosure, not verified clinical outcomes.
• It notes that the model assumes temporal coherence in user language, which may not capture non-linear external factors.
• It highlights ethical trade-offs, particularly regarding privacy and the potential for false positives (stigma) versus false negatives (missed interventions).
• It concludes that while the technical challenge is feasible, real-world deployment depends on institutional capacity, policy, and the availability of effective intervention resources.

The authors explicitly state that their results provide evidence that this challenge is feasible, but they do not claim to have solved the broader problem of suicide prevention. They call for future work on generalizability across languages and platforms, and for further research into interpretability and dataset construction.