psytechlab at CLPsych 2026: Utilising Natural Language Processing methods and Large Language Models for Social Media Text Analysis

Technical Summary: Psytechlab at CLPsych 2026

Problem Statement
The paper addresses the challenge of analyzing social media text to understand mental health states and user well-being. While social media offers a rich data source for identifying patterns related to depression, anxiety, and other disorders, existing approaches face significant limitations: reliance on potentially noisy self-disclosed labels, the difficulty of obtaining clinically reliable annotated data, and the tendency to assign static labels to inherently dynamic mental states. To address these issues, the authors participated in the CLPsych Shared Task 2026, which utilized a dataset professionally annotated using the MIND framework. This framework models human experience as a continually evolving series of self-states. The team was tasked with solving three specific challenges:

1. Task 1: Predicting dominant Adaptive and Maladaptive ABCD (Affect, Behavior, Cognition, Desire) subelements and estimating the presence value (1–5 scale) of self-states in posts.
2. Task 2: Identifying "Switch" (sudden changes in well-being) and "Escalation" (gradual intensification) events within chronological sequences of posts.
3. Task 3: Generating timeline-level summaries of self-state dynamics (Task 3.1) and identifying recurrent patterns of deterioration or improvement (Task 3.2).

Methodology
The Psytechlab team employed a hybrid approach combining traditional NLP pipelines, supervised learning, and Large Language Models (LLMs).

• Task 1 (ABCD Prediction & Presence Estimation):

  • Pipeline Approach (Submission 1): The team split posts into sentences and classified each into subelements or an "irrelevant" class. To address data scarcity, they augmented the dataset by translating existing data from a semantically aligned source (Buyanov et al., 2025) and generating synthetic examples using a local Qwen3.5-35B-A3B model. They trained classifiers using BERT and ModernBERT, achieving comparable performance (0.80 Macro F1 for ModernBERT).
  • Aggregation: Sentence-level predictions were aggregated using weights derived from the proportion of subelements in the source dataset. The dominant subelement determined the state and valence.
  • Presence Estimation: ModernBERT was used in regression mode on the full post text to estimate the presence value for each valence.
  • LLM Approach (Submission 2): The local Qwen3.5-35B-A3B model was prompted with subelement names to solve the classification task directly, while ModernBERT handled presence estimation.

• Task 2 (Switch and Escalation Detection):

  • Feature Extraction: Posts were split into sentences, and embeddings from MiniLM-L6-v2 were combined with probability vectors from sentiment and emotion models (BERTweet, EmoRoBERTa, twitter-roberta-emotion).
  • Classification: A BiLSTM model predicted "Switch" and "Escalation" labels separately. Hyperparameters were optimized using Optuna.
  • Temporal Features (Submission 2): An additional submission incorporated temporal features of the posts into the same architecture.
  • Alternative Models: The team experimented with Tempoformer and HoRoBERT but faced challenges in training Tempoformer within the competition timeframe.

• Task 3 (Timeline Summarization):

  • Task 3.1: The team utilized zero-shot and few-shot prompting with the local Qwen3.5-35B-A3B model. Prompts included the MIND framework description and intermediate summaries of individual posts derived from Task 1 predictions.
  • Task 3.2: For identifying recurrent patterns, the team used Llama-3.2-3B-Instruct. The LLM was instructed to detect well-being change directions, identify ABCD elements, and assign self-state labels. Statistical analysis of adaptive vs. maladaptive states across change types was used to generate summaries of example timelines.

Key Results
The team's system achieved results that outperformed almost all baselines, though it did not reach the top tier for all tasks.

• Task 1: Achieved an Average Subelement Macro F1 of 0.274 (Rank 12/17) and an Average RMSE of 1.407 for presence estimation (Rank 15/17). The authors attribute the lower ranking to the difficulty of classifying the "irrelevant" sentence class, which introduced noise during aggregation.
• Task 2: Achieved a Combined Macro F1 of 0.372 (Rank 15/18). The authors note that "Switch" detection performed significantly worse than "Escalation," hypothesizing that the "Switch" label relies on a hidden intermediate layer (Global Assessment of Functioning scores) that is difficult to infer directly from text.
• Task 3: The system achieved notable success, ranking 8th out of 13 for Task 3.1 (Score 7.3) and 5th/6th out of 9 for Task 3.2 (Improvement/Deterioration scores). The authors were surprised that a comparably small model with simple prompts achieved high scores on consistency and contradiction metrics.

Significance and Contributions
The paper claims that by developing and testing these mental health-state estimation systems, the team contributed to the broader goal of improving mental health support systems. The work demonstrates the viability of combining sentence-level classification with LLM-based summarization for analyzing dynamic mental states.

The authors explicitly highlight the following limitations and modest claims:

• Approximation, not Diagnosis: The models output approximations of annotated self-state patterns, not direct evidence of a person's clinical mental condition.
• Data Constraints: The use of translated and synthetically generated data may have introduced distributional mismatches.
• Privacy and Scope: The study was conducted in a restricted-access environment using only locally deployed LLMs to preserve privacy. The authors emphasize that their systems are strictly for research purposes and should not be used for clinical diagnosis or autonomous decision-making.
• Future Work: The analysis suggests that future improvements require better handling of "irrelevant" text classes, more robust temporal modeling for "Switch" detection, and careful integration of content data without compromising the structural accuracy of summaries.