💬 NLP

Team MKC at CLPsych 2026: Capturing and Characterizing Mental Health Changes through Social Media Timeline Dynamics

This paper presents Team MKC's LLM-based pipeline for the CLPsych 2026 shared task, which offers a unified framework to jointly perform post-level mental health assessment and user-level temporal modeling by analyzing sequentially ordered social media posts.

Original authors: Kyomin Hwang, Hyeonjin Kim, Hyunho Lee, Nojun Kwak

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kyomin Hwang, Hyeonjin Kim, Hyunho Lee, Nojun Kwak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand a person's emotional journey just by reading their diary entries over time. Some days they write about feeling great; other days, they sound worried or sad. Sometimes, there's a sudden shift from happy to sad, or a slow, creeping feeling of getting worse.

This paper is about a team of researchers (Team MKC) who built a digital detective system using advanced AI (called Large Language Models) to do exactly that. They entered a competition called CLPsych 2026, where the goal was to analyze social media posts to spot changes in mental health.

Here is how their system works, broken down into simple steps:

1. The "Microscope" (Step 1 & 2)

First, the AI looks at one single post at a time. Think of this like a microscope zooming in on a single sentence.

What it does: It tries to label the post with specific tags. Is the person being "adaptive" (coping well) or "maladaptive" (struggling)? It breaks these feelings down into 12 tiny categories (like "feeling hopeless" or "feeling hopeful").
The Challenge: The training data was very messy. Most posts were about common feelings, but very few were about rare, extreme feelings. It's like trying to learn to identify rare birds when you only have 500 photos of them, but 4,000 photos of pigeons.
The Fix: To stop the AI from ignoring the rare birds, the researchers taught it to pay extra attention to the rare examples. They also used a trick called "K-fold cross-validation," which is like having five different detectives look at the same clues and then voting on the answer to make sure they don't miss anything.

2. The "Scorecard" (Step 2)

After labeling the post, the AI gives it a score from 1 to 5.

What it does: It asks, "How strong is this feeling?"
The Result: It produces a number that represents how "healthy" or "unhealthy" that specific post sounds.

3. The "Traffic Light" (Step 3)

Now, the AI looks at two posts in a row to see what happened between them.

What it does: It tries to spot two specific things:
1. The Switch: A sudden, jarring change (like a red light turning green instantly).
2. The Escalation: A slow, creeping change (like a traffic light slowly turning from yellow to red).
The Result: It flags the exact moment a person's mood shifted significantly.

4. The "Storyteller" (Step 4)

Once the AI has analyzed the individual posts and the shifts, it acts like a summarizer.

What it does: It reads the whole timeline of posts and writes a short story or summary of what happened.
The Catch: The researchers found that when they tried to "teach" the AI to write these stories by showing it examples (fine-tuning), it actually did worse than just asking the AI to write the story from scratch using its existing knowledge. It's like how a student might memorize a specific essay so well they forget how to write a new one on a similar topic. The "zero-shot" (no examples) approach worked better here.

5. The "Pattern Finder" (Step 5)

Finally, the AI looks at many people's timelines to find common patterns.

What it does: It asks, "Do people who get better usually start by doing X? Do people who get worse usually start by doing Y?"
The Result: It tries to identify the "signature" of improvement or deterioration—recurring patterns that show up across different people.

The Big Challenges They Faced

Too Little Data: The whole dataset was tiny (only about 500 examples for some tasks). It's like trying to learn a new language with only a few flashcards. Because of this, the AI got confused easily and sometimes "memorized" the training data instead of learning the rules (a problem called overfitting).
The "Rare Bird" Problem: Because some mental health states were so rare in the data, the AI struggled to recognize them. The researchers suggested that in the future, they might need to use AI to invent fake but realistic examples of these rare states to help the system learn better.

The Bottom Line

The team built a pipeline that takes a stream of social media posts and turns them into a structured report:

Labels what is happening in each post.
Scores the intensity.
Detects sudden or slow changes.
Summarizes the story.
Finds repeating patterns of recovery or decline.

Important Note: The authors are very clear that this is a research tool, not a doctor. They warn that if the AI makes a mistake (like saying someone is fine when they aren't, or vice versa), it could have serious real-world consequences. They emphasize that this system should never be used to diagnose people without a human mental health professional overseeing it.

Technical Summary: Team MKC at CLPsych 2026

Problem Statement

The paper addresses the challenge of capturing and characterizing mental health changes through social media timelines, a critical task given the global prevalence of mental health disorders and the limited accessibility of professional care. While Large Language Models (LLMs) have shown promise in general domains, their application to clinical psychology is hindered by the scarcity of training data due to privacy concerns. The authors focus on the CLPsych 2026 shared task, which utilizes a dataset introduced by Tsakalidis et al. (2022b) to identify "Moments of Change" (MOC)—temporal points where shifts in a user's mood become observable. The core problem involves a pipeline of five interrelated tasks:

Post-level Assessment: Classifying ABCD sub-elements and self-states (adaptive/maladaptive) for individual posts.
User-level Aggregation: Estimating the presence rate of each mental state across a user's history.
Temporal Modeling: Identifying "Switches" (sudden changes) and "Escalations" (gradual intensifications) in chronologically ordered posts.
Summarization: Generating summaries of change events.
Pattern Extraction: Detecting recurrent dynamic signatures of improvement and deterioration.

A significant technical hurdle identified is the severe class imbalance and data sparsity within the dataset, where some sub-dimensions contain only a single training instance or none at all.

Methodology

The authors propose a unified, five-step LLM-based pipeline designed to integrate post-level assessment with user-level temporal modeling.

Step 1: Post-Level Classification

The system classifies 12 independent targets per post (6 subdimensions for adaptive states and 6 for maladaptive states).

Architecture: Qwen3-4B-Embedding (Qwen Embed) serves as the feature extractor, fine-tuned via LoRA.
Training Strategy: The embeddings are fed into 12 independent classifiers trained with Cross-Entropy loss. To address class imbalance, the authors employ inverse-frequency weighting (clipped to a maximum of 10 to prevent instability) and a 5-fold cross-validation strategy with ensembling.

Step 2: Self-State Presence Rating

This step predicts numerical scores (1–5 Likert scale) for adaptive and maladaptive self-states.

Architecture: Dedicated predictors regress scalar values from the Step 1 embeddings.
Training Strategy: Models use weighted Huber loss, where weights are derived from the inverse frequency of score bins. The output is scaled to the [1, 5] range. Like Step 1, a 5-fold cross-validation ensemble is used.

Step 3: Moment of Change (MOC) Detection

This step detects two transition types: Switch (sudden change between consecutive posts) and Escalation (gradual intensification).

Architecture: Qwen Embed is fine-tuned with LoRA, attached to two independent linear classifiers.
Input: A sliding window of size 2 (posts at $t-1$ and $t$ ) to predict transitions at time $t$ .
Training Strategy: Binary Cross-Entropy with adaptive class weighting and a 5-fold cross-validation ensemble. The authors note that larger models (8B) overfit on the small dataset (~500 samples), leading them to prefer the 4B model.

Step 4: Sequence Summarization

The goal is to generate a natural language summary of the entire sequence, incorporating annotations from previous steps.

Architecture: Supervised Fine-Tuning (SFT) on Qwen3.5-4B and Qwen3-4B-Instruct using LoRA.
Inference: The model takes predictions from Steps 1–3 as input rather than ground truth, faithfully simulating the pipeline setting.
Challenge: The authors observed that fine-tuning on this small dataset sometimes degraded performance compared to zero-shot prompting, likely due to noise propagation and the model overwriting pre-trained knowledge.

Step 5: Dynamic Signature Extraction

This final step analyzes how self-state components interact across sequences to identify recurrent patterns of deterioration and improvement.

Approach: A zero-shot approach using Qwen3.5-9B.
Process: To avoid Out-of-Memory errors, sequences are split into groups to extract signatures, which are then summarized.

Key Results

Steps 1 & 2 (Classification & Regression): On the validation set, adaptive weighting consistently improved performance for Step 1. However, for Step 2, unweighted models performed better. On the test set, K-fold ensembling provided clear improvements over single-model baselines for both steps, attributed to the more complete utilization of the training data.
Step 3 (MOC Detection): The Qwen3-4B model outperformed the 8B variant, confirming that larger models overfit on the limited dataset. The K-fold ensemble further boosted the F1 score to 0.55.
Step 4 (Summarization): The fine-tuned models (Qwen3.5-4B and Qwen3-4B) underperformed the zero-shot baseline in terms of consistency and contradiction scores. The authors attribute this to noise from preceding steps and the risk of fine-tuning erasing pre-trained knowledge in a low-resource regime.
Step 5 (Signatures): The zero-shot approach yielded scores of 0.7266 for Improvement and 0.2301 for Deterioration.

Significance and Contributions

The paper presents a comprehensive pipeline that unifies post-level assessment and user-level temporal modeling for mental health analysis. Its primary contributions include:

Unified Framework: A cohesive system that handles the spectrum from granular post classification to high-level temporal pattern extraction.
Robustness to Imbalance: The effective use of inverse-frequency weighting and K-fold ensembling to mitigate the severe class imbalance and data sparsity inherent in mental health datasets.
Empirical Insights on Model Scaling: The study highlights that in low-resource, high-noise domains, smaller models (4B) combined with ensemble strategies can outperform larger models (8B) that are prone to overfitting.
Caution on Fine-Tuning: The results suggest that in extremely low-resource settings, zero-shot prompting may sometimes be superior to fine-tuning, as the latter risks overwriting valuable pre-trained knowledge.

Limitations and Future Work

The authors acknowledge significant limitations, primarily the scarcity of training data (approx. 500 samples) and the resulting class imbalance. They note that some sub-dimensions had insufficient data for reliable learning.

Proposed Solution: The authors suggest using LLMs to generate synthetic user posts for underrepresented sub-dimensions to create a more balanced dataset.
Ethical Considerations: The paper emphasizes that this pipeline is a research artifact, not a diagnostic tool. It warns of the risks of false positives/negatives in clinical settings and the ethical complexities of generating synthetic data for sensitive topics like self-harm, advocating for clinical expert review before any such data is used for training.