Sentiment Analysis of Thought-Related Language in the Context of Cognitive Disorders Using Machine Learning, Transformer-based Architecture and Large Language Models

The human mind is a constant stream of commentary, a running narration of beliefs, worries, and hopes that shapes how we experience the world. Psychologists have long understood that the way people talk to themselves matters; when that internal dialogue turns harsh or distorted, it can trap individuals in cycles of anxiety and depression. For decades, identifying these harmful thought patterns required a trained professional to listen carefully and interpret the nuances of a patient's language. Today, as people share their inner lives more openly on social media, a new question has emerged: can computers learn to read these digital thoughts with the same care? This field, known as computational mental health, seeks to use the tools of language analysis to detect patterns in text that signal emotional distress. The challenge lies in distinguishing between a fleeting bad mood and a deep-seated negative way of thinking, a task that requires not just counting words, but understanding the weight of what is being said.

A team of researchers at the Islamic University of Technology has taken a significant step toward answering this question by building a system to classify the emotional tone of thoughts expressed online. They did not simply ask a computer to guess if a sentence was happy or sad; instead, they focused specifically on "thought-related" language—statements that reveal how a person interprets their reality. To do this, they gathered a massive collection of 156,785 posts from social media platforms like Reddit, Twitter, and Facebook. They needed a way to sort these millions of words into two categories: positive and negative. First, they used a specialized computer program designed to understand the slang and informal tone of social media to assign an initial label to each post. To ensure these labels were trustworthy, they then brought in two licensed psychologists. These experts reviewed a random sample of 500 posts, rating them on a scale to see if the computer's initial guess matched human judgment. The experts agreed with the computer's classification the vast majority of the time, confirming that the dataset was reliable enough to train a machine to recognize the difference between constructive and destructive thinking.

With a clean, verified dataset in hand, the researchers put a wide variety of computer models to the test to see which one could best perform this classification. They started with traditional methods, which are like simple, efficient calculators that look for specific words and their frequencies. They also tested advanced "transformer" models, which are more complex systems capable of understanding the context and relationship between words in a sentence, similar to how a human reader grasps the full meaning of a story rather than just individual vocabulary. Finally, they tried a large language model, a powerful type of artificial intelligence designed to generate human-like text, by asking it to classify the thoughts using specific instructions. The results were surprising to those who assume that bigger, more complex models are always better. The traditional, simpler models actually outperformed the sophisticated ones. A specific type of simple algorithm, known as logistic regression, achieved the highest accuracy, correctly identifying the sentiment of the thoughts nearly 93 percent of the time. The more complex transformer models, while still very good, scored slightly lower, around 84 percent accuracy. The large language model, despite its reputation for intelligence, performed the weakest in this specific task, getting it right about 80 percent of the time.

The researchers found that for the short, direct sentences typical of social media, the simpler models were actually more effective. These models succeeded because the negative and positive thoughts in the dataset often contained clear, explicit words that signaled their tone, making it unnecessary for the computer to perform deep, complex reasoning to understand them. The more advanced models, which are designed to handle long, nuanced narratives, did not have an advantage here and sometimes struggled with the specific task of binary classification. The study also revealed that the system was remarkably consistent; when the researchers tested the models on different chunks of the data, the results remained stable, showing that the system was not just memorizing answers but genuinely learning the patterns of thought.

Crucially, the authors emphasize that this system is a tool for research, not a diagnostic device for individuals. The computer is trained to recognize the emotional tone of a sentence, not to diagnose a mental health condition or determine if a person is suffering from a specific disorder. A negative thought posted online might reflect a temporary frustration rather than a clinical issue, and the system is designed to reflect that linguistic reality, not to replace a doctor's judgment. The work serves as a foundation for future research, offering a large, verified dataset and a clear demonstration that lightweight, efficient tools can be highly effective for analyzing the emotional landscape of human thought at scale. By proving that simple, transparent methods can outperform complex ones in this specific context, the study suggests that the path forward for understanding mental health through language may lie in clarity and precision rather than sheer computational power.