Repurposing Large Language Models as Instruments to Detect Transdiagnostic Speech Deviations across the Psychosis–Affective Spectrum
This study demonstrates that quantifying deviations from large language model expectations (surprisal) serves as a scalable, robust marker for detecting transdiagnostic speech alterations, particularly in schizophrenia-spectrum and bipolar disorders, though its ability to distinguish individual patients remains modest.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
For decades, psychiatry has relied heavily on the spoken word. When a doctor assesses a patient, they listen not just to what is said, but to how it is said. They pay attention to the flow of ideas, the connection between sentences, and the rhythm of speech. These patterns can reveal deep disturbances in thinking, such as when a person's thoughts become disorganized or jump erratically from one topic to another. This is a core feature of conditions like schizophrenia, where the very structure of language can break down. However, listening to a patient and judging these subtle shifts is a difficult task for humans. It requires intense training, and even experts often disagree on what they hear. Because there are no simple biological tests, like a blood test, to diagnose these conditions, the field has long searched for a more objective way to measure these speech patterns.
Recently, a new kind of computer program has emerged that might offer a solution. These are large language models, the same technology that powers many modern chatbots. They are trained on vast amounts of text from books, websites, and conversations, learning the statistical rules of how words usually follow one another. In essence, they build a massive internal map of what a normal, healthy conversation sounds like. Researchers have begun to wonder if these programs could act as a new kind of instrument. If a person's speech deviates significantly from the patterns the computer expects, could that deviation serve as a measurable sign of illness? This question sits at the intersection of artificial intelligence and mental health, asking whether a machine's ability to predict the next word in a sentence can help us understand the human mind.
A team of researchers in Germany set out to test this idea with a large group of people. They gathered 371 participants, including 178 healthy individuals and 194 patients diagnosed with different psychiatric conditions. The patients fell into three main groups: those with schizophrenia-spectrum disorders, those with bipolar disorder, and those with major depressive disorder. To ensure everyone was speaking under similar conditions, the researchers asked each person to look at four pictures and tell a story about what was happening in the scene. This is a standard test used in psychology to encourage natural storytelling. The researchers recorded these stories, transcribed them into text, and then fed them into a sophisticated language model.
The computer was instructed to act as if it were a healthy person. As it read each story, it calculated a score for every sentence based on how surprised it was by the words it encountered. If the story flowed in a way the computer expected, the surprise score was low. If the story took an unexpected turn, used strange word combinations, or broke the rules of grammar in a way that disrupted the flow, the surprise score went up. The researchers called this score "surprisal." They then compared these scores across the different groups, carefully adjusting for factors like age, education, and the average length of the sentences, to make sure the results were not just a reflection of how long people spoke.
The results revealed a clear pattern that distinguished the groups. The stories told by people with schizophrenia-spectrum disorders consistently produced much higher surprise scores than those told by the healthy volunteers. This means their speech was significantly less predictable to the computer, deviating from the statistical norms of healthy language. People with bipolar disorder also showed elevated scores, though the effect was not as strong as in the schizophrenia group. Surprisingly, the stories from people with major depressive disorder were indistinguishable from those of the healthy controls. Their speech was just as predictable as the healthy volunteers, suggesting that the specific type of language disruption measured by this method is not a general feature of all mental illness, but is tied more closely to the psychosis spectrum.
The researchers also looked at whether these scores matched up with the severity of the patients' symptoms. They found that higher surprise scores were linked to greater difficulties in thinking and speaking, such as the disorganized thoughts often seen in schizophrenia. However, this link was mostly driven by the differences between the diagnostic groups rather than by how sick an individual patient was on a given day. In other words, the computer was better at telling the difference between a person with schizophrenia and a healthy person than it was at telling how severe a specific patient's symptoms were at that moment. The study also confirmed that these results were not simply because patients spoke in shorter sentences or had less education; the effect remained even after the researchers accounted for these factors.
While the computer's ability to distinguish between groups was promising, it was not perfect. The technology could identify patterns that human raters might miss, but it was not yet accurate enough to be used as a standalone diagnostic tool. The study suggests that this method captures a real and measurable aspect of how language breaks down in certain psychiatric conditions, offering a new, objective way to look at speech. It shows that the statistical expectations of a machine, trained on the collective language of humanity, can detect the subtle, often invisible fractures in the speech of those struggling with psychosis. This does not replace the need for human doctors, but it offers a potential new tool to help them see what is happening in the mind through the words a person chooses to speak.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.