💻 computer science

Automatic detection of depression in clinical interviews with large language models

This study demonstrates that automated language analysis of clinical interviews using transformer-based models and ensemble strategies can effectively detect major depressive disorder in Cantonese speakers, significantly improving processing efficiency while highlighting the importance of transcription quality and conversational context for optimal performance.

Original authors: Longdi Xian, Kit Ying Chan, Jie Chen, Joey W Y Chan, Ngan Yin Chan, Bei Huang, Yun-Kwok Wing, Tim M H Li

Published 2026-09-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Longdi Xian, Kit Ying Chan, Jie Chen, Joey W Y Chan, Ngan Yin Chan, Bei Huang, Yun-Kwok Wing, Tim M H Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Depression is a common and serious condition that affects millions of people worldwide, yet finding it early often depends on a person's ability to access a trained mental health professional. In many places, these experts are scarce, and the process of diagnosis can be slow, leaving many without the help they need. To address this, researchers have begun exploring whether computers can learn to spot signs of depression by listening to how people speak. This field relies on the idea that the way a person describes their feelings, the words they choose, and the rhythm of their conversation can reveal their mental state. Recent advances in artificial intelligence have created powerful tools capable of understanding human language, offering a new way to analyze these conversations automatically. The goal is not to replace doctors, but to create a fast, accessible tool that can help identify who needs care, potentially reaching people who would otherwise go undiagnosed.

A team of researchers at the Chinese University of Hong Kong set out to test whether these artificial intelligence tools could accurately detect major depressive disorder using recordings of clinical interviews. They worked with 299 Cantonese-speaking participants, including 194 patients diagnosed with depression and 105 healthy individuals. The researchers recorded structured interviews where a clinician asked questions and the participant responded. To see if they could make the process faster and more scalable, they compared two ways of turning these audio recordings into text. The first method was the traditional approach, where humans manually wrote down every word spoken, a process that took about 45 minutes for each interview. The second method used an automated system to transcribe the audio instantly. They then fed these texts into several different computer models designed to understand language, testing whether the models could tell the difference between the healthy participants and those with depression.

The study found that using the automated transcription system was a massive leap forward in efficiency. It reduced the time needed to process each interview from roughly 45 minutes down to just under 10 minutes, a 79 percent reduction. This speed makes it possible to analyze large numbers of interviews quickly, which is essential for real-world use. However, the researchers discovered that this speed came with a small trade-off. The models trained on the manually written texts were slightly better at identifying depression than those trained on the automatically generated texts. This suggests that while machines are getting very good at listening, they still occasionally miss the subtle nuances that a human transcriber catches. Despite this small gap, the automated versions were still highly effective, proving that the technology is ready for practical application where speed is critical.

Another key discovery was about what parts of the conversation the computer should listen to. The researchers tested two scenarios: one where the model only read the patient's answers, and another where it read the entire conversation, including the doctor's questions. They found that the models performed better when they had the full context of the dialogue. A patient's short answer, such as "not really" or "sometimes," might seem vague on its own, but when the model sees the question that came before it, the meaning becomes much clearer. For instance, if a doctor asks about sleep patterns and the patient says "sometimes," the computer understands this refers to sleep disturbances. This context helps the model make more accurate judgments, showing that the interaction between the doctor and the patient holds valuable clues that are lost if only the patient's voice is analyzed.

To make their detection even more reliable, the team combined the predictions of several different computer models into a single, stronger system. They tested various ways of merging these opinions, such as taking the average of their scores or looking for the strongest signal among them. This combined approach, known as an ensemble, improved the accuracy of the detection beyond what any single model could achieve on its own. The best-performing combination was then tested on a completely new group of 169 young people who had not been part of the original study. In this independent test, the combined system successfully identified depression with a high level of accuracy, performing just as well as the best single model and slightly better than the others. This confirmed that the system could generalize its findings to new people, not just the ones it was trained on.

The results of this study offer a promising path forward for mental health care. The researchers demonstrated that it is possible to build a system that detects depression from speech with high accuracy while drastically cutting down the time required for analysis. Although the automated transcripts were not perfect, they were good enough to support a system that works well in the real world. The study also highlighted that including the doctor's questions in the analysis provides crucial context that improves the computer's understanding. While the system is not yet a replacement for a human diagnosis, it represents a significant step toward making mental health screening faster, more accessible, and available to more people who need it. The work suggests that with the right tools, technology can help bridge the gap between those suffering from depression and the care they deserve.

Technical Summary: Automatic Detection of Depression in Clinical Interviews with Large Language Models

Problem Statement
Accurate and timely detection of Major Depressive Disorder (MDD) is critical for effective treatment, yet access to trained mental health professionals remains a significant barrier. Traditional assessment relies on clinical judgment, which is not easily scalable. While Large Language Models (LLMs) offer potential for analyzing language patterns in clinical interviews to detect depression, several challenges hinder their practical application:

Data Processing Efficiency: Manually transcribing clinical audio interviews is labor-intensive, creating a bottleneck for real-time or large-scale deployment.
Data Source Variability: It is unclear whether models perform better using only patient speech or the full interviewer-interviewee interaction, and how transcription quality (manual vs. automatic) impacts performance.
Model Optimization: While ensemble methods are known to improve diagnostic accuracy, the optimal aggregation strategies (e.g., mean, max, variance) for depression detection in clinical dialogues remain under-explored.

Methodology
The study utilized structured clinical interview recordings from 299 Cantonese-speaking participants (194 with MDD, 105 healthy controls) collected between 2020 and 2022. The research workflow involved four primary stages:

Data Preparation and Transcription:
- Manual Transcription: Human experts transcribed interviews, retaining speaker roles.
- Automatic Transcription: Audio was processed using speaker diarization (PyAnnote-Audio) followed by Automatic Speech Recognition (ASR). Five ASR tools were benchmarked, with Tencent Cloud Speech-to-Text selected for the study due to the lowest Word Error Rate (WER) of 0.102 in Cantonese.
- Dataset Construction: Four distinct datasets were created: Manual Interaction, Manual Interviewee-only, Automatic Interaction, and Automatic Interviewee-only.
Model Architecture and Training:
- Seven pre-trained Chinese transformer architectures were evaluated: BERT, RoBERTa, XLNet, ELECTRA, ALBERT, MentalBERT, and DistilBERT.
- Training Strategy: A participant-level stratified five-fold cross-validation was employed to prevent data leakage. Long interviews were tokenized into overlapping windows (max 512 tokens, 128 overlap).
- Hyperparameters: Models were trained with a learning rate of $1\times10^{-5}$ , batch size of 8, and early stopping based on validation AUC.
Ensemble Construction:
- The study systematically compared individual models against ensemble approaches.
- Aggregation Methods: Nine statistical aggregation techniques were tested, including Mean, Max, Min, Standard Deviation (SD), Median, Variance, Covariance, and Upper/Lower Quartiles.
- Ensemble Strategies: Three approaches were tested: (a) ensembling all models, (b) ensembling the top $n$ models based on performance ranking, and (c) ensembling all possible combinations of $n$ models to find the global optimum.
Evaluation and Validation:
- Internal Metrics: Performance was measured using AUC (primary metric), F1 Score, Sensitivity, Specificity, PPV, and NPV.
- External Validation: The best-performing ensemble (selected solely on internal data) was tested on an independent external cohort of 169 youth participants (88 HC, 81 MDD) to assess generalizability.

Key Results

Efficiency Gains: Automatic transcription reduced the average processing time per interview from approximately 45 minutes (manual) to 9.65 minutes, representing a 79% reduction in time.
Transcription Quality: While automatic transcription improved efficiency, it resulted in a slight, though often non-significant, reduction in model discrimination compared to manual transcripts. For example, in interviewee-only data, DistilBERT achieved an AUC of 0.78 with manual transcripts versus 0.70 with automatic transcripts ( $p=.004$ ).
Speaker Role Impact: Models trained on complete interviewer-interviewee interactions generally achieved higher AUCs than those trained on interviewee-only text. This was statistically significant for RoBERTa, ALBERT, and MentalBERT in manual transcripts, and for most models in automatic transcripts. The authors suggest the interviewer's questions provide necessary context for interpreting short patient responses.
Ensemble Performance:
- Ensemble models significantly outperformed the best individual models in internal validation (e.g., a 3-model ensemble of ELECTRA, ALBERT, and RoBERTa using Mean aggregation achieved an AUC of 0.87 on automatic interaction text, surpassing RoBERTa alone at 0.82).
- The optimal aggregation method varied by condition: Lower-quartile aggregation was optimal for manual transcripts, while Mean aggregation was optimal for automatic transcripts.
External Validation: In the independent external cohort, the final ensemble achieved an AUC of 0.75. This was identical to the standalone RoBERTa model (AUC 0.75) and slightly higher than DistilBERT (AUC 0.73). The improvement seen in internal ensemble selection did not translate to a statistically significant gain over the best single model in the external setting.

Significance and Claims
The paper claims that automated language analysis is a feasible approach for depression detection, offering a viable balance between efficiency and accuracy. The study highlights three main contributions:

Scalability: Demonstrating that high-quality ASR (specifically Tencent Cloud for Cantonese) can drastically reduce data preparation time without completely compromising model performance, making large-scale screening more practical.
Contextual Importance: Providing evidence that including the full clinical dialogue (interviewer and interviewee) yields better diagnostic performance than patient speech alone, likely due to the contextual cues provided by the clinician's questions.
Methodological Rigor: Systematically evaluating various ensemble aggregation strategies and demonstrating that while ensembles improve internal performance, their external generalizability must be rigorously tested, as internal gains do not always guarantee superior external discrimination.

The authors conclude that while automated tools are promising, factors such as transcription quality, conversational context, and the necessity of independent external validation are critical when interpreting model performance for clinical applications. The study does not claim the models are ready for immediate clinical deployment but supports their use as research tools and potential aids in mental health assessment.

Technical Summary: Automatic Detection of Depression in Clinical Interviews with Large Language Models

More like this