Technical Summary: Dep-LLM

Problem Statement

Automatic Depression Detection (ADD) from clinical interview transcripts faces two critical bottlenecks that hinder the deployment of supervised systems:

1. Long-Context Reasoning Challenges: Clinical interviews are lengthy and multi-topic, with depression clues being complex but sparsely distributed and often intertwined with irrelevant content (e.g., greetings). Off-the-shelf Large Language Models (LLMs) tend to collapse these dialogues into holistic, superficial judgments, overlooking specific clinical clues or producing medically unreliable hallucinations.
2. Data Scarcity and High Training Costs: Supervised methods and fine-tuned LLMs rely heavily on large-scale, labeled clinical data, which is scarce due to privacy and ethical concerns. Furthermore, the computational cost of training or fine-tuning foundation models creates a prohibitive barrier for clinical institutions with limited budgets.

Existing approaches, including supervised deep learning and generative reasoning with LLMs, often lack internal mechanisms to verify the reliability of their own reasoning or depend intensively on scarce labeled data and computational resources.

Methodology: Dep-LLM

The authors propose Dep-LLM, a novel training-free framework that operates entirely on frozen, off-the-shelf foundation LLMs. The framework mirrors the step-by-step reasoning process of clinical psychiatrists through a three-stage pipeline, requiring no task-specific fine-tuning, no labeled depression corpus, and no auxiliary trainable parameters.

1. CoT Depression Multi-factor Analysis

To address the difficulty of modeling sparse clues in long contexts, the first stage employs a Chain-of-Thought (CoT) scheme to structurally decompose the raw interview dialogue. This process involves three sequential prompting steps on the same frozen LLM:

• Thematic Content Extraction: The dialogue is decomposed into four atomic themes aligned with the Structured Clinical Interview for DSM-5 (SCID): Family Relationship, Work Satisfaction, Mental State, and Medical History. Trivial chitchat is filtered out.
• Integrative Overall Synthesis: The atomic themes are integrated into an overall narrative summary to capture global clues (e.g., cross-theme emotional consistency) that isolated themes cannot express.
• Depression Multi-factor Analysis: The model performs a fine-grained assessment over the full multi-factor space (5 themes $\times$ 3 labels: Depressive, Healthy, Neutral). For each (theme, label) pair, the LLM generates a possibility score ($P_{ij}$) and an explicit, evidence-grounded rationale ($R_{ij}$).

2. Semantic Confidence Analysis and Modulation

Since training-free LLMs are prone to producing locally plausible but clinically unreliable explanations, the second stage quantifies the epistemic reliability of each rationale without supervised verification.

• Token-Level Entropy: Confidence is derived from the token-level predictive distribution of the LLM. The rationale confidence ($C$) is defined as the reciprocal of the Shannon entropy ($H$) of the token distribution. Low entropy (high confidence) indicates the model is decisive and likely relying on internalized knowledge, whereas high entropy suggests uncertainty or hallucination.
• Contrastive Modulation: To prevent scale-driven artifacts (e.g., longer themes dominating), the system applies an intra-label and inter-theme modulation. This mechanism amplifies signals from themes where the rationale is confident while suppressing uncertain ones, effectively weighting the most reliable diagnostic factors.

3. Collaborative Multi-factor Prediction

The final stage synthesizes the multi-factor assessments into a single diagnostic decision via confidence-weighted fusion:

• Aggregation: Support scores for each label are computed by fusing possibility scores with the modulated confidences of the five themes.
• Decision Rule: A depressive coefficient ($\delta$) is calculated, incorporating depressive evidence, healthy evidence, and calibrated neutral evidence. The final prediction is determined by comparing $\delta$ against a decision threshold ($\delta^*$).
• Transparency: All coefficients are either analytically derived from the modulation process or fixed as transparent hyperparameters, ensuring the system remains interpretable and free of learned weights.

Key Contributions

1. Structured Multi-factor Framework: The authors introduce Dep-LLM, a framework that addresses reasoning in long SCID-style dialogues by decomposing them into clinically aligned themes and verifying reliability through a confidence mechanism.
2. Training-Free Efficiency: The framework is implemented entirely under a training-free setting, eliminating the overhead of expensive model training and the need for scarce labeled data while maintaining strong performance in clinical scenarios.
3. Integrated Architecture: The architecture uniquely integrates CoT reasoning, semantic confidence analysis (based on token entropy), and collaborative multi-factor fusion, jointly reinforcing the interpretability and rationality of the automatic diagnosis.

Experimental Results

Extensive experiments were conducted on the DAIC-WOZ and E-DAIC datasets.

• Vs. Zero-Shot Baselines: Dep-LLM was evaluated across 21 foundation LLMs (including Llama-2/3/4, Qwen-2.5/3/3.5, and Gemma-2/3 families, ranging from 4B to 17B parameters). It surpassed zero-shot baselines on nearly all 9 metrics (Accuracy, Macro F1, Weighted-Average F1, etc.) for almost all tested models. Notably, it bridged the reasoning gap for weaker models (e.g., improving MaF1 by over 20% on Llama3-8B-Base) and achieved the best scores when paired with Gemma3-12B-Instruct (MaF1: 0.818).
• Vs. Supervised Domain-Specific LLMs: Dep-LLM outperformed representative supervised models (e.g., MentalBERT, MentaLLaMA, BioMistral) that rely on extensive pre-training and fine-tuning on mental health corpora. For instance, on DAIC-WOZ, Dep-LLM achieved an Accuracy of 0.851 compared to 0.766 for the best supervised baseline (MentalBERT).
• Vs. Closed-Source Commercial LLMs: Dep-LLM outperformed the latest closed-source commercial LLMs (including GPT-5.5, Gemini-3.1-Pro, Claude-Opus-4.6, and DeepSeek-V4) across most metrics on both datasets, despite being built on a significantly smaller (12B) open-source foundation model.
• Ablation Studies: Experiments confirmed that removing the CoT reasoning, individual themes, or the confidence modulation mechanism led to significant performance drops, validating the necessity of each component.

Significance and Claims

The paper claims that Dep-LLM offers a generalizable and backbone-agnostic augmentation that systematically strengthens vanilla LLMs in long-context depression reasoning. Its primary significance lies in demonstrating that a well-structured, training-free pipeline can rival and surpass supervised domain-specific models and frontier commercial LLMs in high-stakes clinical scenarios.

By relying on structured CoT decomposition and entropy-grounded confidence modulation, the framework achieves high accuracy and interpretability without the prohibitive costs of data collection and model training. This makes Dep-LLM particularly suitable for real-world clinical settings where data privacy is paramount and computational budgets are constrained. The authors emphasize that for clinical applications, the quality of the reasoning structure and reliability verification is more critical than mere parameter scale.