Technical Summary: Rafiq – An Arabic Conversational AI System for Early Depression Detection

Problem Statement

Depression is a leading cause of global disability, yet Arabic-speaking communities remain significantly underserved by clinically grounded conversational AI. Existing mental health chatbots are predominantly English-centric, while Arabic-language alternatives often lack clinical validation, structured screening protocols, or evidence-grounded response generation. Furthermore, there is a critical shortage of specialized Arabic Natural Language Processing (NLP) tools that integrate symptom monitoring with standardized clinical instruments like the Patient Health Questionnaire-9 (PHQ-9) within a natural conversational flow.

Methodology

The authors present Rafiq, an end-to-end conversational healthcare informatics system designed for early depression detection and psychoeducation in Arabic. The system is architected as a hybrid design combining a transparent, audit-friendly symbolic layer with neural components, orchestrated via a six-node LangGraph workflow and deployed through a React Native mobile application with a FastAPI backend.

The system comprises four core technical modules:

1. Arabic Mental Health Knowledge Base (KB):

   • Constructed via a three-phase curation methodology (collection, Markdown conversion with manual review, and processing).
   • Sourced from four authoritative clinical texts: DSM-5-TR (Arabic translation), MSD Manual, WebTeb, and WHO.
   • Resulted in 7 Markdown files totaling 14,624 words, covering definitions, symptoms, causes, treatments, and care strategies.
   • Documents were chunked using a structure-aware strategy (MarkdownHeaderTextSplitter) to preserve semantic coherence of diagnostic criteria.

2. Symptom Detection Module:

   • Utilizes GPT-4o-mini with a keyword-guided prompting strategy.
   • Relies on a clinically validated 217-expression Arabic lexicon mapped to the nine PHQ-9 symptom criteria.
   • The lexicon was expanded to include gender variants and Levantine (Shami) dialectal forms.
   • Evaluation employed a "leakage-aware" train-test split where test-set keywords were withheld from the prompt to ensure the model learned semantic generalization rather than exact matching.

3. PHQ-9 Integration Module:

   • Implements a three-phase protocol: (i) Proposal and Consent, (ii) Structured Administration (using selectable buttons to prevent free-text ambiguity), and (iii) Scoring and Feedback.
   • Triggered automatically when the symptom detector identifies three or more unique symptoms cumulatively.
   • The questionnaire logic and scoring thresholds were reviewed and approved by licensed clinical psychologists.

4. Retrieval-Augmented Generation (RAG) Pipeline:

   • Embedding Benchmark: A systematic benchmark was conducted on a purpose-built dataset of 33 queries and 99 annotated query-document pairs to evaluate six Arabic sentence embedding models.
   • Model Selection: Arabic-Triplet-Matryoshka-V2 was selected as the optimal embedder, outperforming general multilingual models (including a state-of-the-art Mixture-of-Experts model) which failed to transfer effectively to the mental health domain.
   • Generation: GPT-4o-mini was selected over GPT-4o based on RAGAS evaluation, offering comparable factual grounding with lower latency and cost.
   • Safety: The system prompt enforces strict constraints: no diagnostic terms, no medical advice, and mandatory referral to professionals for crisis situations.

Key Contributions

The paper outlines six primary contributions:

1. Structured Knowledge Base: A curated, retrieval-ready Arabic mental health KB derived from authoritative clinical sources.
2. Embedding Benchmark: The first systematic evaluation of Arabic sentence embeddings for mental health retrieval, identifying Arabic-Triplet-Matryoshka-V2 as the superior model (MAP = 80%) and demonstrating the failure of general multilingual MoE models in this specific domain.
3. Symptom Detection: An LLM-based classifier achieving 98% accuracy and an F1-score of 0.98 using a keyword-guided approach without task-specific fine-tuning.
4. PHQ-9 Integration: A novel three-phase protocol for administering and scoring the PHQ-9 within a natural conversational flow, a first for Arabic chatbots.
5. RAG-Based Psychoeducation: A semantic retrieval pipeline evaluated with RAGAS, showing high faithfulness (0.88) and context precision (0.91).
6. Clinical Validation: A human evaluation by five licensed psychologists across five scenarios, confirming the system's safety and clinical appropriateness.

Results

• Embedding Performance: Arabic-Triplet-Matryoshka-V2 achieved MAP = 80%, NDCG = 86%, and Precision@1 = 93% at k=2. In contrast, the state-of-the-art multilingual MoE model (nomic-embed-text-v2-moe) scored near zero (MAP = 4%), highlighting the necessity of domain-specific validation.
• Symptom Detection: The classifier achieved an overall accuracy of 0.98 and a macro-averaged F1-score of 0.98 on a leakage-aware test set of 415 instances.
• RAG Evaluation: GPT-4o-mini matched GPT-4o in faithfulness (0.88 vs 0.90) and achieved higher answer relevancy (0.84 vs 0.79), while operating with significantly lower latency.
• Clinical Validation: Five licensed psychologists rated the system with a mean overall score of 4.54/5.0. No critical safety concerns were identified across five distinct conversation scenarios, including suicidal ideation and postpartum depression inquiries.

Significance and Claims

The paper claims that Rafiq demonstrates the feasibility of integrating structured clinical screening with evidence-grounded conversational AI for early depression detection in Arabic. It addresses a documented gap in Arabic mental health technology by moving beyond general information provision to a system that performs:

• Real-time symptom monitoring.
• Structured clinical screening (PHQ-9).
• Retrieval-based psychoeducation grounded in authoritative sources.

The authors emphasize that the system's hybrid architecture—combining a transparent, clinically validated keyword lexicon with neural components—enhances trustworthiness and auditability. They assert that the work provides a baseline and methodology for future development of culturally appropriate Arabic mental health technologies, while explicitly maintaining that Rafiq is a supportive companion and screening tool, not a diagnostic or therapeutic system. The study concludes that such systems can broaden access to psychological support while maintaining appropriate clinical boundaries.