🧬 biology

An Explainable XGBoost Framework for Anxiety and Depression Risk Prediction with SHAP and Counterfactual Reasoning

This study presents an explainable XGBoost framework for predicting anxiety and depression risk that addresses class imbalance and evaluation rigor while integrating SHAP and counterfactual reasoning to provide transparent, interpretable insights into model decisions without replacing clinical diagnosis.

Original authors: Anima Dahal, Bibash Basnet

Published 2026-09-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anima Dahal, Bibash Basnet

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the modern landscape of healthcare, artificial intelligence has emerged as a powerful tool for sifting through vast amounts of human data to find patterns that might otherwise remain hidden. When applied to mental health, these systems aim to look at the answers people give to questions about their lives, their habits, and their feelings to spot early signs of anxiety or depression. The challenge, however, is that these conditions are complex and often subtle, and the data used to study them is frequently unbalanced, with far more people reporting good mental health than those reporting struggles. Furthermore, a computer model that makes a prediction is often a black box; it gives an answer without explaining why, which makes it difficult for doctors to trust or use that information responsibly. To be truly useful, a system must not only predict risk accurately but also show its work, revealing which factors drove the decision and ensuring it treats different groups of people fairly.

A team of researchers at Nilai University has developed a new framework designed to meet these exact challenges. They built a sophisticated computer program capable of assessing the risk of anxiety and depression using a large collection of survey responses and personal details. Rather than simply training a model to guess correctly as often as possible, the researchers focused on creating a system that is transparent, reliable, and carefully tested. They started with a dataset containing information from more than 24,000 participants, including their answers to standard questionnaires about stress, sleep, and mood, alongside details like age, gender, and education. Because the number of people with anxiety or depression in the group was much smaller than those without, the team had to teach the computer to pay extra attention to the minority cases, ensuring it did not simply ignore them to achieve a high overall score.

The researchers split their data into three distinct groups to ensure their results were honest and not just a lucky guess. They used one group to teach the computer, a second group to fine-tune how the system makes decisions, and a third, completely isolated group of 3,644 people to test the final result. This strict separation prevented the system from memorizing the test answers, a common pitfall that leads to overly optimistic results. They also adjusted the system's decision-making process. Instead of using a standard rule where a score above fifty percent counts as a positive result, they raised the bar significantly. They found that a much higher threshold was necessary to make the predictions reliable, effectively telling the system to be very sure before flagging someone as being at risk.

When the final test was run on the isolated group of 3,644 participants, the system performed with remarkable precision. For anxiety, it achieved an AUC of 0.98, and for depression, it achieved an AUC of 0.95. More importantly, the system did not just give a yes or no answer; it provided a clear explanation for its choices. Using a method that breaks down the influence of each question, the researchers found that answers related to stress were the most powerful drivers for both anxiety and depression predictions. Sleep-related questions also played a major role, particularly for depression. The system could show exactly how a person's specific answers pushed the final score up or down, allowing a clinician to see the logic behind the risk assessment.

To further test the system's logic, the researchers asked a different kind of question: what would have to change for the computer to change its mind? They created hypothetical scenarios where they slightly altered a person's answers to see if the risk score would drop. They discovered that changing just a few key answers, such as reducing reported stress levels or improving sleep quality in the data, was often enough to move a person from a high-risk category to a lower one. This demonstrated that the model was sensitive to specific changes and not just making random guesses. The team also checked to ensure the system worked equally well for men and women, finding that while the exact numbers varied slightly, the ability to distinguish between high and low risk remained consistent across genders.

The researchers emphasize that this system is not a replacement for a doctor's diagnosis. Mental health involves deep personal and contextual factors that a computer cannot fully capture. Instead, this framework is designed as a decision-support tool, a way to help practitioners identify individuals who might benefit from a closer look. By combining high accuracy with clear explanations and fair testing across different groups, the study offers a more trustworthy path forward for using artificial intelligence in mental health screening. It shows that when technology is built with care and transparency, it can illuminate the path to better care without losing sight of the human complexity at its core.

Technical Summary: An Explainable XGBoost Framework for Anxiety and Depression Risk Prediction

Problem Statement
While machine learning (ML) offers potential for the early identification of anxiety and depression risks, existing approaches often prioritize aggregate predictive performance at the expense of model interpretability, robustness, and systematic evaluation. Common limitations in current mental health prediction literature include:

Insufficient Explainability: Models often function as "black boxes," failing to provide insights into the specific factors driving individual decisions.
Methodological Flaws: Many studies suffer from data leakage, where the test set is inadvertently used during hyperparameter optimization, feature selection, or threshold tuning, leading to overly optimistic performance estimates.
Neglect of Critical Evaluation Dimensions: Issues such as class imbalance, probability calibration, decision-threshold selection, and subgroup fairness are frequently under-addressed.
Lack of Counterfactual Reasoning: Few studies explore how hypothetical changes in input features could alter model predictions, limiting the utility of the models for understanding decision sensitivity.

Methodology
The authors propose a comprehensive, leakage-controlled framework for predicting anxiety and depression risks using demographic and psychological questionnaire data (PHQ-9, GAD-7, PSS, ISI). The study treats anxiety and depression as two independent binary classification tasks ( $Y_A$ and $Y_D$ ) using a dataset of 24,292 participants.

Data Partitioning and Leakage Prevention: The dataset was strictly divided into training, validation, and isolated test sets (3,644 participants). The test set was reserved exclusively for final evaluation and was not used for hyperparameter tuning, feature selection, threshold selection, or calibration.
Model Development:
- Baseline Models: Logistic Regression (with balanced class weights) and Random Forest were established as benchmarks.
- Primary Model: Optimized XGBoost classifiers were developed for both targets. Hyperparameters were tuned via five-fold cross-validation on the training set, using PR-AUC as the optimization objective to address class imbalance.
Threshold Optimization: Instead of relying on a default 0.50 threshold, decision thresholds were selected using the validation set. A threshold of 0.90 was chosen for both models to balance precision and recall, acknowledging the high cost of false negatives in screening contexts.
Probability Calibration: Models underwent probability calibration (post-training) to ensure predicted probabilities reflected observed outcome likelihoods, evaluated using the Brier score.
Explainability and Reasoning:
- SHAP (SHapley Additive exPlanations): Used for global feature ranking (mean absolute SHAP values) and local interpretation (waterfall plots) to identify influential features and individual decision drivers.
- Counterfactual Analysis: Generated hypothetical scenarios to determine the minimal set of feature changes required to shift a prediction across the 0.90 decision threshold.
Subgroup Analysis: Performance was evaluated across demographic subgroups (gender, education, habits) to assess consistency and potential fairness issues, without using these groups for model optimization.

Key Results
On the isolated test set of 3,644 participants, the optimized XGBoost models demonstrated strong discrimination and reliability:

Performance Metrics (Threshold = 0.90):
- Anxiety: Accuracy 97.59%, ROC-AUC 0.9601, Precision 47.17%, Recall 60.98%, F1-score 0.5319.
- Depression: Accuracy 94.62%, ROC-AUC 0.9386, Precision 50.25%, Recall 51.27%, F1-score 0.5075.
- Note: The high accuracy reflects the correct classification of the majority negative class; the authors emphasize that precision, recall, and PR-AUC are critical for evaluating performance on the imbalanced positive classes.
Comparison: The optimized XGBoost models outperformed baseline Logistic Regression, Random Forest, and default XGBoost configurations, particularly in terms of ROC-AUC and PR-AUC.
Calibration: Calibration significantly improved probability reliability, reducing the Brier score for anxiety from 0.0542 to 0.0151 and for depression from 0.0921 to 0.0355, without altering the ranking-based discrimination (ROC-AUC/PR-AUC).
Feature Importance (SHAP): Stress-related features from the Perceived Stress Scale (PSS Q1, Q2, Q3) were the most influential predictors for both conditions. Sleep-related features (ISI) also contributed notably, particularly for depression.
Counterfactuals: Analysis showed that modifying specific combinations of PSS and ISI scores could successfully lower predicted probabilities below the 0.90 threshold, demonstrating model sensitivity to input variations.
Subgroup Fairness: Discrimination (ROC-AUC) remained broadly comparable across gender groups (e.g., Anxiety ROC-AUC: 0.9594 for females, 0.9612 for males), though variations were observed in precision and recall metrics.

Significance and Claims
The paper claims to address a research gap by moving beyond simple accuracy reporting to provide a trustworthy, interpretable, and systematically evaluated framework. Its primary contributions include:

Rigorous Evaluation Protocol: By strictly isolating the test set and using validation data for all optimization steps, the study provides an unbiased estimate of generalization performance.
Holistic Framework: The integration of class-imbalance handling, threshold optimization, probability calibration, SHAP-based explanations, counterfactual reasoning, and subgroup analysis offers a more transparent approach to AI-assisted mental health assessment than predictive performance alone.
Clinical Contextualization: The authors explicitly state that the framework is intended as a risk-screening and decision-support tool, not a replacement for clinical diagnosis. The high decision threshold (0.90) and the focus on explainability are designed to support practitioners in prioritizing individuals for further assessment while maintaining transparency regarding model limitations and the non-causal nature of the predictions.

The study concludes that combining optimized XGBoost with these interpretability and evaluation techniques enhances the reliability and utility of AI in mental health risk assessment, provided the system is used alongside professional clinical judgment.

Technical Summary: An Explainable XGBoost Framework for Anxiety and Depression Risk Prediction

More like this