Refining E-detectioN Tools for Emerging mental disoRders (ENTER) to identify young people at clinical high risk for psychosis in the community: model development and internal validation

Technical Summary: ENTER – Detecting Psychosis Risk in the Community

Problem Statement
Early detection of individuals at Clinical High Risk for Psychosis (CHR-P) is critical for improving long-term outcomes and enabling preventive care. However, current detection rates are low (5–14% of first-episode cases are identified during the CHR-P phase), largely due to reliance on idiosyncratic, resource-dependent referral pathways. Existing community screening tools, such as the Prodromal Questionnaire-16 (PQ-16) with a cut-off of ≥6, offer acceptable sensitivity but poor specificity, leading to high false-positive rates and inefficient triage. There is a need for scalable, digital screening approaches that integrate multimodal data (symptoms, cognition, and environmental factors) to improve discrimination and clinical utility in community settings.

Methodology
The study developed and internally validated the E-DetectioN Tool for Emerging mental disoRders (ENTER), a multimodal digital screener, using data from 2,522 participants aged 12–35 recruited from the UK (Glasgow, London) and Italy (Pavia) between April 2022 and December 2024.

• Data Collection: Participants completed an online screener assessing:
  • Symptoms: PQ-16 score and PQ-16 distress.
  • Cognition: Processing speed via the Digit Symbol Substitution Test (DSST).
  • Environment: Psychosis Polyrisk Score (PPS) comprising 20 environmental risk factors (e.g., migration, deprivation, parental age, urbanicity).
• Gold Standard Assessment: Participants scoring ≥6 on the PQ-16 were invited for a Comprehensive Assessment of At Risk Mental States (CAARMS) interview. A random subset of those scoring below the threshold was also assessed to mitigate selection bias.
• Labeling: Of the 2,522 participants, 495 underwent CAARMS assessment (124 CHR-P+, 371 CHR-P-). The remaining 2,027 participants were "unlabelled."
• Model Development:
  • Algorithms: Penalized logistic regression (Elastic Net), Random Forest, and Extreme Gradient Boosting (XGBoost).
  • Validation Strategy: Repeated nested cross-validation (5 folds, 10 repetitions) with semi-supervised learning (SSL).
  • Semi-Supervised Learning: To utilize the large volume of unlabelled data, a pseudolabeling framework was employed. An initial model trained on labelled data generated risk estimates for unlabelled data. High-confidence predictions (top 5% and bottom 5% with probability ≤0.30) were assigned pseudolabels and iteratively combined with labelled data to augment the training set.
  • Benchmark: Models were compared against the standard PQ-16 cut-off (≥6) logistic regression model.
• Evaluation Metrics: Discrimination (Harrell's C-index), calibration (intercept and slope), overall performance (Brier score), and clinical utility (Decision Curve Analysis).

Key Results

• Discrimination: The ENTER multimodal models significantly outperformed the PQ-16 cut-off alone.
  • PQ-16 Cut-off: C-index = 0.625 (95% CI: 0.570–0.682).
  • ENTER Random Forest: C-index = 0.744 (95% CI: 0.697–0.790).
  • ENTER Logistic Regression: C-index = 0.745 (95% CI: 0.696–0.791).
  • ENTER XGBoost: C-index = 0.738 (95% CI: 0.687–0.784).
• Calibration: The ENTER Random Forest model demonstrated superior calibration compared to other models, with a slope of 1.02 (95% CI: 0.78–1.32), indicating well-calibrated risk estimates across the spectrum. The PQ-16 cut-off showed poor calibration (slope = 0.39).
• Variable Importance: Beyond PQ-16 derived factors, migration-related factors, parental factors, and deprivation-related factors (from the PPS) were the strongest contributors to risk prediction.
• Clinical Utility: Decision curve analysis indicated that at a risk threshold of 25% (where missing a case is considered three times more harmful than an unnecessary assessment), the ENTER Random Forest model provided a net benefit of 1.8% over the PQ-16 cut-off. This translates to identifying approximately two additional CHR-P individuals per 100 assessments without increasing unnecessary assessments.
• Subgroup Analysis: Discrimination and calibration did not significantly differ by age, gender, country, or ethnicity, though overall performance (Brier score) was poorer for non-binary and Asian participants in the Random Forest model, likely due to sample size constraints.

Significance and Claims
The authors claim that the ENTER tool demonstrates the potential value of scalable, multimodal digital screening for identifying CHR-P individuals in the community. By integrating clinical, cognitive, and environmental data, ENTER addresses the limited detection impact of current symptom-only approaches.

• Improved Triage: The study suggests that multimodal digital screening can support more informative community triage, potentially identifying approximately 50 additional CHR-P individuals in this specific sample compared to the PQ-16 cut-off alone.
• Methodological Contribution: The successful application of a semi-supervised learning framework allows for the incorporation of unlabelled data from large-scale online studies, a common challenge in linking digital screening to gold-standard clinical interviews.
• Implementation Implications: The authors note that while the Random Forest model showed superior calibration, the choice of algorithm has implementation trade-offs regarding false positives versus false negatives. They posit that such tools could reduce access barriers and support resource allocation, particularly for populations underserved by traditional mental health services.

The paper concludes that while ENTER shows promise, external validation and implementation studies are required to establish real-world clinical utility. The authors explicitly state that the preprint has not been peer-reviewed and should not be used to guide clinical practice.