CLINPREAI: AN AGENTIC AI SYSTEM FOR EARLY POSTPARTUM DEPRESSION RISK PREDICTION FROM MULTIMODAL EHR DATA

This paper introduces ClinPreAI, an autonomous agentic AI system that successfully predicts early postpartum depression risk from multimodal electronic health record data, outperforming traditional AutoML and commercial solutions while democratizing sophisticated clinical modeling for users without machine learning expertise.

The Gist

Imagine you are a doctor trying to predict which new mothers might struggle with depression after their baby is born. This is a tough job because the signs are often hidden, scattered across different types of records, and hard to spot until it's too late.

This paper introduces ClinPreAI, a new kind of "AI detective" designed to solve this mystery. Here is how it works, explained simply:

1. The Problem: The "Needle in a Haystack"

Every year, about 1 in 7 new mothers experiences postpartum depression (PPD). Doctors usually wait until after the baby is born to screen for it, but by then, the mother might already be suffering.

The challenge is that the clues are messy. Some clues are in neat boxes (like "Does she have diabetes?" or "What is her age?"), but the most important clues are often hidden in long, handwritten notes by social workers (like "She seems worried about housing" or "She mentioned feeling guilty"). Traditional computer programs are bad at reading these messy notes, and they often miss the big picture.

2. The Solution: The "Autonomous AI Architect"

Instead of a human data scientist spending months building a computer model, the authors built ClinPreAI. Think of this not as a simple calculator, but as a team of autonomous AI architects.

When you give ClinPreAI a pile of patient data, it doesn't just run a single test. It acts like a self-driving car for data science:

• The Researcher: It reads the data and asks, "What do we know about this patient?"
• The Planner: It draws a blueprint: "Okay, let's try combining the social worker notes with the medical history."
• The Builder: It writes the computer code to build the model.
• The Inspector: It checks its own work. If the code crashes or the model makes silly mistakes, the Inspector fixes it automatically.
• The Teacher: It explains why it made a prediction, so doctors can trust it.

It keeps trying, failing, fixing, and trying again until it finds the best possible way to predict the risk.

3. The Experiment: A High-Stakes Test

The team tested this AI on 4,161 pregnant women who were already in the hospital for complications (like preterm labor or high blood pressure). These women were at higher risk to begin with.

The AI had to predict if these women would score high on a depression test (the EPDS) within six months after giving birth.

The Results:

• The Old Way (Traditional AutoML): Like a standard calculator, it got it right about 64% of the time.
• The Commercial Way (AWS Canvas): Like a generic store-bought tool, it only got it right about 54% of the time.
• The "Zero-Shot" AI (Just asking a chatbot): Like asking a smart friend to guess without seeing the data, it got it right about 51-52% of the time.
• ClinPreAI (The Agent): It got it right about 68% of the time.

The Analogy: If you were trying to find a lost dog in a park:

• The Commercial tool is a person with a map but no eyes; they guess based on where dogs usually are.
• The Zero-shot AI is a person who has never seen the park and just guesses.
• ClinPreAI is a detective who brings a drone, a tracker, and a team of experts, checks the map, looks at the grass, listens for barking, and then makes the prediction. It finds the dog more often.

4. The Big Surprise: The "Recurrence" Trap

Here is the most important, and slightly sad, discovery.

When the researchers asked the AI, "What clues did you use to make your guess?" the AI pointed to one thing above all others: Did this woman have depression or anxiety before she got pregnant?

• The Good News: If a woman had a history of mental health struggles, the AI was very good at flagging her as high risk.
• The Bad News: If a woman had no history of mental health issues, the AI struggled to predict if she would develop depression for the first time.

It's like a weather forecaster who is great at predicting rain if it's already cloudy, but terrible at predicting a sudden storm on a clear day. The current medical records simply don't have enough clues to spot "new" cases of depression that haven't happened yet.

5. Why This Matters

Even though the AI isn't perfect at finding new cases, this paper is a huge step forward for two reasons:

1. Democratization: You don't need to be a math genius or a computer programmer to build these tools anymore. ClinPreAI does the heavy lifting, allowing doctors and nurses to build powerful prediction tools themselves.
2. The "Second Brain" Effect: Even when the AI made a mistake, the clinical experts who reviewed the cases said, "Actually, the AI's reasoning made sense." The AI was good at summarizing long, messy notes into clear summaries. It's like having a super-fast assistant who reads a 50-page file and tells you, "Here are the three most important things to worry about."

The Bottom Line

ClinPreAI is a breakthrough because it proves that AI agents can build their own medical tools, handling messy data better than humans or standard software can. While it currently relies heavily on knowing a patient's past mental health history, it opens the door for a future where AI helps doctors spot the invisible risks of postpartum depression much earlier, potentially saving lives and families from unnecessary suffering.

Technical Summary

1. Problem Statement

Postpartum depression (PPD) affects 10–15% of individuals annually and is a leading preventable cause of maternal mortality. Current screening methods, primarily the Edinburgh Postnatal Depression Scale (EPDS), suffer from low adherence (only ~40% screened), scoring errors, and a lack of early identification.

• The Gap: Traditional machine learning (ML) models for PPD prediction typically rely only on structured Electronic Health Record (EHR) data, ignoring unstructured clinical narratives (e.g., social worker notes) that capture critical psychosocial stressors. Furthermore, existing ML pipelines require significant manual feature engineering and domain expertise, limiting their adoption by clinicians.
• The Challenge: Developing a system that can autonomously integrate multimodal data (structured variables + unstructured text) to predict PPD risk early in pregnancy (during the first antepartum hospitalization) without requiring specialized ML training from the end-user.

2. Methodology

Dataset and Cohort

• Source: Retrospective cohort of 4,161 pregnant individuals hospitalized for medical/obstetric complications at Texas Children's Hospital (2012–2025).
• Outcome: Binary classification based on EPDS score ≥10 within 6 months postpartum (31.0% prevalence).
• Data Modalities:
  • Structured: 27 clinical variables (demographics, obstetric history, psychiatric history, medical history, pregnancy complications).
  • Unstructured: Social worker notes (capturing housing instability, IPV, substance use, etc.) and extracted Human Phenotype Ontology (HPO) codes.

The ClinPreAI System

The core contribution is ClinPreAI, an agentic AI system designed to autonomously design, implement, and evaluate ML solutions. It operates via five specialized modules in an iterative loop:

1. Research Module: Analyzes input data and domain context.
2. Planning Module: Designs the strategy (feature engineering, model selection, metrics).
3. Coding Module: Generates executable code (hosted on AWS SageMaker).
4. Debugging Module: Automatically detects and fixes execution errors.
5. Interpretation Module: Generates performance reports, SHAP values, and clinical insights.

Experimental Design

The study compared ClinPreAI against several baselines across five dataset configurations:

1. Structured Only: 27 clinical variables.
2. Text Only: Social worker notes and HPO terms.
3. Multimodal: Combined structured + text.
4. Latent Multimodal v1: Multimodal + dimensionality reduction/clustering features.
5. Latent Multimodal v2: v1 + domain-specific HPO risk scores.

Baselines Included:

• Traditional AutoML (ClinPreAI AutoML, AWS Canvas).
• Commercial LLMs (Zero-shot Claude Opus/Sonnet, Qwen2.5-7B).
• Hybrid approaches (LLM-Enhanced XGBoost: using LLM to generate pseudo-EPDS scores as features for XGBoost).
• Traditional ML (Logistic Regression, Random Forest, XGBoost).

Analysis Techniques

• Ablation Studies: Systematic removal of mental health history features and patients with prior history to test for de novo risk prediction.
• Unsupervised Learning: PCA, t-SNE, UMAP, and clustering (K-means, DBSCAN) to analyze patient subgroups.
• Error Analysis: Automated identification of decision boundaries, confidence calibration, and demographic disparities.
• Clinical Validation: Independent review by perinatal mental health specialists on misclassified cases.

3. Key Results

Performance Metrics (F1 Score)

• Structured Data: ClinPreAI Agent achieved 0.68 ± 0.03, outperforming traditional AutoML (0.64) and commercial solutions (AWS Canvas: 0.54–0.55).
• Multimodal Data: ClinPreAI Agent achieved 0.65 ± 0.04, matching the specialized LLM-XGBoost hybrid (0.65) and significantly outperforming zero-shot LLMs (Claude Opus: 0.51–0.52).
• Text Only: The Agent achieved 0.63, while zero-shot LLMs hovered around 0.52–0.56. Text-only models generally struggled compared to multimodal approaches.

Key Findings from Ablation & Unsupervised Analysis

• Dominance of Mental Health History: The predictive signal is overwhelmingly driven by prior mental health history. When patients with documented depression/anxiety were excluded, model performance collapsed to near-random levels (F1 ~0.50), indicating current EHR data cannot effectively predict de novo PPD risk.
• No Discrete Clusters: Unsupervised learning revealed that patients do not form distinct "high-risk" vs. "low-risk" clusters. Instead, risk exists on a continuous, heterogeneous spectrum, suggesting binary EPDS categorization may oversimplify the phenotype.
• Error Patterns: Errors were concentrated near decision boundaries. False positives often involved patients with documented risk factors who did not develop PPD (possibly due to successful interventions), while false negatives often involved patients with sparse documentation.

Clinical Validation

• Expert review of misclassified cases showed 75% concordance between the model's reasoning and expert clinical judgment.
• Experts noted that many "errors" were actually due to the inherent difficulty of prospective prediction (intervening events like delivery complications) or limitations in the ground-truth EPDS label (e.g., successful treatment masking symptoms).
• LLM-generated summaries of clinical notes were rated as highly relevant and clinically valid.

4. Key Contributions

1. First Agentic AI in Perinatal Mental Health: Demonstrates the first application of autonomous AI agents for PPD prediction, proving they can match or exceed specialized human-designed models.
2. Democratization of ML: Shows that agentic systems can lower barriers for domain experts (clinicians) to develop robust predictive tools without ML training.
3. Multimodal Integration: Validates that combining structured EHR data with unstructured social worker notes improves prediction, though the agent's ability to autonomously engineer features is crucial.
4. Comprehensive Benchmarking: Provides a rigorous comparison of Agentic AI, AutoML, Commercial Cloud tools, and Zero-shot LLMs, highlighting the limitations of current commercial platforms in handling complex clinical text.
5. Interpretability & Error Analysis: The system provides built-in mechanisms for error clustering and bias detection (e.g., identifying racial disparities in prediction errors), essential for clinical deployment.

5. Significance and Limitations

Significance:

• Paradigm Shift: Moves clinical AI from static, pre-trained models to adaptive, self-correcting agents capable of iterative refinement.
• Clinical Utility: While the model struggles with de novo prediction, it effectively identifies high-risk patients with known histories and provides high-quality clinical summaries that can reduce clinician workload.
• Scalability: The agent architecture offers a reusable template for other clinical prediction tasks involving multimodal data.

Limitations:

• Data Signal: The model relies heavily on prior mental health history; it cannot currently predict PPD in patients without such documentation using available EHR data.
• Ground Truth Issues: The use of a single time-point EPDS score as the gold standard may misclassify patients who developed symptoms later or were successfully treated before screening.
• Technical Variance: The agentic system showed higher variance in performance when handling complex engineered features (Latent Multimodal v1), indicating a need for more stable planning constraints.
• Generalizability: Single-center retrospective study; external validation in diverse populations is required.

Conclusion:
ClinPreAI demonstrates that autonomous AI agents can successfully navigate the complexities of multimodal clinical data to build predictive models that rival specialized human approaches. While the current predictive signal is limited by the availability of mental health history in EHRs, the system's ability to automate experimentation, ensure interpretability, and synthesize clinical narratives represents a significant step forward in deploying AI for perinatal mental health.