Development and Validation of an AI-Based Decision Support Tool (PROMETHEUS) for Predicting Methotrexate Success in Ectopic Pregnancy

Technical Summary: Development and Validation of PROMETHEUS

Problem Statement
Ectopic pregnancy (EP) is a life-threatening condition requiring precise management to minimize morbidity and mortality. While methotrexate (MTX) therapy is a standard non-invasive treatment with success rates approaching 90% in well-selected cases, predicting which patients will respond successfully versus those requiring surgical intervention remains clinically challenging. Current guidelines rely on established criteria, but the heterogeneity of patient presentations often leads to treatment failures, necessitating secondary surgeries. There is a recognized need for more accurate, data-driven tools to optimize treatment selection, reduce unnecessary surgical interventions, and enhance personalized care. While Artificial Intelligence (AI) has shown promise in various medical fields, its application specifically for predicting MTX success in EP is still in its infancy, with existing models often limited in scope or algorithmic diversity.

Methodology
This study employed a retrospective nested cohort design to develop and validate an AI-driven decision support tool named PROMETHEUS (Prediction of Methotrexate for Ectopic Pregnancy Treatment Success).

• Data Source and Cohort: The study analyzed 1,003 electronic medical records from Tepecik Education and Research Hospital (2013–2023). After applying exclusion criteria (missing data, heterotopic pregnancy, ruptured EP, unstable patients, and expectant management cases), a final cohort of 602 confirmed EP cases was utilized.
• Data Partitioning: The dataset was randomly split into a training set (80%, n=482) and a validation set (20%, n=120).
• Feature Engineering: Initially, 24 clinical parameters were recorded, covering patient history, physical examination, ultrasound findings, and laboratory values at admission. To ensure the model's applicability in real-time decision-making, parameters recorded after treatment decisions (e.g., Day 4/7 beta-hCG) were excluded.
• Feature Selection: Through multivariate analysis, two parameters (history of previous ectopic pregnancy and vaginal bleeding) were excluded from the final model due to inconsistent behavior and potential noise, leaving 22 parameters for the predictive model.
• Algorithm Development: The study evaluated 15 different AI algorithms, including Bagging, J48, Random Forest, Logistic Regression, and Neural Networks (MLP). The goal was to classify patients into two binary outcomes: successful MTX therapy versus the need for surgical intervention.
• Evaluation Metrics: Model performance was assessed using accuracy, sensitivity, specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), and the Area Under the Curve (AUC).

Key Contributions

1. Development of PROMETHEUS: The primary contribution is the creation of a user-friendly, mobile- and desktop-compatible decision support tool designed to predict MTX treatment success based on baseline clinical data.
2. Comprehensive Algorithmic Comparison: Unlike many studies that test a limited number of models, this research systematically evaluated 15 distinct AI algorithms to identify the optimal architecture for this specific clinical problem.
3. Identification of the Bagging Algorithm: The study determined that the Bagging algorithm provided the optimal balance of performance metrics, outperforming other models including J48, Decision Table, Random Forest, and Jrip.
4. Clinical Utility Focus: The tool is explicitly designed to complement, not replace, clinical judgment, offering a probability score to assist gynecologists in treatment planning.

Results

• Clinical Characteristics: Significant differences were observed between treatment groups. Patients requiring surgery (either primary or following MTX failure) presented with larger ectopic mass sizes, higher rates of pelvic pain, guarding, rebound tenderness, and free fluid on ultrasound. Laboratory analysis revealed that the surgical groups had significantly lower hemoglobin and lymphocyte levels, and higher white blood cell and neutrophil counts compared to those successfully treated with MTX.
• Model Performance: All five top-performing algorithms demonstrated high accuracy with statistically significant associations (p < 0.001) between predicted and actual outcomes.
  • Bagging Algorithm: Achieved the highest overall accuracy of 93.86%. It demonstrated a sensitivity of 92.7%, specificity of 94.8%, and an AUC of 93.8%.
  • Comparison: The J48 algorithm followed closely with 93.84% accuracy. Traditional statistical models, such as Logistic Regression, achieved lower accuracy (approx. 85%) in this specific dataset compared to the ensemble methods.
• Validation: The model was validated on the reserved 20% cohort, confirming its robustness and discriminatory power.

Significance and Claims
The paper positions PROMETHEUS as an innovative step toward integrating AI into the management of early pregnancy complications. The authors claim that the tool offers a high-accuracy method for predicting MTX success, potentially reducing treatment-related morbidity and streamlining clinical workflows by minimizing unnecessary surgeries.

The study emphasizes that PROMETHEUS is one of the first AI-based systems specifically designed to predict treatment success rather than just diagnosis. By utilizing a broad range of algorithms and focusing on a single, robust classifier (Bagging), the authors argue that the tool achieves a level of reliability suitable for clinical support. The authors modestly note that while the internal validation is robust, the findings are currently limited to a single institution. They state that the tool is not yet deployed in routine practice but serves as a foundation for prospective validation studies and future multi-institutional research to expand its clinical applicability. The ultimate goal is to provide a supplementary layer of evidence-based support to clinicians, particularly in settings where access to senior specialists may be limited.