Predicting the need for medical care after toxin exposure using SHAP-interpretable gradient boosting

This study demonstrates that XGBoost models, interpreted via SHAP, can accurately and reliably predict the need for medical care following toxin exposure using only initial poison control center call data, offering a promising tool to support expert triage decisions.

The Gist

Imagine you are the operator at a busy 911-style hotline, but instead of fires or crimes, people are calling because they (or their kids) accidentally swallowed a cleaning product, inhaled gas, or took too much medicine.

Your job is terrifyingly important: Do I tell them to stay home and drink water, or do I tell them to rush to the hospital immediately?

If you send them to the hospital when they don't need it, you clog up the ER, waste money, and make waiting times longer for everyone. If you tell them to stay home when they do need help, they could get very sick or even die.

This paper is about building a super-smart digital assistant to help the human operators make that split-second decision.

The Problem: The "Knowledge Gap"

In France (and many places), the experts who know exactly what to do—medical toxicologists—are retiring, and there aren't enough new ones being trained. Meanwhile, the phone lines are ringing off the hook.

Currently, doctors have to guess based on a few rules. If it's a specific poison like "rat poison," they have a rulebook. But what if the caller says, "I don't know what it is, it was just a weird blue liquid"? Or what if it's a rare plant no one has seen before? The old rulebooks fail here.

The Solution: A "Triage Robot"

The researchers built a machine learning model (a type of AI) using 257,000 real phone calls from a poison control center in Lyon. They taught the AI to look at the information available during the first phone call (like: "How old is the person?", "What happened?", "What are the symptoms?", "How much did they take?") and predict the outcome.

They tested two scenarios:

1. The "Go or Stay" Game (Binary): Does this person need to go to a medical facility (Emergency or Urgent Care) or can they stay home?
2. The "Traffic Light" Game (Three-Class): Do they need the Emergency Room (Red light), Urgent Care (Yellow light), or can they Stay Home (Green light)?

How Good is the Robot?

The AI is surprisingly good at this.

• Accuracy: It got the "Go or Stay" decision right about 80% of the time.
• The Winner: The best model was called CatBoost. Think of it as a team of thousands of tiny decision-makers (like a committee of experts) who vote on the answer. They found that the most important things to ask were:
  • The Circumstance: Was it a suicide attempt? (High risk). Was it a cooking accident? (Lower risk).
  • The Symptoms: Is the person having trouble breathing? (High risk). Do they just have a weird taste in their mouth? (Low risk).
  • The Poison: Was it a snake bite? (High risk). Was it a drop of eye drops? (Low risk).

The "Black Box" Problem (And How They Solved It)

Usually, AI is like a black box: you put data in, and an answer comes out, but you have no idea why the AI made that choice. Doctors hate this because they can't trust a machine they don't understand.

The researchers used a special tool called SHAP (which sounds like "shapley," a math concept). Think of SHAP as a magnifying glass that breaks down the AI's decision.

• It shows the doctor: "I sent this person to the ER not just because they took a pill, but specifically because they tried to commit suicide AND they are having trouble breathing."
• It proved the AI isn't making random guesses; it's thinking just like a human expert would.

The "Generalist's Tax"

The paper admits a small flaw. If you build a robot that is an expert on one specific poison (like just Acetaminophen/Tylenol), it might be 95% accurate. But if you build a robot that knows every poison in the world, it might drop to 80% accuracy for any single one.

The authors call this the "Generalist's Tax." They are willing to pay a small price in perfect accuracy to get a tool that works for everything, including the thousands of rare poisons that don't have their own rulebooks yet.

The Bottom Line

This isn't about replacing the human doctors. It's about giving them a co-pilot.

Imagine a flight simulator. The human pilot (the doctor) is still in the chair, but the computer (the AI) is constantly scanning the instruments and saying, "Hey, based on these 250,000 past flights, this combination of symptoms usually means we need to land immediately."

This tool could help:

1. Save lives by catching the rare, dangerous cases faster.
2. Save money by stopping unnecessary trips to the ER for minor issues.
3. Reduce stress for the operators, who can now rely on data to back up their gut feelings.

In short: It's a smart, explainable safety net designed to catch the dangerous cases before they fall through the cracks.

Technical Summary

1. Problem Statement

The study addresses a critical gap in clinical toxicology, particularly in France, where funding for poison control centers (PCCs) is decreasing and the training of new toxicologists has stalled.

• The Challenge: Healthcare professionals (HPs) at PCCs must rapidly triage callers to determine if a patient needs to stay home, visit a non-emergency facility, or go to an emergency department (ED). This decision relies on limited information available during an initial phone call (symptoms, agent, dose, route) without access to biological markers (e.g., blood tests) which are unavailable to the caller.
• Limitations of Existing Tools:
  • Severity Scores (e.g., PSS): Require biological data and expert knowledge, making them unsuitable for initial phone triage.
  • Expert Systems: Rely on rigid, manually curated rules that are difficult to maintain and do not scale across the thousands of possible toxic substances.
  • Substance-Specific ML Models: Existing machine learning models are often trained on single substances (e.g., methanol, acetaminophen), leaving a "long tail" of rare or unknown substances without decision support.
• Goal: Develop a generalist, interpretable machine learning model that predicts the need for medical care using only data available during the initial PCC call, applicable across a wide variety of toxins.

2. Methodology

Data Source and Preprocessing

• Dataset: Retrospective data from the Lyon Poison Control Call Center (2000–2025).
• Sample Size: Started with 612,569 cases; after filtering for mono-intoxications and excluding missing recommendations, 257,652 cases remained for model development.
• Target Variables:
  • Binary Task: Stay at home vs. Go to healthcare facility (Emergency + Non-emergency merged).
  • Ternary Task: Stay at home vs. Non-emergency facility vs. Emergency facility.
• Features: Extracted from initial call logs, including demographics (age, weight, sex), exposure details (agent, estimated amount, route, circumstance), and symptoms encoded via SNOMED CT (Systemized Nomenclature of Medicine – Clinical Terms).
• Handling Missing Data: Missing values were retained as-is, leveraging the native capability of gradient boosting algorithms to handle missingness.

Machine Learning Models

The study evaluated several decision-tree-based algorithms known for handling tabular data:

• Random Forest (RF)
• XGBoost
• LightGBM
• CatBoost
• HistGradientBoostingClassifier (HBC)

Training Strategy:

• Split: 90% training, 10% test.
• Validation: 10-fold stratified cross-validation to optimize decision thresholds and prevent overfitting.
• Class Imbalance: Addressed by adjusting loss function weights and optimizing for the Macro F1-score (balancing precision and recall) rather than simple accuracy, given the skewed distribution of outcomes (63.3% home, 29.0% emergency, 7.7% non-emergency).

Interpretability (Explainability)

To overcome the "black box" nature of ML and ensure clinical trust, the authors employed SHAP (SHapley Additive exPlanations):

• TreeSHAP: Used to calculate feature importance and the direction of influence (positive/negative) for specific feature values.
• Analysis: Identified which specific values (e.g., "Suicide attempt," "Viperidae venom") pushed predictions toward specific classes (Home vs. ED).

3. Key Results

Performance Metrics

• Binary Classification (Home vs. Healthcare):
  • Best Model: CatBoost.
  • Metrics: F1-score = 0.801, ROC AUC = 0.890.
  • All gradient boosting models performed similarly well, significantly outperforming Random Forest.
• Ternary Classification (Home vs. Non-Emergency vs. Emergency):
  • Best Model: LightGBM (slightly ahead of CatBoost).
  • Metrics: Macro F1-score = 0.654, Multi-class ROC AUC = 0.867.
  • Performance was lower than the binary task, reflecting the inherent difficulty of distinguishing between non-emergency and emergency care without biological markers.

Feature Importance

Both built-in model weights and SHAP analysis consistently identified the top three predictors:

1. Circumstance of Exposure: (e.g., suicide attempt, accidental, therapeutic error).
2. SNOMED Codes (Symptoms): (e.g., respiratory distress, coma, syncope).
3. Agent: (The specific toxin involved).

Specific Clinical Insights from SHAP:

• High-Risk Indicators: Suicide attempts, respiratory distress, extrapyramidal syndrome, syncope, and exposure to Viperidae venom or button batteries strongly pushed predictions toward Emergency Department (ED) referral.
• Low-Risk Indicators: Dietary exposures, therapeutic errors (in some contexts), ocular pruritus, nasal irritation, and distinctive tastes pushed predictions toward Home/Exposure site.
• Demographics: Sex and weight had minimal discriminative power compared to clinical and circumstantial features.

Benchmarking

The model was compared against existing specialized models and scoring systems:

• Vs. Organophosphate Model (Hosseini et al.): The generalist model achieved an F1 of 0.77 on a subset of organophosphate cases (vs. 0.91 for the specialized model), illustrating the "Generalist's Tax" (trade-off between breadth and peak precision).
• Vs. PICU Admission Model (Patel et al.): The generalist model significantly outperformed the specialized model across all age groups (Accuracy ~0.84–0.85 vs. 0.53–0.69).
• Vs. Carbon Monoxide Mortality Model (Rose et al.): Achieved a superior AUC of 0.89 compared to the specialized model's 0.70.

4. Key Contributions

1. Generalist Triage Tool: First ML model capable of predicting triage outcomes for any toxin using only initial call data, filling the gap for rare or unknown substances where no specific algorithm exists.
2. Interpretability: Successfully integrated SHAP to provide "post-hoc" explanations, demonstrating that the model's logic aligns with clinical reasoning (e.g., correctly identifying suicide attempts and coma as high-risk factors).
3. Performance vs. Specialization: Demonstrated that while specialized models may have higher peak performance on single toxins, a generalist gradient-boosted model offers superior or comparable performance across diverse datasets and age groups, acting as a "safety net" for the long tail of toxicology.
4. Data Efficiency: Proved that high-accuracy predictions are possible without biological markers, relying solely on structured phone-call data.

5. Significance and Limitations

Significance:

• Resource Optimization: Could help PCCs in France and elsewhere manage the shortage of toxicologists by automating the triage of simple cases, reserving human expert time for complex, high-risk scenarios.
• Standardization: Reduces variability in advice given by different operators, ensuring consistent patient outcomes.
• Clinical Integration: The model is designed for real-time inference (near-instantaneous), making it feasible for integration into PCC software workflows.

Limitations:

• Single-Center Data: Trained on data from Lyon, France; generalizability to other regions with different demographics or recording practices requires external validation.
• Mono-intoxication Focus: The model excludes multi-intoxication cases (12.5% of data), which are often the most severe.
• Target Variable Bias: The model predicts the recommendation of the HP, not the clinical outcome (e.g., death or recovery). The authors argue this is necessary due to the "success paradox" (99% of cases recover), but it risks encoding existing human biases.
• Non-Emergency Class Performance: The model struggled slightly more with the "Non-Emergency" class, likely due to data imbalance and the inherent ambiguity of these cases.

Conclusion:
The study validates that gradient-boosted tree models, when paired with SHAP interpretability, can serve as a robust, clinically relevant decision-support tool for poison control triage. It offers a scalable solution to the growing demand for toxicology expertise, provided it undergoes prospective validation and multi-center testing.