📊 statistics

Estimating Treatment Effects for Depression in Longitudinal Therapy Switching Settings

Using a proprietary longitudinal MDD clinical trial dataset, this study evaluates eight causal estimators to predict individualized treatment effects for depression medication switching, finding that Causal Forest provides the most robust estimates and reveals that while dose intensification is generally beneficial, specific patient subsets may paradoxically respond better to lower-intensity regimens.

Original authors: Xinyu Qin, Martin Katzman, Alexandria Greifenberger, Elssa Toumeh, Sachinthya Lokuge, Tia Sternat, Ruiheng Yu, Lu Wang

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Xinyu Qin, Martin Katzman, Alexandria Greifenberger, Elssa Toumeh, Sachinthya Lokuge, Tia Sternat, Ruiheng Yu, Lu Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but the clues are messy and the suspects are lying. This is the world of causal inference, a branch of science dedicated to figuring out what actually causes something to happen, rather than just noticing that two things happen at the same time. In medicine, this is the difference between knowing that "people who take medicine A get better" and knowing that "medicine A makes people get better." The tricky part is that doctors don't pick treatments randomly; they choose based on how sick a patient is, their history, and other factors. This creates a "confounding" problem, where it looks like the medicine is working (or failing) simply because of who was chosen to take it. To solve this, scientists use counterfactuals—a fancy word for asking, "What would have happened if this patient had taken a different medicine instead?" Since we can't travel back in time to see that alternate reality, we have to use math and computer models to guess the answer. Getting this right is crucial because if we guess wrong, we might recommend a treatment that doesn't help, or worse, one that hurts.

Now, picture a patient with depression visiting their doctor. They start on one medication, but it doesn't work well or has side effects, so the doctor switches them to another. This happens over and over, creating a winding path of different drugs. The big question this paper tackles is: How can we predict, for a specific person, which switch will actually help them feel better next time? The authors took a massive, real-world dataset of patients with Major Depressive Disorder (MDD) who were switching between six different antidepressant treatments. They treated this like a prediction game: given a patient's current symptoms and history, could a computer model guess what their depression score would be if they stayed on their current drug versus if they switched to one of the other five options?

The researchers tested eight different computer "guessers" (estimators) to see which one was the best detective. They found that a method called Causal Forest was the clear winner. Think of Causal Forest as a team of many small, honest decision-makers. Instead of trying to memorize the whole story at once, they split the data into tiny pieces, look at the patterns in each piece, and then combine their answers. This "honest splitting" prevents the model from cheating by just memorizing the data. The results showed that Causal Forest was significantly better at predicting the right outcome than the other seven methods, including some very popular ones called "meta-learners." In fact, those other methods were so bad at this specific task that they performed worse than just guessing the average result for everyone!

When the team looked at the actual predictions, they found some surprising truths. If you just looked at the raw data without doing any "detective work" to fix the confounding, it looked like switching to certain drugs would drop a patient's depression score by huge amounts (around 6 or 7 points). But once the Causal Forest model adjusted for the fact that sicker patients were more likely to switch, the predicted benefits were much more modest—usually a drop of only 0.3 to 0.9 points. This suggests that while switching can help, the magic isn't as dramatic as a quick glance at the numbers might suggest.

Interestingly, the model found that for most patients, increasing the dose of a drug (like going from 60mg to a higher dose of duloxetine) was generally helpful. However, there was a twist: for some specific groups of patients, a lower-intensity regimen actually worked better than a high-intensity one. The study also confirmed that the "honest" way of building the tree models (splitting the data so the model doesn't peek at the answer while learning) was critical; without it, the model's accuracy dropped. While the paper doesn't claim this is a cure-all, it suggests that using these advanced, honest computer models could help doctors make smarter, personalized decisions about when to switch a patient's medication, moving beyond simple guesses to data-driven choices that actually account for the messy reality of human health.

Technical Summary: Estimating Treatment Effects for Depression in Longitudinal Therapy Switching Settings

Problem Statement
The paper addresses the challenge of estimating individualized treatment effects (ITEs) for patients with Major Depressive Disorder (MDD) in longitudinal settings where treatment switching occurs. In routine care, clinicians frequently switch antidepressant regimens due to inadequate response or adverse effects. Estimating the counterfactual outcome of an alternative treatment at the next visit is difficult due to three primary factors:

Confounding: Treatment assignment is driven by patient characteristics (e.g., symptom severity, comorbidities) that also influence outcomes.
Time-Varying Selection: Switching decisions depend on prior responses and evolving clinical history, causing systematic differences between patients who switch and those who continue.
Unobserved Counterfactuals: Only the factual outcome of the assigned treatment is observed; outcomes under alternative treatments are missing.

Existing methods often focus on binary treatments, static settings, or synthetic data, leaving longitudinal switching with multiple treatment options understudied. Furthermore, standard evaluation based solely on factual prediction accuracy can be misleading under confounding, as models may memorize selection-driven patterns rather than learning true counterfactual relationships.

Methodology
The authors utilize a proprietary longitudinal dataset from an MDD clinical trial program containing 6,430 decision samples derived from 836 unique patients. Each sample links covariates and the treatment at visit $v$ to the Hamilton Depression Rating Scale (HAMD-17) total score at visit $v+1$ . The dataset covers six treatment categories, including various dosages of duloxetine, venlafaxine, and paroxetine.

The study formulates a "next-visit counterfactual prediction" task and benchmarks eight estimators:

Primary Method: Causal Forest (CF), extended to handle $K=6$ treatments via a baseline-referenced pairwise approach. The most frequent treatment (duloxetine 60 mg QD) serves as a baseline. $K-1$ binary forests estimate the effect of each target treatment against the baseline, and a separate XGBoost model predicts the baseline outcome. These are combined to reconstruct potential outcomes for all treatments. CF employs "honest splitting" to prevent adaptive overfitting.
Baseline Methods: Seven other estimators including Meta-learners (S-learner, T-learner, X-learner, DR-learner), a residual-based approach (R-learner), an efficient plug-in learner (EP-learner), and a knowledge-graph enhanced estimator (KG-TREAT). All baselines use XGBoost as the base learner and are adapted to the multi-treatment setting using the same baseline-referenced construction.

Evaluation Protocol
The authors propose a unified evaluation protocol comprising four dimensions:

Factual Prediction Error: Measured via RMSE, MAE, MSE, and Expected Calibration Error (ECE).
Treatment Effect Calibration: Uses an observational calibration proxy ( $\Delta_{OCP}$ ), defined as the mean outcome difference between treatment groups in the test set, to compute ATE calibration error. This serves as a consistency signal rather than a causal ground truth.
Decision Quality: Measured by the Inverse-Propensity-Weighted (IPW) policy value, where a lower value indicates better predicted treatment selection.
Per-Treatment Effects: Comparison of predicted ATEs against observational differences.

Experiments were conducted across ten random seeds with patient-level splitting to prevent data leakage, using 95% bootstrap confidence intervals for uncertainty quantification.

Key Results

Performance Superiority of Causal Forest: CF achieved the best performance across all metrics, with an RMSE of $4.03 \pm 0.12$ . It significantly outperformed all baselines ( $p < 0.001$ for meta-learners, $p = 0.032$ for EP-learner).
Failure of Meta-Learners: S, T, X, and DR-learners exhibited "catastrophic failure," with RMSEs ranging from 14.75 to 28.20 (worse than predicting the unconditional mean). They failed to distinguish counterfactual learning from memorizing selection-driven outcome patterns, showing negative calibration slopes and near-zero predicted treatment effects.
Robustness of CF: CF demonstrated superior calibration (ECE $0.35$, slope $\approx 1.0$ ) and decision quality (IPW value $2.05$). Ablation studies confirmed that "honest splitting" was critical, as disabling it increased RMSE by 4.2%.
Treatment-Specific Findings:
- Confounding Adjustment: Crude observational comparisons suggested large symptom reductions (e.g., 6–7 HAMD-17 points) for switching to paroxetine or higher-dose duloxetine. After causal adjustment, CF estimated modest benefits (0.3–0.9 points), highlighting the substantial bias in unadjusted observational data.
- Specific Switches: CF predicted moderate benefits for switching to duloxetine 40 mg BID, venlafaxine 150–225 mg titration, and paroxetine 20 mg QD relative to the baseline.
- Counterintuitive Finding: For specific patient subsets, a lower-intensity regimen (duloxetine 40 mg BID) was predicted to outperform a higher-intensity alternative, challenging the assumption that higher doses are always superior.

Significance and Claims
The paper claims to provide:

A Unified Evaluation Protocol: A framework for systematically comparing treatment effect estimators in switched observational settings where counterfactuals are unobserved, utilizing factual error, proxy-based calibration, and policy value.
A Comprehensive Benchmark: Evidence that tree-based methods with honest splitting (specifically CF) substantially outperform meta-learners in multi-treatment switching scenarios. The study extends CF to multiple treatments via a baseline-referenced construction, providing implementation details for replication.
Empirical Evidence for Clinical Decision Support: The findings offer prospectively testable candidates for clinical decision support. The analysis demonstrates that while crude comparisons overstate gains, confounding-adjusted estimates yield modest, actionable magnitudes. The identification of specific switch directions (e.g., dose intensification generally being beneficial, with noted exceptions) provides data-driven guidance for treatment selection in MDD care.

The authors conclude that while the study provides robust retrospective evidence, prospective validation is required to confirm whether CF-guided recommendations improve real-world decision-making and patient outcomes.

Technical Summary: Estimating Treatment Effects for Depression in Longitudinal Therapy Switching Settings

More like this