Combining BART and Principal Stratification to estimate the effect of intermediate on primary outcomes with application to estimating the effect of family planning on employment in sub-Saharan Africa

1. Problem Statement

The paper addresses a specific challenge in causal inference: estimating the causal effect of an intermediate variable (treatment uptake) on a primary outcome, when the intermediate variable is subject to confounding.

• Context: The authors focus on the relationship between Family Planning (FP) programs and empowerment-related outcomes (specifically, modern contraceptive use and employment) in Sub-Saharan Africa.
• The Causal Dilemma: While the assignment to an FP program is unconfounded (due to experimental or quasi-experimental design), the actual uptake of FP (the intermediate variable) is not. Uptake and the primary outcome (employment) likely share common causes (confounders) such as socioeconomic status, education, or personal motivation.
• Limitation of Standard Approaches: Traditional regression adjustments or instrumental variable methods often rely on strong parametric assumptions (e.g., linearity, normality) which may not hold in complex, real-world data, potentially leading to biased estimates of the intermediate effect.

2. Methodology: Prince BART

To address these challenges, the authors propose a novel framework called Prince BART, which integrates Principal Stratification with Bayesian Additive Regression Trees (BART).

A. Principal Stratification Framework

The authors utilize the principal stratification approach to define causal effects within subgroups defined by potential responses to the treatment.

• Strata Definition: Women are categorized into latent strata based on their potential FP uptake status under both treatment (program access) and control (no access) conditions.
  • Compliers: Women who would use FP if assigned to the program but would not if not assigned.
  • Always-takers/Never-takers: Women who would use FP regardless of assignment or never use it, respectively.
• Identification: Within the "complier" stratum (or other specific strata of interest), the intermediate variable (FP uptake) becomes unconfounded relative to the program assignment, allowing for the estimation of the causal effect of uptake on the outcome.

B. Bayesian Additive Regression Trees (BART)

To implement this framework without relying on restrictive parametric assumptions, the authors employ BART for two critical modeling tasks:

1. Modeling Stratum Membership: BART is used to estimate the probability of a woman belonging to a specific principal stratum (e.g., the probability of being a "complier") based on observed covariates. BART's non-parametric nature allows it to capture complex, non-linear interactions between covariates that determine stratum membership.
2. Modeling Potential Outcomes: BART is used to model the potential outcomes (employment status) within each stratum. This avoids the need to assume a specific functional form (like a linear relationship) between covariates and the outcome.

C. The Combined Approach (Prince BART)

The "Prince BART" estimator combines these elements to:

• Estimate the distribution of principal strata.
• Estimate the causal effect of the intermediate (FP uptake) on the primary outcome (employment) specifically within the relevant strata.
• Provide full posterior distributions for uncertainty quantification, leveraging the Bayesian nature of BART.

3. Key Contributions

• Methodological Innovation: The paper introduces "Prince BART," a flexible, non-parametric approach to principal stratification. It overcomes the limitations of traditional parametric models (like logistic regression or linear models) which often fail to capture complex data structures in social science applications.
• Handling Confounding in Intermediates: It provides a rigorous solution for estimating the effect of an intermediate variable when that variable is confounded, by leveraging the randomization of the initial program assignment to define unconfounded strata.
• Empirical Application: The method is applied to a significant real-world dataset: quasi-experimental data from a Family Planning program trial in six cities in Nigeria during the 2010s.

4. Results

• Simulation Study: The authors conducted extensive simulations to validate Prince BART. The results demonstrated that Prince BART outperformed alternative modeling approaches based on parametric assumptions, particularly in scenarios involving non-linear relationships and complex interactions among covariates. It showed lower bias and better coverage properties for confidence intervals.
• Empirical Findings (Nigeria Data):
  • When applied to the Nigerian data, Prince BART yielded different estimates compared to parametric alternatives.
  • The study quantified the causal effect of modern contraceptive use on employment among women affected by the FP program.
  • The findings highlighted that the relationship between contraceptive use and employment is heterogeneous and depends heavily on the specific sub-population (stratum) being analyzed. The parametric models likely missed these nuances, leading to potentially misleading policy conclusions.

5. Significance

• Policy Relevance: The study offers more reliable evidence for policymakers in Sub-Saharan Africa regarding the efficacy of Family Planning programs. By accurately isolating the effect of contraceptive uptake on employment, it helps determine whether FP programs successfully empower women economically through the mechanism of reproductive control.
• Statistical Advancement: Prince BART serves as a template for future causal inference research where intermediate variables are of interest. It demonstrates how modern machine learning techniques (BART) can be rigorously integrated into the causal inference framework (Principal Stratification) to relax assumptions and improve robustness.
• Generalizability: While applied to FP and employment, the methodology is applicable to any field where an intermediate behavior (mediated by a randomized intervention) influences a final outcome, and where confounding of that intermediate behavior is a concern.