📊 statistics

Combining BART and Principal Stratification to estimate the effect of intermediate variables on primary outcomes with application to estimating the effect of family planning on employment in Nigeria and Senegal

This paper proposes and validates a novel two-step methodology combining Bayesian Additive Regression Trees (BART) with principal stratification to estimate and generalize the causal effect of modern contraceptive use on empowerment outcomes, demonstrating its application to employment data among urban women in Nigeria and Senegal.

Original authors: Lucas Godoy Garraza, Ilene Speizer, Leontine Alkema

Published 2026-03-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Lucas Godoy Garraza, Ilene Speizer, Leontine Alkema

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Recipe for Better Answers

Imagine you are a chef trying to figure out if a new, secret spice (let's call it "Family Planning") makes your customers (women) more likely to get a job (Employment).

You have a problem: You can't just force half your customers to eat the spice and the other half not to, because that's unethical and impossible. Instead, you have a "promoter" (a radio show or a community program) that encourages some people to try the spice.

The researchers in this paper are trying to answer two tricky questions:

The "Who" Question: Does the spice actually help the people who listened to the promoter and tried it? (And are those people different from everyone else?)
The "Where" Question: If the spice works for that specific group, will it also work for all the women in the country, including those who didn't hear the radio show?

To solve this, they invented a new cooking method called "Prince BART Generalized." Let's break it down.

Part 1: The "Complier" Problem (Who actually listened?)

In the real world, not everyone listens to the radio show.

The "Compliers": Women who didn't use contraception before, heard the show, and then started using it.
The "Never-takers": Women who heard the show but said, "No thanks," and didn't use it.
The "Always-takers": Women who were going to use it anyway, regardless of the show.

The Trap: If you just compare "Users" vs. "Non-Users," you get a messy result. Why? Because women who choose to use contraception might already be more ambitious, educated, or motivated than those who don't. It's like comparing marathon runners to couch potatoes to see if running shoes make you faster. The shoes might help, but the runners were already fit!

The Solution (Principal Stratification):
The researchers decided to focus only on the "Compliers"—the people whose behavior changed because of the program. They treated the radio show as a "nudge" (an instrument) rather than the treatment itself.

The Tool (BART):
To figure out who these compliers were and how the spice affected them, they used a fancy computer algorithm called BART (Bayesian Additive Regression Trees).

Analogy: Imagine trying to predict the weather. A simple linear model is like saying, "If it's 10 degrees hotter, it will rain." That's too simple.
BART is like a team of 200 different meteorologists, each looking at a tiny, specific part of the sky (clouds, wind, humidity, barometric pressure) and making a small guess. They combine all their tiny guesses to make one incredibly accurate prediction.
In this paper, BART looks at hundreds of details (age, education, wealth, religion, past jobs) to figure out exactly which women are the "Compliers" and how much the spice helped them.

The Result: They found that for the women who were nudged into using contraception, their chances of getting a job increased significantly.

Part 2: The "Generalization" Problem (Does it work for everyone?)

Here is the second hurdle. The "Compliers" in the study might be very different from the average woman in the whole country.

Analogy: Imagine you test a new fertilizer on a small patch of soil in a greenhouse. The plants grow huge! But will that fertilizer work on the dry, rocky soil of the entire desert? Maybe not. The "Compliers" in the study might be more educated or live in cities, while the "Target Population" includes rural women with different challenges.

The Solution (The Bayesian Bootstrap):
The researchers wanted to take their findings from the "Greenhouse" (the study cities) and apply them to the "Desert" (the whole country).

They used a massive national survey (the DHS) that has data on everyone in the country.
They used a statistical trick called the Bayesian Bootstrap.
Analogy: Imagine you have a bag of marbles representing the women in the study. You want to know what the bag of marbles looks like for the entire country. You take a handful of marbles from the national survey, look at their colors (covariates like age, education), and then you "resample" them thousands of times to create a perfect, flexible map of the whole country's population.
They then "poured" their specific findings about the "Compliers" onto this new map. They asked: "If we apply our spice to this specific mix of people in the whole country, what happens?"

The Result:

In Nigeria: The effect was even stronger when applied to the whole country than it was in the study group. This is because the study group had a lot of women who were already struggling to find jobs, but the broader population included many women who were just waiting for the right opportunity (contraception) to start working.
In Senegal: The effect was similar in both groups. The population was more uniform, so the study results translated well.

Part 3: The "What If?" Checks (Robustness)

The researchers knew they were making some big assumptions. So, they played "Devil's Advocate."

Analogy: They asked, "What if there's a secret ingredient we didn't measure? What if the cities where the program ran were just naturally better at creating jobs, regardless of the spice?"
They ran simulations to see how much of a "secret ingredient" would be needed to make their results disappear.
The Verdict: They found that for their results to be wrong, there would have to be a massive, invisible force (like a super-powerful city-level motivation factor) that they completely missed. Since that seems unlikely, they are confident their results are real.

The Takeaway

This paper is a masterclass in causal inference. It shows that:

Family Planning works: When women gain control over their fertility, they are much more likely to enter the workforce.
It's not just about the program: The women who joined the program were a specific group, but the benefits of contraception are likely even greater when applied to the broader population of women in Nigeria and Senegal.
New Tools are needed: Old, simple math (linear equations) wasn't good enough to handle the complexity of human behavior. By using "forest of trees" (BART) and "resampling maps" (Bayesian Bootstrap), the researchers could see the truth that simpler methods would have missed.

In short: Giving women the power to plan their families is a powerful economic engine, and this paper provides a sophisticated, reliable way to prove it.

1. Problem Statement

The paper addresses two interconnected challenges in causal inference:

Internal Validity (Estimating the Complier Effect): In observational studies where treatment (modern contraceptive use) is not randomized, but an "encouragement" (Family Planning programs) acts as an Instrumental Variable (IV), researchers aim to estimate the causal effect of the treatment on an outcome (employment). Standard IV estimators (like Two-Stage Least Squares, 2SLS) often rely on linear assumptions that fail to capture complex non-linearities and effect heterogeneity. Furthermore, the effect is only identifiable for "compliers"—individuals whose treatment status changes due to the instrument.
External Validity (Generalization): The "complier" subpopulation identified in a specific study (e.g., specific cities in Nigeria/Senegal) often differs systematically from the broader target population (e.g., all urban women in the country). These differences in covariate distributions (effect modifiers) mean the Local Average Treatment Effect (LATE) estimated in the study sample may not equal the Population Average Treatment Effect (PATE) in the target population.

The authors propose a unified framework to flexibly estimate the complier-specific Conditional Average Treatment Effects (CATEs) and then generalize these results to a broader target population using external survey data, while rigorously accounting for uncertainty and complex sampling designs.

2. Methodology: The "Prince BART Generalized" (PBG) Approach

The authors propose a two-step Bayesian nonparametric approach:

Step 1: Internal Validity (Estimating CATEs in the Source Sample)

Framework: The authors use Principal Stratification (PS) to define latent strata (compliers, never-takers, always-takers, defiers). The goal is to estimate the CATE for compliers: $CATE_c(x) = E[Y(1) - Y(0) | X=x, G=c]$ .
Modeling: Instead of parametric logistic regression, they employ Bayesian Additive Regression Trees (BART).
- Mixture Model: They use a Bayesian mixture model where the latent stratum membership ( $G^*$ ) is unobserved for some units.
- BART Implementation: BART is used to model:
  1. The probability of being a complier given covariates ( $P(G^*=c|X)$ ).
  2. The probability of the outcome given stratum, instrument, and covariates ( $P(Y|G^*, Z, X)$ ).
- Data Augmentation (DA): A Markov Chain Monte Carlo (MCMC) algorithm iterates between imputing the latent stratum membership and updating the BART models.
- Propensity Score Adjustment: To prevent regularization-induced confounding, the estimated instrument propensity score ( $\hat{e}(X)$ ) is included as a covariate in the BART models.
- Overlap Enforcement: Observations with extreme propensity scores (outside [0.1, 0.9]) are trimmed to ensure practical overlap.

Step 2: External Validity (Generalizing to the Target Population)

Goal: Estimate the PATE by averaging the estimated CATEs over the covariate distribution of the target population ( $P_{target}(X)$ ).
Data Source: The target population covariate distribution is derived from large, complex survey data (Demographic and Health Surveys - DHS), which uses stratified two-stage cluster sampling.
Scaled Bayesian Bootstrap (BB):
- Standard Bayesian Bootstrap assumes i.i.d. data, which is invalid for complex survey designs.
- The authors propose a Scaled Bayesian Bootstrap that resamples Primary Sampling Units (PSUs) with weights proportional to their size and survey weights.
- This generates posterior draws of the covariate distribution $P_{target}(X)$ .
Generalization: For each posterior draw of the CATE function and the target covariate distribution, the PATE is calculated as:
$PATE = \int CATE_c(x) dP_{target}(x)$
Missing Covariates: For covariates present in the source study (MLE) but missing in the target survey (DHS) (e.g., specific FP attitudes), the authors use a BART-based multiple imputation strategy to predict these values in the target population based on observed covariates.

3. Key Contributions

Integration of BART and Principal Stratification: The paper tailors the "Prince BART" approach (previously used for truncation-by-death) to instrumental variable settings. This allows for flexible, nonparametric estimation of heterogeneous treatment effects without relying on restrictive linear functional forms.
Generalization with Complex Survey Data: The authors develop a Scaled Bayesian Bootstrap specifically designed to handle the complex sampling design (stratification, clustering, weighting) of national surveys like the DHS. This allows for valid uncertainty quantification when transporting results from a study sample to a national population.
Robustness Framework: The paper provides a comprehensive suite of sensitivity analyses:
- Internal: Sensitivity to unobserved city-level confounding (using a calibrated sensitivity parameter).
- External: Sensitivity to violations of "Included Support" (lack of overlap in covariate space) and "Conditional Transportability" (unobserved effect modifiers).
Interpretability: The use of "surrogate shallow trees" to summarize the complex BART outputs, identifying specific subgroups with distinct treatment effects.

4. Results

Simulation Study

The authors compared Prince BART against 2SLS, $\kappa$ -weighting, and parametric Principal Stratification (PStrata) across four data-generating processes (DGPs) with varying degrees of non-linearity and effect heterogeneity.
Findings:
- In simple linear settings, all methods performed similarly.
- In complex settings with non-linearities and heterogeneity (Designs C and D), Prince BART and $\kappa$ -weighting significantly outperformed 2SLS and PStrata in terms of Mean Squared Error (MSE) and coverage probability.
- Prince BART maintained nominal coverage levels (95%) even when 2SLS and PStrata suffered from severe under-coverage due to model misspecification.

Empirical Application: Nigeria and Senegal

Context: Estimating the effect of modern contraceptive use on employment among urban women using MLE (source) and DHS (target) data.
Complier Effects (Source Sample):
- Nigeria: Prince BART estimated a positive effect of 0.37 (SD 0.13) on employment for compliers.
- Senegal: Prince BART estimated a positive effect of 0.47 (SD 0.16).
- Note: Standard 2SLS and $\k$ -weighting produced estimates outside the logical range of probabilities (e.g., >1) in Senegal, highlighting the instability of ratio estimators in this context.
Heterogeneity: Significant heterogeneity was found. In Nigeria, the largest effects were among married women with primary education or less who had worked previously. The smallest (or negative) effects were among never-married, wealthier women who had not worked.
Generalized Effects (Target Population):
- Nigeria: The PATE was estimated at 0.50–0.53, notably higher than the source sample LATE (0.37). This is because the target population has a higher proportion of subgroups (e.g., married, lower education) that benefit more from contraception.
- Senegal: The PATE (0.48) was similar to the source LATE (0.47), reflecting a more homogeneous population structure.
Robustness:
- Sensitivity to unobserved city-level confounding suggested that confounders would need to be several times stronger than observed residual variability to nullify the results.
- Sensitivity to lack of support (extrapolating to rural women) showed that even assuming zero effect for underrepresented groups, the PATE remained positive and substantial.

5. Significance

Policy Relevance: The study provides robust evidence that modern contraceptive use significantly increases female employment in Nigeria and Senegal. Crucially, it demonstrates that the benefits are likely larger in the general population than in the specific study sample due to the demographic composition of the broader population.
Methodological Advancement: The paper offers a practical, flexible toolkit for researchers dealing with:
- Non-randomized treatments with instruments.
- Highly heterogeneous treatment effects.
- The need to generalize findings from selective samples to national populations using complex survey data.
Addressing Limitations of IV: By moving away from linear IV assumptions, the approach avoids the "ratio estimator" pitfalls (unbounded estimates, high variance) common in traditional IV methods, providing more reliable and interpretable causal estimates for policy-making.

In conclusion, the paper successfully demonstrates that combining Bayesian nonparametric regression (BART) with principal stratification and a modified Bayesian bootstrap for complex surveys provides a powerful, robust framework for estimating and generalizing causal effects in real-world public health and social science settings.