Opening the Baby Black Box: Explainability in Fertility Prediction from Classical Predictors to Social Media Use and Political Orientation

Technical Summary: Opening the Baby Black Box

Problem Statement
This research addresses the challenge of predicting fertility outcomes—specifically whether an individual will have an additional child within a defined three-year timeframe—using machine learning (ML). While traditional demographic research relies on theory-driven, parametric models, the increasing availability of rich longitudinal data allows for data-driven techniques capable of capturing non-linearities and complex interactions. The study is situated within the context of the PREFER Data Challenge, which aims to benchmark predictive models of fertility behavior while emphasizing both accuracy and interpretability to avoid "black-box" predictions. The core problem is to develop a model that not only achieves competitive predictive performance but also provides explainable insights into the factors driving fertility decisions, including underexplored variables like social media usage and political orientation.

Methodology
The authors employed a Random Forest classifier trained on data from the LISS (Longitudinal Internet Studies for the Social Sciences) panel in the Netherlands. The methodology followed a rigorous pipeline:

1. Data and Sample Selection: The training set consisted of 987 individuals aged 18–45 in 2020, with outcome labels indicating childbearing by 2023.
2. Variable Selection: A hybrid approach was used, combining a theory-driven literature review with data-driven selection. Initial candidates included classical correlates (fertility intentions, health, socio-demographics, partnership status) and newer factors (personality, risk attitudes, social media use, political orientation). From an initial pool of 1,344 potential predictors (after filtering for missing values and open-ended questions), LASSO regression with L1 regularization was applied to identify the most informative features, resulting in a final set of 39 predictors.
3. Feature Engineering: Selected variables were transformed into factors to handle categorical data, ordinal scales with sparse categories, and missing values. Missing values were handled by creating specific categories or using forced imputation from past values where appropriate (e.g., for time-invariant traits). Interactions, such as between age and partnership status, were explicitly constructed.
4. Model Training and Tuning: The Random Forest model was tuned using repeated (3 times), stratified 10-fold cross-validation. An entropy-based space-filling design tested 250 hyperparameter combinations (number of trees, depth, minimum samples per leaf, etc.). An efficient grid search via "racing" with ANOVA models was used to maximize the F1 metric.
5. Explainability and Interpretation: To "open the black box," the study applied multiple variable importance (VI) measures:
   • Embedded methods: Impurity-based improvement and permutation-based importance.
   • Model-agnostic methods: Variance-based approaches (FIRM), permutation-based tests on the F1 score, and aggregated Shapley values (mean absolute values).
   • Marginal Effects: Partial Dependence Plots (PDPs) were used to visualize non-linear relationships between predictors and the predicted probability of having a child.
   • Interaction Analysis: The H-statistic was used to quantify global and pairwise interaction strengths between predictors.

Key Results

• Predictive Performance: The model achieved an F1 score of 0.697 on the test set, which is competitive within the PREFER challenge context, though the authors note that fertility prediction remains a difficult task overall.
• Variable Importance: The most influential predictors were classical fertility intentions (timing and number of desired children), partnership status, partner age, and partner satisfaction. These variables consistently ranked highest across all VI metrics.
• Newer Predictors: Social media use (specifically weekly Instagram frequency) and political orientation (social-liberal alignment) were selected as relevant predictors but showed significantly lower importance scores compared to classical factors.
  • Instagram Use: PDPs indicated a small marginal effect, with a notable increase in predicted probability for the "missing" category compared to frequent users.
  • Political Orientation: Showed a nearly flat marginal effect in isolation.
• Interactions: The H-statistic revealed that classical predictors (intentions, partnership-age interaction) had the strongest interaction effects. However, the newer predictors did show meaningful two-way interactions:
  • Instagram use interacted significantly with the partnership-age combination and income.
  • Political orientation interacted with education level, personal values, partnership duration, and gender/employment status.
  • Conditional PDPs suggested that the effect of political orientation is complex and often dominated by education levels or specific demographic subgroups (e.g., university-educated individuals with non-social-liberal views showed higher predicted probabilities).

Contributions
The paper makes three primary contributions:

1. Methodological Framework: It demonstrates a rigorous hybrid approach to feature selection that integrates theoretical insights with LASSO-based reduction of high-dimensional data.
2. Comprehensive Interpretability: It applies a diverse suite of explainability tools (embedded and model-agnostic) to evaluate the roles of both classical and emerging predictors in fertility outcomes.
3. Empirical Insights: It provides evidence that while classical predictors remain dominant, newer variables like social media use and political orientation contribute to predictive accuracy, particularly through complex interactions with traditional demographic factors.

Significance and Claims
The authors claim that their work offers a robust framework for explaining ML models in fertility research, yielding insights into the factors underlying fertility decisions. They emphasize that their findings confirm the central role of fertility intentions and partnership dynamics while highlighting that newer, less-studied variables possess predictive value, particularly in specific contexts revealed by interaction analysis.

Crucially, the authors maintain a modest stance regarding causality. They explicitly state that the results reflect correlational patterns identified by a model optimized for predictive accuracy and should not be interpreted as causal effects. Instead, the study aims to suggest plausible mechanisms and hypotheses for future empirical testing. The work supports the objectives of the PREFER Data Challenge by demonstrating how advanced ML methods, when paired with transparent interpretation tools, can inform demographic research and policy without sacrificing interpretability.