Identifying the post-pandemic determinants of low performing students in Latin America through Interpretable Machine Learning methods

Using interpretable machine learning on 2022 PISA data from ten Latin American countries, this study identifies that student underachievement is primarily driven by a combination of individual socioeconomic factors (such as poverty, minority language status, and child labor) and systemic school-level disadvantages (including poor ICT infrastructure and low teacher certification), with repetition and household wealth emerging as consistent predictors across the region.

The Gist

Imagine Latin America's education system as a massive, complex race. For years, many runners have been struggling to keep up. Then, the pandemic hit like a sudden, heavy storm, knocking many off their feet and leaving them far behind.

This paper is like a team of detectives using a super-smart, futuristic microscope (Machine Learning) to figure out exactly why some students are stuck at the very bottom of the race, unable to catch up even after the storm has passed.

Here is the breakdown of their investigation in simple terms:

1. The Mission: Finding the "Stuck" Runners

The researchers didn't just look at the average student. They zoomed in on the two groups in the deepest trouble:

• The "Lost" Group: Students who can't even solve basic problems (Level 0).
• The "Struggling" Group: Students who can do a little bit but aren't ready for real-world challenges (Level 1).

They wanted to know: What specific things are holding these kids back, and what would it take to help them move up a step?

2. The Tool: The "Magic Crystal Ball" (Machine Learning)

Instead of asking students to fill out a simple survey and guessing the answers, the researchers used Machine Learning. Think of this as a super-smart robot that can look at millions of data points at once and find hidden patterns that humans would miss.

But there's a catch: usually, these robots are "black boxes"—they give an answer but won't explain how they got there. To fix this, the researchers used a special technique called SHAP.

• The Analogy: Imagine a group of 10 friends trying to guess the weight of a watermelon. The robot doesn't just say "It's heavy." It says, "The watermelon is heavy because it's green (contribution: +2kg), because it's big (contribution: +5kg), but because it's hollow inside (contribution: -1kg)."
• The Result: This allowed the researchers to see exactly which factors were pushing a student down and which were helping them up.

3. The Suspects: Who is to Blame?

The investigation revealed a "perfect storm" of bad luck for the students at the very bottom. If you were to draw a picture of the student most likely to be stuck at the bottom, they would look like this:

• The Home Front: They come from a very poor family, speak a minority language at home (like an Indigenous language), and have zero digital devices (no tablet, no computer).
• The Student's Life: They have to work for money half the week to help their family, they have repeated a grade before (stuck in the same class), and they feel like they don't belong at school.
• The School: They attend a school that is like a house with a leaking roof. It has bad teachers (many aren't certified), no internet, broken computers, and a scary or unsafe atmosphere.

The Metaphor: Imagine trying to run a race while wearing heavy boots (poverty), carrying a backpack full of rocks (working for money), and running on a track made of mud (bad school climate). No wonder they can't catch up!

4. The Differences: Bottom vs. Low

The researchers found that the "struggling" group (Level 1) and the "lost" group (Level 0) have slightly different problems:

• For the "Lost" (Level 0): The biggest hurdles are repeating a grade, speaking a different language, and having no digital devices.
• For the "Struggling" (Level 1): The biggest hurdles are working for money, not having enough homework time, and their mother's education level.

5. The Country-by-Country Clues

The study looked at 10 Latin American countries. While the general problems were similar everywhere, there were some local flavors:

• In Mexico, the "lost" students were often working almost the whole week and missing months of school.
• In Peru, the "lost" students were often Indigenous and attending schools with almost no technology.
• In Panama, the mothers of the "lost" students often hadn't even finished primary school.

6. The Solution: How to Help Them Up

The paper suggests that we can't just use a "one-size-fits-all" solution. We need to change the specific things holding them back:

• For the "Lost" (Level 0): We need to stop them from repeating grades, give them financial support so they don't have to work, and provide them with laptops and books. We also need to make schools feel safer and more welcoming for Indigenous students.
• For the "Struggling" (Level 1): We need to help them do more homework (maybe by offering after-school classes), support their families so they don't need the kids to work, and improve the quality of teaching so they can actually learn the basics.

The Big Takeaway

The pandemic didn't just pause school; it broke the ladder for the poorest kids. This paper uses high-tech detective work to show us exactly where the rungs of that ladder are missing.

The lesson is simple: You can't fix the problem by just telling kids to "try harder." You have to fix the poverty, the lack of technology, and the bad school environments that are holding them down. If we fix those specific things, we can help these students climb back up the ladder.

Technical Summary

Here is a detailed technical summary of the paper "Identifying the Post-Pandemic Determinants of Low Performing Students in Latin America Through Interpretable Machine Learning Methods."

1. Problem Statement

The paper addresses the severe learning crisis in Latin America (LAC), exacerbated by the post-pandemic context. Key issues include:

• High Prevalence of Learning Poverty: Approximately 75% of LAC students are "low performers" in mathematics (below Level 2), and over 55% lack basic reading skills.
• Post-Pandemic Disruption: The region experienced the longest school closures globally (avg. 269 days), leading to an estimated loss of 0.9–1.1 years of schooling, disproportionately affecting disadvantaged students.
• Methodological Gap: Existing literature often aggregates student performance or focuses on richer education systems. There is a lack of granular analysis distinguishing between the "bottom performers" (Level 0: no competency) and "low performers" (Level 1: below basic competency) to identify specific, actionable determinants for targeted interventions.
• Black Box Limitation: While Machine Learning (ML) offers superior predictive power, its lack of interpretability has limited its adoption in educational policy research, particularly in the Global South.

2. Methodology

The study employs a robust Explainable Machine Learning (XAI) framework using 2022 PISA data from 10 LAC countries (Argentina, Brazil, Chile, Colombia, Dominican Republic, Mexico, Panama, Peru, Paraguay, Uruguay).

Data and Variables

• Sample: Two binary classification samples were constructed:
  1. Bottom Performers: Level 0 (no competency) vs. Level 1. ($N \approx 16,236$)
  2. Low Performers: Level 1 vs. Level 2 (basic competency). ($N \approx 9,484$)
• Covariates: A comprehensive set of student/family variables (e.g., SES, repetition, digital devices, paid work, parental education) and school variables (e.g., ICT infrastructure, teacher certification, school climate, autonomy).

Modeling Approach

1. Base Learners: Five binary classification models were trained and optimized via hyperparameter tuning (Grid Search) and 5-fold stratified cross-validation:
   • Logistic Regression (Logit)
   • Logistic Regression with Lasso penalty (LogitLasso)
   • Random Forest (RF)
   • Gradient Boosting (GB)
   • Neural Networks (NN)
2. Stacking Ensemble: A Stacked Generalization approach was used to combine the five base models. A meta-model (linear regression with non-negative weights summing to 1) was trained on the cross-validated predictions of the base learners to produce the final ensemble. This leverages the strengths of linear models (interpretability) and tree-based/non-linear models (complexity handling).
3. Interpretability (SHAP): To overcome the "black box" nature of the ensemble, Shapley Additive Explanations (SHAP) were applied.
   • Global Explanations: Average absolute SHAP values ($I_j$) were calculated to rank the overall importance of features across the entire population.
   • Local Explanations: The study identified students at the extremes of the SHAP contribution distribution (highest probability of being Level 0 vs. Level 1) to profile specific "at-risk" student archetypes.
   • Interaction Effects: SHAP interaction values were used to analyze how factors like SES, gender, and ICT stock compound each other.

3. Key Contributions

• Granular Segmentation: First study to disaggregate LAC learning poverty into "bottom" (Level 0) and "low" (Level 1) performers, revealing distinct determinants for each group.
• Regional Scale with IML: Expands the application of Interpretable Machine Learning (IML) to the Global South, moving beyond single-country studies or developed nations.
• Post-Pandemic Evidence: Provides the first large-scale analysis of learning determinants using PISA 2022 data, capturing the specific impact of the pandemic era.
• Policy-Ready Profiling: Moves beyond variable ranking to construct detailed "student profiles" (e.g., specific combinations of covariates) that define the most at-risk students, offering a blueprint for Early Warning Systems.

4. Key Results

A. Global Determinants (Feature Importance)

• Top Drivers for Bottom Performers (Level 0 vs. 1): Primary grade repetition, school climate, teacher support for well-being, and speaking a minority language are the strongest predictors.
• Top Drivers for Low Performers (Level 1 vs. 2): Household wealth (SES), family educational support, and digital device ownership become more dominant. Paid work and homework intensity also gain significance.
• School Factors: ICT infrastructure (PCs per student), school size, and teacher certification rates are critical differentiators.

B. Local Explanations (Student Profiles)

The study constructed profiles for students with the highest probability of being at the bottom of the distribution:

• The "Level 0" Profile: A student who speaks a minority language, has repeated a grade, possesses no digital devices, comes from a very poor family (SES index < -2), works for payment half the week, and attends a school with a bad climate, weak ICT infrastructure, and only ~33% certified teachers.
• The "Level 1" Profile (Transition): A student from a wealthier family, with books and devices, no grade repetition, no paid work, and attending a private or autonomous school with high teacher certification and connectivity.

C. Country-Level Heterogeneity

While primary repetition, household wealth, and ICT inputs appear in the top 10 determinants for at least 8 of the 10 countries, specific local nuances exist:

• Mexico: Paid work (nearly full week) and long-term school absence are critical drivers.
• Peru/Paraguay: Indigenous background and extremely low teacher professional development (PD) attendance (16% in Paraguay) are prominent.
• Chile: Skipping school and low maternal education are specific drivers for the lowest performers.

D. Intersectionality

• Gender & SES: Poverty impacts male students more severely regarding the risk of falling to Level 0, whereas female students are more prone to stagnating at Level 1 when family educational support is low.
• Repetition & Wealth: The negative effect of grade repetition is compounded significantly for students from poor households.

5. Significance and Policy Implications

The paper argues for a two-way intervention strategy:

1. Targeting the Bottom (Level 0): Policies must address structural barriers: financial support to reduce child labor, bilingual education for indigenous/minority groups, and cash transfers to alleviate poverty.
2. Moving Up (Level 1 to 2): Focus on educational inputs: increasing digital device access, improving school climate, and boosting teacher quality (certification and PD).
3. Infrastructure: The study highlights that ICT access is a sturdy barrier; only 46% of children in the region live in households with internet. Investment here is medium-term but critical.
4. Methodological Shift: The use of SHAP values allows policymakers to move from generic "average" interventions to precision targeting of specific student archetypes, potentially integrating these models into "Early Warning Systems" for real-time intervention.

Conclusion: The study demonstrates that learning poverty in LAC is not monolithic. By using interpretable ML, the authors provide a granular map of the drivers pushing students to the very bottom of the learning distribution, offering a data-driven foundation for equitable post-pandemic recovery policies.