Academic resilience in the Latin America region post COVID-19 pandemic -- an explainable machine learning analysis of its determinants and heterogeneity using alternative definitions

Using explainable machine learning on PISA 2022 data from nine Latin American countries, this study identifies distinct household and school-level determinants of student academic resilience post-pandemic, revealing how factors like digital access, teaching quality, and pandemic disruptions differentially influence disadvantaged students' ability to achieve high performance.

The Gist

Imagine the education system in Latin America as a massive, long-distance race. Before the pandemic, many runners were already starting far behind the starting line because of poverty and lack of resources. Then, the pandemic hit like a sudden, heavy storm that closed the track for nearly a year (an average of 270 days). When the track finally reopened, the gap between the runners who had resources and those who didn't had grown even wider.

However, in every group of disadvantaged runners, there are a few "super runners." These are students who, despite starting in the mud with no shoes and facing the storm, still managed to finish the race with high scores. In the academic world, we call them Academic Resilient Students.

This paper is like a high-tech detective story. The authors wanted to figure out: What makes these "super runners" tick? Why do some disadvantaged students succeed while others struggle?

Here is the breakdown of their investigation, explained simply:

1. The Detective Tool: The "AI Crystal Ball"

Usually, researchers use simple math to find answers. But this team used something cooler: Explainable Machine Learning (specifically a method called SHAP).

Think of this like a super-smart AI crystal ball. Instead of just giving a yes/no answer, the AI looks at thousands of students and says, "Okay, for this specific student, the fact that they have 10 books at home added 5 points to their success score, but working a part-time job took away 3 points." It breaks down exactly which factors push a student up and which ones pull them down.

2. The Two Different Maps

The researchers didn't just use one definition of "success." They drew two different maps to see if the answer changed:

• Map A (The "Hard" Standard): Did the student pass the basic test in Math, Reading, and Science?

  • The Findings: On this map, the most important factors were at home.
  • The Analogy: It's like a garden. If the soil (home) has good tools (digital devices), plenty of water (books), and the gardener (student) is a boy who spends time weeding (doing homework) and isn't too tired from working a second job, the plant is more likely to bloom.
  • Key Takeaway: For this group, what happens inside the house matters most.

• Map B (The "Fair" Standard): Did the student do better than we would expect based on their poverty level and their school's poverty level?

  • The Findings: On this map, the most important factors shifted to the school.
  • The Analogy: Now we are looking at the school as a "training camp." The most important things were the size of the camp, how many computers were connected to the internet, the ratio of teachers to students, and how many teachers were actually certified experts.
  • Key Takeaway: When we account for how tough the home situation is, the quality of the school environment becomes the deciding factor.

3. The Pandemic's Shadow

The study also looked at how the storm (the pandemic) affected the runners.

• Longer Closures = Slower Runners: The longer a school was closed, the less likely a student was to be resilient. It's like if you stop training for a marathon for 10 months; you lose your fitness.
• Barriers to Remote Learning: If a student couldn't get online or had bad internet, their chances of bouncing back dropped significantly.
• The Silver Lining: Students who did manage to attend remote learning classes had a better chance of resilience. It was like having a lifeline thrown to them during the storm.

4. The "Super Runner" vs. The "Struggling Runner" Profiles

The researchers created two character profiles to show the extreme differences:

• The "Super Runner" (High Resilience):

  • Home: Has a private school education, lots of books, many digital devices, and parents who are educated.
  • Habits: Does homework for hours, is curious, empathetic, and satisfied with life.
  • School: Goes to a school with highly certified teachers and lots of tech.
  • Pandemic: Only missed about 240 days of school.

• The "Struggling Runner" (Low Resilience):

  • Home: Parents didn't finish primary school, very few books or devices.
  • Habits: Repeated a grade (failed a year), does very little homework, feels stressed, and works paid jobs to help the family.
  • School: Attends a small, underfunded school with few certified teachers.
  • Pandemic: Missed 300 days of school and had huge barriers to remote learning.

5. The Big Lesson for Policymakers

The paper concludes that there is no "one size fits all" solution.

• If you want to help the poorest students pass the basic tests, fix the home environment (give them books, devices, and reduce the need for them to work).
• If you want to help them excel beyond their circumstances, fix the schools (hire better teachers, ensure internet access, and keep class sizes small).

In a nutshell: This study uses smart AI to tell us that while a student's home life is the foundation, the school is the roof that protects them. To help disadvantaged students in Latin America recover from the pandemic, we need to strengthen both the foundation and the roof.

Technical Summary

Here is a detailed technical summary of the paper "Academic Resilience in the Latin America Region Post COVID-19 Pandemic—An Explainable Machine Learning Analysis of Its Determinants and Heterogeneity Using Alternative Definitions."

1. Problem Statement and Context

The Latin America and Caribbean (LAC) region faces a severe learning crisis exacerbated by the COVID-19 pandemic. Schools in the region remained closed for an average of 270 days (1.42 years of schooling), disproportionately affecting disadvantaged students. The poorest students (bottom decile) lost nearly four times more years of education than the wealthiest students.

Despite these adverse conditions, a subset of students from disadvantaged backgrounds achieves high academic performance. This phenomenon is defined as Student Academic Resilience (SAR). While the concept is well-established, there is a lack of empirical evidence specific to the LAC region regarding the determinants of SAR, particularly in the post-pandemic context. Furthermore, existing literature lacks consensus on how to measure SAR, often relying on single-subject proficiency or fixed thresholds that may not capture the region's specific learning crisis.

Research Questions:

1. What are the principal determinants of SAR across the LAC region, and do they vary by the definition of the SAR indicator?
2. How do these determinants vary across education sub-systems (Private vs. Public; Urban vs. Rural)?
3. What are the specific profiles of students with the highest and lowest probabilities of being academically resilient?

2. Data and Definitions

Data Source:
The study utilizes PISA 2022 data from nine LAC countries: Argentina, Brazil, Chile, Colombia, Dominican Republic, Mexico, Panama, Peru, and Uruguay. The analysis focuses on the bottom 40% of students based on the Economic, Social, and Cultural Status (ESCS) index.

Operationalization of SAR:
The authors propose four alternative binary indicators of SAR to address measurement inconsistencies:

• SAR1 (Fixed Threshold): Students in the bottom 2 SES quintiles achieving Level 2 or above in Math, Reading, and Science simultaneously.
• SAR3 (Inequality-Adjusted SAR1): A subset of SAR1, restricted to students in the top 50% of the most unequal schools (based on the correlation between school performance and family SES).
• SAR2 (Probabilistic Threshold): Based on a 3-level multilevel logit model. It identifies students in the top 40% of the predicted probability of achieving Level 2, after controlling for family SES, school SES, and country random effects.
• SAR4 (Efficiency-Adjusted SAR2): A subset of SAR2 that excludes students from the most "efficient" schools (those with the highest random intercepts), isolating resilience driven by factors other than school-level efficiency.

3. Methodology

The study employs a Machine Learning (ML) approach combined with Explainable AI (XAI) to handle non-linear relationships and complex interactions between covariates.

Model Selection:

• Three models were compared: Logistic Regression (Logit), Neural Networks (NN), and Gradient Boosted Trees (GBT).
• GBT (specifically XGBoost) was selected as the superior model based on performance metrics (AUROC and AUPRC), outperforming Logit by 11–23% and NN by 5–10%. GBT was chosen for its ability to capture non-linearities and interaction effects.
• Training utilized stratified 5-fold cross-validation with an 80/20 train-test split.

Explainability (SHAP):
To interpret the "black box" GBT model, the authors applied the SHAP (Shapley Additive Explanations) method.

• Global Importance: Ranked covariates by average absolute SHAP values to identify leading drivers.
• Local Explanations: Used beeswarm plots and partial dependence plots to visualize the magnitude and direction of feature contributions for individual predictions.
• Sub-sample Analysis: Separate GBT models were trained for Private vs. Public and Urban vs. Rural schools to detect heterogeneity.

4. Key Results

A. Global Determinants (Whole Sample)

The leading determinants of SAR shift significantly depending on the indicator used:

• For SAR1 (Raw Achievement): Determinants are primarily student/family inputs. Key factors include:
  • Digital Devices: Higher number of devices at home.
  • Books: Higher number of books at home.
  • Demographics: Male gender.
  • Behavior: Homework engagement, lower repetition rates (primary/secondary), and lower paid work intensity.
  • Personality: Assertiveness, curiosity, and empathy.
• For SAR2 (Net of SES): Determinants shift to the school domain. Key factors include:
  • School Resources: Ratio of PCs connected to the internet, Student-Teacher Ratio (STR).
  • Teacher Quality: Proportion of certified teachers and professional development (PD) rates.
  • Context: School size and the proportion of disadvantaged students in the school.

B. Impact of the Pandemic (COVID-19 Variables)

• School Closures: There is a strong negative association between the length of school closures and SAR. An increase in closure days from the regional mean (230) to 400 days reduces the odds of resilience by 10–25%.
• Remote Learning Barriers: Higher systemic and material barriers to remote learning significantly decrease the probability of SAR.
• Remote Participation: Conversely, a higher proportion of students in a school engaging in remote learning correlates with higher SAR probabilities.

C. Heterogeneity (Sub-samples)

• Public vs. Private:
  • In Public Schools, determinants like gender, repetition, teacher certification, and internet-connected PCs are more critical.
  • In Private Schools, the ranking of determinants is more similar to the global average, but parental education and teacher PD attendance become more prominent in the SAR2 model.
• Urban vs. Rural:
  • Rural Schools: SAR is more heavily influenced by family inputs (mother/father education, digital devices) and teaching quality (certification, PD) rather than raw resource stock. Gender and homework engagement are more relevant here than in urban settings.
  • Urban Schools: Determinants are more balanced between family and school factors, with STR and connectivity playing a larger role.

D. Local Profiles (Extreme Cases)

• High Resilience Profile: Attends a private school, non-repeater, high number of books/devices, high life satisfaction, low paid work, high empathy, and attends a school with high teacher certification (83%) and PD participation (70%).
• Low Resilience Profile: Repeater (primary and secondary), negative personality traits (low empathy/stress management), low homework, low-educated parents (incomplete primary), and attends a school with high disadvantage and no internet-connected PCs.

5. Key Contributions

1. Regional Focus: First study to specifically investigate SAR determinants for the LAC region using a large, multi-country sample (9 countries).
2. Methodological Innovation: Application of Explainable Machine Learning (SHAP values) to educational resilience, moving beyond traditional linear regression to capture complex, non-linear interactions.
3. Measurement Diversity: Introduction of four distinct SAR indicators (SAR1–SAR4), demonstrating that the choice of definition fundamentally alters the identified policy levers (shifting focus from family inputs to school resources).
4. Post-Pandemic Evidence: Provides empirical evidence linking specific pandemic variables (closure duration, remote learning barriers) directly to the probability of student resilience.

6. Significance and Policy Implications

The paper argues that understanding the drivers of SAR is a crucial mechanism for learning recovery and reducing inequality in LAC.

• Targeted Interventions: Policies should not be one-size-fits-all. For public and rural schools, improving teacher quality (certification/PD) and digital connectivity is paramount. For private/urban contexts, family support and homework structures may be more influential.
• Pandemic Recovery: Reducing school closure durations and mitigating barriers to remote learning are critical for preventing the further erosion of resilience among disadvantaged students.
• Holistic Approach: The findings suggest that lifting disadvantaged students requires a dual approach: strengthening household inputs (books, devices, parental education) and optimizing school environments (teacher quality, resource ratios).

Limitations: The study relies on cross-sectional data (PISA 2022), meaning results show associations rather than causal relationships. Future research should explore intersectionality and causal mechanisms.