Instructional Quality and PISA Mathematics Trajectories, 2012–2022: Frequentist and Bayesian Conditional Latent Growth Curve Analysis

Technical Summary: Instructional Quality and PISA Mathematics Trajectories, 2012–2022

Problem Statement
This study addresses two distinct but analytically linked questions regarding the "knowledge economy" framework and educational policy. First, it investigates the validity of the "co-movement hypothesis," which assumes that national improvements in education quality, research and development (R&D) investment, and labor market participation are mutually reinforcing over time. While cross-sectional data consistently show these pillars are correlated, it remains untested whether countries that improve on one pillar simultaneously improve on the others longitudinally. Second, the study examines whether national trends in instructional quality—specifically teacher support and disciplinary climate—predict the trajectory of student learning outcomes (PISA mathematics scores) over a decade. This addresses a gap in the literature where within-cycle associations between teaching quality and achievement are well-documented, but their predictive power across cycles at the country level remains unknown.

Methodology
The research employs a secondary cross-national longitudinal design using data from 52 countries across four PISA cycles (2012, 2015, 2018, 2022).

• Data Sources: The study integrates PISA mathematics scores (via the learningtower R package), World Bank Development Indicators (R&D expenditure as % of GDP), and ILO ILOSTAT data (prime-age employment-to-population ratio). Instructional quality variables (Teacher Support Index [TEACHSUP] and Disciplinary Climate Index [DISCLIM]) were derived from PISA student questionnaires.
• Standardization: All outcome variables were z-score standardized using the 2012 wave as the baseline. Time was coded as metric years since 2012.
• Analytical Framework:
  • Primary Analysis: A Bayesian parallel Latent Growth Curve Model (LGCM) was implemented using brms and rstan to simultaneously estimate growth parameters for PISA mathematics, R&D, and employment. This approach allows for the estimation of cross-domain intercept and slope correlations. Weakly informative priors were used to regularize estimates given the small cluster size (N=52 countries).
  • Conditional Analysis: A series of conditional growth models (both frequentist mixed-effects using lme4 and Bayesian equivalents using brms) tested whether instructional quality variables (2012 TEACHSUP level, 2012–2022 change in TEACHSUP, and 2022 DISCLIM) moderated the rate of PISA mathematics change.
• Diagnostics: Convergence was assessed via $\hat{R}$ statistics and effective sample sizes. Model fit was evaluated using LOO-CV (Bayesian) and AIC (frequentist).

Key Results

1. Divergent Trajectories and Decoupling: The population-average slopes for the three domains diverged significantly. PISA mathematics scores declined ($\beta = -0.025$/year), while R&D expenditure ($\beta = +0.020$/year) and prime-age employment ($\beta = +0.064$/year) increased.
2. Lack of Co-movement: While cross-domain intercept correlations were positive and credible (indicating that high-performing countries start with higher levels in all domains), all cross-domain slope correlations were near zero and non-credible. This indicates that growth rates in education, R&D, and employment evolved independently over the decade, supporting the "decoupling hypothesis" rather than the co-movement hypothesis.
3. Instructional Quality Predictors: Conditional models revealed that neither the 2012 level of teacher support, the decade-long change in teacher support, nor the 2022 disciplinary climate significantly moderated the rate of PISA mathematics change. Interaction terms were small ($\beta$ range: 0.003–0.006) and their 95% credible/confidence intervals crossed zero.
4. The Turkey Case: Turkey served as an illustrative case, showing a unique profile where PISA mathematics scores improved significantly (+32.51 points) despite experiencing one of the largest declines in perceived teacher support in the sample. This further underscores the independence of PISA trajectories from teacher support trends at the country level.

Key Contributions

• Methodological: The study demonstrates the utility of Bayesian parallel LGCM for modeling cross-national knowledge economy co-movement. It highlights how Bayesian estimation with weakly informative priors provides robustness against convergence failures and boundary estimates in small-cluster structural equation models, a common issue in cross-national panel research.
• Empirical: It provides the first longitudinal test of the knowledge economy pillar hypothesis, finding that while these pillars cluster cross-sectionally, they decouple longitudinally. This challenges the assumption that composite indices aggregating these pillars provide valid summaries of national development trajectories.
• Reproducibility: The authors provide a fully reproducible pipeline, including the analysis script and processed dataset, integrating data from PISA, the World Bank, and the ILO.

Significance and Claims
The paper claims that the structural independence of knowledge economy pillars suggests that composite rankings (e.g., Global Competitiveness Index) may obscure critical temporal heterogeneity relevant for policy design. Policymakers relying on aggregated indices may miss the fact that a country can improve in one domain (e.g., R&D) while stagnating or declining in another (e.g., education).

Regarding instructional quality, the study concludes that national trends in student-perceived teacher support do not predict decadal PISA trajectories. The authors attribute this null finding to:

1. Temporal Mismatch: TEACHSUP measures a specific cohort's perception of current teachers, which does not align with the decadal outcome trends of different cohorts.
2. Reference-Group Effects: High-performing systems often have students who rate teachers lower, creating a negative cross-sectional correlation that complicates longitudinal interpretation.
3. Aggregation Bias: Country-level means may mask the within-country variation where instructional quality effects are actually operative.

The study cautions educational program developers against using PISA trend data on teacher support as a direct proxy for system-wide instructional quality reforms, noting that a decline in perceived support does not necessarily indicate a decline in teaching quality or a failure of reform. Instead, the findings suggest that the policy levers governing education, research, and employment operate on distinct timescales and through separate institutional channels.