Cross-construct Optimization of TIMSS Mathematics Affective Scales

Technical Summary: Cross-construct Optimization of TIMSS Mathematics Affective Scales

Problem Statement
Large-scale international assessments, such as the Trends in International Mathematics and Science Study (TIMSS), utilize extensive questionnaires to measure students' affective engagement with mathematics. Specifically, TIMSS 2023 assesses three distinct dimensions for Grade 8 students: Students Like Learning Mathematics (SLM), Students Confident in Mathematics (SCM), and Students Value Mathematics (SVM). These scales collectively comprise 26 items. While these dimensions are theoretically linked within the Three-dimensional Model of Attitude (TMA) and Expectancy-Value Theory, they are typically administered and analyzed as independent constructs. This approach imposes a substantial survey burden on students and educators. The study addresses a fundamental measurement question: whether these scales operate as an integrated affective network where items from one dimension can predict classifications in another, thereby allowing for a "cross-construct optimization" that reduces item count without sacrificing predictive accuracy.

Methodology
The study utilized data from 232,970 Grade 8 students across 44 countries from the TIMSS 2023 international database. The analytical approach employed a cross-diagnostic optimization framework adapted from psychiatric screening literature, applied here to educational assessment.

1. Network Modeling: All 26 items from the SLM, SCM, and SVM scales were pooled and analyzed using a Gaussian Graphical Model (GGM). The GGM, estimated via graphical LASSO with Extended Bayesian Information Criterion (EBIC) tuning, modeled the conditional dependence structure among items. Each node represented an item, and edges represented partial correlations controlling for all other items.
2. Bridge Centrality Analysis: Betweenness centrality was calculated for each item to identify "bridge items"—nodes that lie on the shortest paths between other nodes, particularly those spanning different scale boundaries. Items were ranked by this centrality metric.
3. Sequential Item Selection: A sequential selection process was conducted to determine the minimum item set required to predict the official TIMSS three-level classifications (High, Intermediate, Low) for each scale. Items were added one by one in order of their betweenness centrality (bridge-first).
   • Cross-construct Prediction: Crucially, items from one scale were permitted to predict the classification of another scale (e.g., SVM items predicting SLM).
   • Plateau Detection: The process stopped when the predictive performance (measured by macro-averaged AUROC) reached 99% of the highest single-item baseline achievable within the target scale. This threshold allowed for a negligible accuracy loss (<1%) in exchange for a parsimonious instrument.
4. Validation: The procedure was replicated independently for each of the 44 countries to assess the cross-national consistency of the bridge item structure. Performance was evaluated using stratified 80/20 train-test splits and 5-fold cross-validation.

Key Results

• Structural Hub Identification: The GGM analysis revealed that items from the Students Value Mathematics (SVM) scale occupied the most central bridge positions in the affective network. The item "I would like a job that involves using mathematics" (BSBM23E) exhibited the highest betweenness centrality and was the first item selected for predicting all three dimensions.
• Optimized Item Sets:
  • SLM: Required 7 items to reach the performance plateau. Notably, 5 of these (71%) were cross-scale items (from SVM or SCM).
  • SCM: Required 6 items. The first four items added were all cross-scale (from SVM or SLM), with no own-scale items entering until the fifth step.
  • SVM: Reached plateau with only 2 items, both drawn from the SVM scale itself (BSBM23E and BSBM23A).
• The Compact Battery: The SLM and SCM solutions shared a common core of six items. Combined with one additional item unique to SLM, a 7-item battery was derived capable of simultaneously screening all three dimensions. This represents a 73% reduction from the original 26-item instrument.
• Predictive Performance: The 7-item battery achieved high predictive accuracy across all dimensions, with AUROC values ranging from .889 (SVM) to .939 (SLM).
• Cross-National Consistency: While the specific bridge items varied, the predictive utility of the framework was robust. The bridge item consistency index (proportion of top-5 global bridge items reproduced locally) averaged .509. However, predictive accuracy (AUROC) remained high and stable (median .87–.95) across all 44 countries, even in nations with lower structural consistency (e.g., Saudi Arabia, Romania, Turkey, Uzbekistan).

Key Contributions

1. Empirical Characterization of TMA: The study provides the first item-level mapping of the conditional dependence structure across TIMSS affective scales in a large international sample. It empirically identifies the "Vision of Mathematics" (SVM) dimension as the structural hub of the network, extending the theoretical TMA framework by specifying the integrative role of value beliefs.
2. Methodological Innovation: It introduces cross-construct optimization to large-scale educational assessment. By leveraging bridge centrality in GGMs, the study demonstrates that items from one construct can efficiently predict classifications in another, challenging the convention of treating scales as isolated pools.
3. Practical Instrument: The study delivers a validated, compact 7-item instrument that replaces the 26-item official battery. To facilitate adoption, the authors provide open-access resources, including a Python module for batch prediction and a browser-based tool for individual student self-assessment.

Significance and Claims
The paper claims that mathematics value beliefs function as the primary integrative node in the affective network, carrying disproportionate cross-construct information. This finding supports the Expectancy-Value theory's assertion that value beliefs mediate the relationship between competence and interest.

The study concludes that a 7-item battery can effectively screen students across all three TMA dimensions with high accuracy, reducing survey burden significantly. This enables periodic monitoring of student affect (e.g., at the start of each term) without the logistical constraints of full-scale administration. The authors note that while the framework is robust globally, countries with lower bridge item consistency (specifically Saudi Arabia, Romania, Turkey, and Uzbekistan) should treat the battery as a starting point requiring local validation, as cultural and institutional factors may alter the conditional dependencies among affective items. The findings offer a principled approach to efficient affective monitoring in both research and classroom settings.