📄 social_science

Student Dispositions and Instructional Practices as Predictors of Science Achievement: A Multilevel Analysis of PISA 2022 Data from Kosovo

Using a multilevel analysis of PISA 2022 data from Kosovo, this study identifies adaptive instruction as the most powerful predictor of science achievement, explaining 19.1% of between-school variance and surpassing the influence of student-level curiosity and creativity, thereby highlighting the critical role of school-level instructional practices in reducing educational inequality.

Original authors: Fidelis Chinedu Onwunyili, Emmanuel Atiatorme, John Amoako Mensah, Adam Maltese

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fidelis Chinedu Onwunyili, Emmanuel Atiatorme, John Amoako Mensah, Adam Maltese

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the science classroom in Kosovo as a giant, bustling video game world. In this world, every player (student) starts with a unique set of "character stats," and every server (school) runs on a specific type of "game engine" (teaching style). A team of researchers from Indiana University decided to download the latest save file from the 2022 global science tournament (PISA) to figure out what actually makes players level up their science scores.

They didn't just look at the players; they looked at the servers too. Using a super-precise statistical tool called a "multilevel model" (think of it as a microscope that can zoom in on a single player and then zoom out to see the whole server at the same time), they analyzed data from 4,891 students across 100 schools.

Here is the story of what they found, the myths they busted, and the numbers that matter.

The Big Discovery: The "Game Engine" Matters Most

The most surprising thing the researchers found is that the school's teaching style is the heavyweight champion of science scores.

Specifically, a teaching style called "adaptive instruction" is the single most powerful predictor of success. Imagine a teacher who doesn't just read from a script to the whole class. Instead, this teacher is like a master chef who tastes every dish and adjusts the seasoning for each person. They slow down for students who are confused, speed up for those who are bored, and change the recipe to fit different learning styles.

The paper shows that schools where teachers do this kind of "adaptive cooking" have science scores that are 32.67 points higher for every standard jump in how well they adapt. This effect is so strong that it outweighs even the most talented student personalities. In fact, this teaching style explained 19.1% of the differences in scores between schools. It's the biggest lever the system has to pull to fix the gap between high-performing and low-performing schools.

The Player Stats: Curiosity is the Secret Weapon

On the student side, the researchers looked at two special "passive skills": Curiosity and Creativity.

Curiosity: This was the strongest player stat. Students who asked "Why?" and "How does this work?" scored higher. The data shows that curiosity is a reliable booster, adding a significant bump to scores even in schools where the teaching style isn't perfect. It's like having a built-in flashlight that helps you find hidden items even in a dark room.
Creativity: This was also a helpful stat, though slightly less powerful than curiosity. Students who felt their school allowed for original solutions and creative thinking did better. It's a useful tool, but it doesn't carry the same heavy lifting power as curiosity.

What the Paper Explicitly Rules Out (The "Myth Busters")

It is just as important to know what didn't work as what did. The researchers tested some ideas that many people might assume are true, but the data said "nope."

Feedback isn't the magic bullet (in this specific context): You might think that teachers giving lots of feedback (like "Good job!" or "Try again here") would be the key. But in this study, feedback practices did not show a significant link to higher science scores. The numbers were so small they were basically zero. The paper suggests that in Kosovo's schools, the way feedback is given might be different from what international studies expect, or perhaps the "adaptive instruction" (changing the lesson) is so powerful that it drowns out the effect of feedback.
The "Perfect Match" Myth: The researchers wondered if a curious student would do extra well in a school with a perfect adaptive teacher (a "match" between player and server). They tested this cross-level interaction. The result? No significant interaction. It turns out that curious students do well regardless of the school's teaching style, and creative students do well regardless of the feedback style. The "magic match" didn't happen; the stats were uniform across the board.
Immigrant Status: The study checked if being an immigrant student made a difference. After controlling for other factors, the data showed no significant difference in science scores between immigrant and native students.

The Numbers: How Sure Are We?

The researchers didn't just guess; they ran the numbers through a rigorous process.

The Clustering: They found that 32.4% of the difference in science scores was due to which school a student attended, not just who the student was. This proved that looking at schools and students together was necessary.
The Confidence: The findings for curiosity, creativity, and adaptive instruction are statistically significant, meaning the odds of these results happening by random chance are extremely low (less than 1 in 1,000 for curiosity and adaptive instruction).
The Limits: The paper is careful to say this is a "snapshot" (cross-sectional data). It shows a strong connection, but it doesn't prove that curiosity causes the high scores or that adaptive teaching causes them. It's a correlation, not a magic wand. Also, the model only explained 8.0% of the total variation in scores, meaning there are still many other factors (like prior knowledge or family resources) that the study didn't measure.

The Bottom Line

If you want to boost science scores in Kosovo, the paper suggests two main moves:

Train teachers to be "Adaptive Chefs." This is the most powerful tool available right now.
Nurture student curiosity. It's a robust skill that helps students succeed even when the system is tough.

The paper doesn't promise a solved problem, but it points the way. It suggests that by tweaking the "game engine" (teaching) and encouraging the "player stats" (curiosity), the whole system can level up. But it also warns that we can't just rely on feedback or hope for a magical match between student and school; the real work lies in the daily, adaptive choices teachers make in the classroom.

Technical Summary: Student Dispositions and Instructional Practices as Predictors of Science Achievement in Kosovo

Problem Statement
This study addresses the persistent underperformance of Kosovo's education system in science, as evidenced by its mean PISA 2022 science score of 353.57, which is approximately 146 points below the OECD mean. While previous research has examined student-level motivational factors and school-level instructional practices separately, few studies have analyzed their joint influence within a unified analytical framework, particularly in low-performing, post-conflict educational contexts. The study seeks to determine the extent to which student-level dispositional resources (curiosity and creativity) and school-level instructional practices (adaptive instruction and feedback) predict science achievement in Kosovo, and whether school-level practices moderate the relationship between student dispositions and achievement.

Methodology
The research employs a cross-sectional, two-level hierarchical linear modeling (multilevel) design using data from the PISA 2022 assessment in Kosovo.

Sample: The analytic sample consists of 4,891 students nested within 100 schools. This sample was derived from the full PISA 2022 Kosovo dataset (6,027 students in 229 schools) after aggregating teacher data to the school level (requiring a minimum of three teachers per school) and addressing missing data on student dispositional variables via multiple imputation.
Outcome Variable: Science achievement was measured using ten plausible values generated by PISA to account for measurement uncertainty.
Predictors:
- Level 1 (Student): Curiosity (intrinsic motivation to explore) and Creativity (perception of creative opportunities), alongside demographic controls: socioeconomic status (ESCS), gender, and immigrant status.
- Level 2 (School): Aggregated teacher reports on Adaptive Instruction (adjusting pedagogy to diverse needs) and Feedback Practices (frequency and quality of feedback).
Analytical Approach: The study utilized the WeMix package in R to estimate weighted multilevel models, incorporating both student-level (W_FSTUWT) and school-level (W_SCHGRNRABWT) sampling weights to ensure population representativeness. To handle the plausible value structure of the outcome, the mixPV wrapper function was used to estimate models separately across all ten plausible values and pool results using Rubin's rules. Missing data on curiosity and creativity (arising from PISA's rotated booklet design) were addressed using the mice package with predictive mean matching.
Model Selection: Four sequential models were estimated. Model selection was guided by mean AICbar values (averaged across plausible values and imputed datasets) rather than likelihood ratio tests, due to the complexity of pooling test statistics in this context.

Key Results
The analysis yielded the following findings based on the final selected model (Model 2):

Intraclass Correlation (ICC): The unconditional model revealed an ICC of .324, indicating that 32.4% of the total variance in science achievement is attributable to between-school differences, justifying the multilevel approach.
Student-Level Predictors:
- Curiosity was the strongest dispositional predictor of science achievement ( $\beta = .084, p < .001$ ).
- Creativity also demonstrated a significant positive effect ( $\beta = .046, p = .005$ ), though its effect size was smaller than that of curiosity.
- Socioeconomic status (ESCS) and gender (male) were significant positive predictors. Immigrant status was not significant.
School-Level Predictors:
- Adaptive Instruction emerged as the strongest overall predictor in the model ( $\beta = .192, p = .005$ ), explaining 19.1% of the between-school variance. A one standard deviation increase in adaptive instruction was associated with a 32.67-point increase in science achievement.
- Feedback Practices were not a significant predictor ( $\beta = -.022, p = .755$ ) after controlling for adaptive instruction.
Cross-Level Interactions: The interactions between student dispositions and instructional practices (Curiosity $\times$ Adaptive Instruction; Creativity $\times$ Feedback) were not statistically significant, suggesting that the benefits of curiosity and creativity are relatively uniform across schools regardless of instructional environment.
Variance Explained: The final model explained 2.7% of within-school variance, 19.1% of between-school variance, and 8.0% of the total variance in science achievement.

Significance and Contributions
The paper claims three primary contributions to the literature and policy discourse:

Contextual Extension: It establishes curiosity and creativity as significant predictors of science achievement in a persistently low-performing, non-Western educational context, extending prior research that has largely focused on high-performing Western systems.
Instructional Leverage: It identifies adaptive instruction as a more powerful lever for reducing between-school achievement inequality than individual student dispositions. The finding that school-level adaptive instruction explains substantially more variance than student-level factors suggests that improving instructional quality is a critical pathway for systemic improvement in Kosovo.
Methodological Rigor: The study demonstrates the methodological importance of proper plausible value handling and two-level weighting in multilevel PISA analyses. The authors note that incorrect analytical approaches can lead to substantially different ICC estimates and standard errors, which has direct implications for the accuracy of policy-relevant inferences.

Limitations and Modesty of Claims
The authors explicitly note several limitations that qualify their conclusions:

Causal Inference: The cross-sectional design precludes causal inference; associations are correlational.
Measurement Constraints: The school-level weight was derived from the teacher questionnaire file rather than the school questionnaire file. Additionally, the moderate correlation between adaptive instruction and feedback practices ( $r = .65$ ) suggests potential multicollinearity, making it difficult to isolate the independent effect of feedback.
Unexplained Variance: The model explains only 8.0% of total variance. Unmeasured factors such as prior academic achievement, cognitive ability, and structural features like Kosovo's split-shift schooling system (which limits instructional time) likely account for significant portions of the unexplained variance.
Generalizability: The lack of significant cross-level interactions may be due to the relative homogeneity of instructional practices across Kosovo schools, limiting the ability to detect moderation effects.

Consequently, the authors frame their findings as establishing "actionable guidance" and "leverage points" for improvement rather than definitive causal mechanisms, emphasizing the need for future longitudinal research and the inclusion of structural school context variables.