📄 social_science

Digital Inequality, ICT Pedagogical Hindrances, and Student Computer and Information Literacy: A Plausible-Value-Based Multilevel Analysis with Design-Based Sensitivity Checks Using ICILS 2023

Using a multilevel analysis of ICILS 2023 data from 12 education systems, this study finds that student computer and information literacy is primarily driven by socioeconomic factors and pedagogical barriers rather than school-level AI or ICT expectations, with implementation hindrances showing a consistent negative association.

Original authors: Sukanya Chaemchoy, Thanakrit Supsin

Published 2026-06-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sukanya Chaemchoy, Thanakrit Supsin

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: What Are They Looking For?

Imagine the world of schools is undergoing a massive renovation. Everyone is talking about "digital transformation"—bringing in new tools like Artificial Intelligence (AI), computers, and high-speed internet.

The researchers asked a simple question: Does the "vibe" of a school regarding these new tools actually change how good students are at using computers?

Specifically, they wanted to know if a school principal's expectations about AI (e.g., "This is great!" or "This is dangerous!") or the obstacles teachers face (e.g., "We don't have enough time" or "The internet is slow") directly translate into better or worse computer skills for the students.

They looked at data from over 24,000 eighth-graders across 12 different countries (like Chile, Korea, Sweden, etc.) to find the answer.

The Main Characters in the Story

To understand the results, think of the school environment as a garden.

The Students (The Plants): Some plants are naturally stronger than others. In this study, "stronger" means having a higher socioeconomic background, more experience with computers at home, and having internet access.
The Principal's Expectations (The Weather Forecast): This is what the school leader thinks about the future. Do they believe AI is a storm to be feared or sunshine to be welcomed?
The Gardening Tools & Barriers (The Soil and Fences): This represents the actual reality of teaching. Do teachers have the time to use technology? Are the computers working? Are there fences (barriers) stopping them from planting new digital seeds?

What Did They Find?

The researchers used a very careful statistical method (like a high-precision microscope) to separate the effects of the students' backgrounds from the school's environment. Here is what they saw:

1. The "Family Background" Factor is the Strongest Force

The most powerful predictor of a student's computer skills wasn't the school's policy or the principal's attitude. It was the student's own background.

The Analogy: Imagine two plants. One is a seed from a rich, well-fertilized garden (high socioeconomic status, lots of computer experience at home). The other is a seed from a rocky, dry patch. Even if you put both in the exact same school garden, the first plant will likely grow taller and stronger.
The Result: Students who already had computers at home, more experience with them, and came from wealthier families scored much higher. This "digital inequality" was the biggest driver of the results.

2. The Principal's "Weather Forecast" Didn't Change the Plants

The researchers checked if a principal's excitement or worry about AI (the "Expectations") made a difference.

The Analogy: Imagine a principal standing on the roof shouting, "AI is going to save us!" or "AI is going to destroy us!" The researchers found that shouting these expectations didn't actually make the plants (students) grow taller or shorter.
The Result: Whether a principal was optimistic or pessimistic about AI had no direct link to how good the students were at using computers.

3. The "Fences" (Barriers) Did Matter, But Gently

The researchers looked at ICT Pedagogical Hindrances (HPED). This is a fancy way of saying "obstacles that stop teachers from using technology in class."

The Analogy: Imagine a garden where the gardener wants to plant new seeds, but there are broken tools, a locked gate, or no water hose. The researchers found that schools with more of these "fences" and broken tools tended to have students with slightly lower computer skills.
The Result: This was the only school-level factor that showed a negative pattern. However, the effect was marginal (weak). It's like saying, "Having a broken shovel makes gardening a little harder," but it doesn't mean the garden will fail completely. It suggests that removing obstacles is more important than talking about the future.

The "Magic" of the Study's Method

The researchers were very careful not to get tricked by the data.

The "Country Filter": They realized that schools in different countries are very different. A school in Denmark is not the same as a school in Uruguay. They used a statistical "filter" (Country Fixed Effects) to compare schools within the same country, rather than comparing Denmark to Uruguay directly. This ensured they were seeing real school effects, not just national differences.
The "Double-Check": They ran the numbers twice using two different mathematical engines (Mplus and IDB Analyzer) to make sure the results weren't a fluke. Both engines agreed: The barriers (fences) were the only school thing that mattered, and even then, only a little bit.

The Bottom Line

If you want to improve students' computer and information skills (CIL), this study suggests:

Don't just talk about AI: A principal's excitement or fear about Artificial Intelligence doesn't directly boost student skills.
Fix the "Fences": The biggest school-level issue is the practical barriers. If teachers are blocked by lack of time, bad internet, or no support, student skills suffer.
Acknowledge the "Soil": The strongest factor remains the student's background. Students from wealthier homes with more computer experience start with a huge advantage.

In short: The study concludes that socioeconomic inequality and practical implementation barriers are the real drivers of digital skills, not the school's "AI climate" or the principal's expectations. The "vibe" of the school matters less than the actual tools and time available to use them.

Technical Summary: Digital Inequality, ICT Pedagogical Hindrances, and Student Computer and Information Literacy

Problem Statement and Research Context
The study addresses the evolving debate in school digital transformation, which has shifted from infrastructure access to questions of implementation, pedagogy, and student digital competence. Specifically, the research investigates how school-level digital conditions—namely principals' expectations regarding generative AI and ICT, and barriers to pedagogical ICT use—relate to students' assessed Computer and Information Literacy (CIL).

A critical gap identified is the lack of clarity regarding whether school-level digital variables (such as AI expectations) are directly associated with student CIL after controlling for student background, school socioeconomic composition, and cross-national differences. Prior research suggests school leaders influence outcomes indirectly by shaping school conditions rather than acting directly on student learning. However, it remains uncertain if these digital conditions predict CIL when confounding factors like socioeconomic stratification and national system differences are rigorously controlled.

Methodology
The study utilizes a secondary analysis of the ICILS 2023 international database, focusing on a pooled sample of 12 education systems (Chile, Chinese Taipei, Cyprus, Denmark, Greece, Korea, Norway, Romania, Slovak Republic, Slovenia, Sweden, and Uruguay) that administered the optional principal AI module. The analytic sample consists of 24,273 Grade 8 students nested within 1,049 schools.

The analytical workflow is organized into three linked stages using specific software tools:

Data Preparation (R): Files were merged, and student socioeconomic status (SES) was decomposed into within-school deviations (IISBW) and school-mean compositional components (IISBM). Unique school identifiers and country-normalized weights (SENWGT) were created.
Primary Modeling (Mplus 9): A weighted two-level multilevel analysis was conducted using plausible values (PV1–PV5) for CIL to ensure accurate inferential statistics. The model sequence included:
- An unconditional model to estimate the Intraclass Correlation Coefficient (ICC).
- Models adding student-level predictors (sex, within-school SES, home internet access, computer experience) and school-level SES.
- Models incorporating 11 country fixed effects (with Denmark as the reference) to isolate within-country associations and reduce confounding by national system differences.
- A full model testing school-level predictors: Positive AI expectations (AIPOS), Negative AI expectations (AINEG), ICT pedagogical hindrances (HPED), Principal ICT-use expectations (EXPLRN), and Teacher ICT-collaboration expectations (EXPTCH).
Sensitivity Checks (IDB Analyzer/SPSS): Design-based Jackknife Repeated Replication (JRR) sensitivity analyses were performed using 75 student replicate weights. These served as comparable single-level regressions to evaluate the robustness of direction and statistical uncertainty, rather than replacing the primary multilevel estimates.

Key Findings

Variance Distribution: Approximately 33.8% of the variance in CIL was located between schools, justifying the multilevel approach.
Student and School Composition: Within-school socioeconomic advantage, school socioeconomic composition (IISBM), computer experience, home internet access, and sex were significant predictors of CIL. School socioeconomic composition was the strongest between-school correlate.
AI and ICT Expectations: After controlling for student background, school SES, and country fixed effects, AI expectation scales (AIPOS, AINEG) and ICT expectation variables (EXPLRN, EXPTCH) showed no direct association with student CIL.
ICT Pedagogical Hindrances (HPED): This variable exhibited the clearest negative school-level pattern. In the primary Mplus model, the association was negative but marginal ( $p = .081$ ). In the design-based JRR sensitivity analyses, the negative direction persisted, with results ranging from borderline to weaker significance depending on the model specification.
Robustness: Multicollinearity diagnostics indicated that the null findings for AI/ICT expectations were not due to severe collinearity. Sensitivity checks using an ICT expectation composite (ICTEXP) confirmed that broader expectation climates were not clearly associated with CIL.

Significance and Claims
The paper claims that differences in student CIL within the ICILS 2023 dataset are more strongly linked to socioeconomic stratification and implementation-related barriers than to school-level AI expectation climates.

The authors interpret the findings with caution:

Implementation over Rhetoric: The results suggest that practical barriers to integrating technology into instruction (HPED) may be more consequential for student CIL than principals' abstract expectations or orientations toward AI.
Contextual vs. Direct: School-level AI expectations should be viewed as contextual features of the school environment rather than direct instructional mechanisms. Their lack of direct association with CIL does not imply irrelevance but suggests they may operate through unmeasured mediating mechanisms (e.g., teacher professional learning or instructional practices).
Interpretation of HPED: The negative association with HPED is characterized as a "suggestive implementation-barrier pattern" rather than a confirmed direct school-level effect. The authors emphasize that while the pattern is consistent, the effect size is modest, and the evidence does not support strong causal claims.

Conclusion
The study concludes that while school digital transformation is critical, the specific school-level variables of AI and ICT expectations do not directly predict student CIL once socioeconomic and national factors are accounted for. Instead, the removal of pedagogical hindrances appears to be a more tangible, albeit modest, factor associated with student digital competence. The findings reinforce the importance of addressing socioeconomic inequality and practical implementation barriers over focusing solely on leadership expectations regarding emerging technologies.