Profiles of generative AI chatbot use for schoolwork and achievement among Indonesian 15-year-olds: A latent class and multilevel analysis of PISA 2025 data
Analyzing PISA 2025 data from 14,168 Indonesian 15-year-olds, this study identifies four distinct generative AI chatbot usage profiles and finds that while rare users score slightly lower in science and reading than selective users, intensive use is not associated with reduced achievement, suggesting schools should focus guidance beyond just frequent users.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern classroom, a new kind of helper has appeared. It is not a person, but a computer program capable of writing essays, solving math problems, and explaining complex ideas in seconds. This technology, known as generative artificial intelligence, has sparked a quiet debate among educators. On one side, there is hope that these tools can offer personalized tutoring to every student, filling gaps in understanding and making learning more efficient. On the other side, there is a fear that relying too heavily on them might make students less engaged, causing them to skip the mental work required to truly learn and remember what they study. The core question is not just whether students are using these tools, but how they are using them and what happens to their learning as a result. Does asking a chatbot to write a draft help a student learn, or does it simply bypass the learning process entirely?
To find answers, researchers turned to a massive snapshot of student life in Indonesia. They analyzed data from the 2025 Programme for International Student Assessment, a global test that measures the skills of 15-year-olds. This specific study focused on nearly 14,000 students across hundreds of schools, representing about 4 million teenagers in the country. The researchers wanted to see if there were different "types" of students based on how they used AI chatbots for their schoolwork. They looked at four specific ways students might use these tools: to summarize a text they had to read, to research a new topic, to draft a writing assignment, or simply to help them learn. By grouping students who used the tools in similar ways, the team could see if certain patterns of use led to better or worse test scores in science and reading.
The analysis revealed four distinct groups of students. The first group, making up about 28 percent of the students, were "rare users." These teenagers almost never turned to chatbots for their schoolwork. The second group, the largest at 35 percent, were "selective learning-support users." These students used the tools occasionally, primarily to get help understanding a concept or learning a new idea, but they did not rely on them for every task. A small group of 7 percent used the tools on a monthly basis for all types of tasks, while the final group, comprising nearly 30 percent, were "intensive users" who used chatbots almost every day for every kind of school assignment. Surprisingly, the differences between these groups were mostly about how often they used the technology, not what they used it for. The researchers had expected to find a group of students who used the tools mainly to offload their work—letting the computer do the thinking for them—but that specific pattern did not emerge as a separate group.
When the researchers compared the test scores of these groups, a clear picture emerged. The students who rarely used chatbots scored slightly lower in both science and reading than those who used them selectively. This gap remained even after the researchers accounted for factors like family income, the type of school the student attended, and their general study habits. However, the students who used the tools intensively, every day for every task, did not score lower than the selective users. They performed just as well. This finding challenges the idea that heavy use of AI automatically leads to poorer learning. It suggests that for these Indonesian teenagers, using a chatbot frequently did not hurt their ability to understand science or read effectively.
The study also looked at whether a student's ability to plan and monitor their own learning changed these results. Some theories suggest that students who are good at managing their own study time might use AI differently than those who are not, perhaps using it to learn rather than to bypass the process. However, the data showed that a student's self-regulation skills did not change the relationship between chatbot use and test scores. Whether a student was good at planning their study or not, the pattern of scores remained the same: rare users scored a bit lower, while frequent users did not suffer. The researchers also explored whether schools that taught students to critically evaluate AI-generated content saw different results. They found that in schools where this practice was common, the gap between rare users and selective users actually grew slightly larger, though this result was not strong enough to be considered a definitive rule.
The researchers are careful to note that this study cannot prove that using chatbots causes better or worse scores. Because the data was collected at a single point in time, it is possible that students who struggle with schoolwork simply do not use these tools, or that students who are already good at learning find ways to use the tools effectively. The study also relied on students reporting their own habits, which can sometimes be imperfect. Despite these limits, the results offer a reassuring perspective for a country where schools are still figuring out how to handle this new technology. The data suggests that there is no penalty for using AI heavily, at least not in the way it was measured here. Instead, the concern may lie with the students who do not use these tools at all, who appear to be falling slightly behind their peers who use them selectively. This points to a new kind of digital divide, where the issue is not just access to technology, but the guidance needed to use it as a learning partner rather than a replacement for thinking.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.