Learners’ Engagement with Artificial Intelligence: Validation of the Cognitive-Affective AI Engagement Model
This study validates the Cognitive-Affective AI Engagement Model (CAAIEM) as a reliable, four-dimensional framework for measuring university students' engagement with LLM-based AI tools, demonstrating strong psychometric properties through confirmatory factor analysis of data from 1,249 Hungarian first-year students while noting the need for further cross-cultural validation.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern classroom, a quiet revolution is taking place, driven not by new textbooks or blackboards, but by intelligent computer programs capable of writing essays, solving problems, and holding conversations. These tools, known as large language models, have moved from the realm of science fiction into the daily lives of students. For educators and researchers, a pressing question has emerged: how do learners actually connect with these technologies? Is their relationship with artificial intelligence merely a matter of checking a box for utility, or does it involve deeper feelings, habits, and future hopes? To understand this, scientists look at engagement, a concept that describes how deeply a person is involved in an activity. Traditionally, this has been viewed through the lens of whether a tool is useful or easy to use. However, as artificial intelligence becomes more conversational and personal, researchers suspect that the picture is more complex. It likely involves how often a student interacts with the tool, how much they enjoy the experience, how confident they feel in their ability to use it effectively, and whether they plan to keep learning with it in the years to come. Understanding these layers is crucial, because if schools and developers only focus on whether a tool works, they may miss the emotional and behavioral drivers that keep students coming back to learn.
A team of researchers from the University of Szeged in Hungary set out to map this uncharted territory. They proposed a new way of looking at how students engage with artificial intelligence, suggesting that it is not a single feeling but a combination of four distinct parts. They called this the Cognitive-Affective AI Engagement Model. The first part is interaction intensity, which simply measures how often and how deeply a student uses these tools in their daily life and studies. The second is instrumental efficacy, which captures the student's belief that the technology actually helps them get their work done faster and better. The third is affective engagement, a term that refers to the emotional side of the experience: do the students feel enjoyment, comfort, and a genuine liking when they use the AI? The final piece is learning continuance intent, which looks forward to the future, measuring whether the student plans to keep learning about and using these technologies as they evolve.
To test whether this four-part model accurately reflected reality, the researchers gathered a large group of first-year university students. They invited 1,249 students from various faculties across the university to participate in a study. The participants, who were mostly between the ages of 18 and 22, filled out a questionnaire designed to measure each of these four areas. The questions asked them to report how frequently they used AI applications, how much they agreed that AI improved their performance, how much they enjoyed using it, and whether they intended to continue learning about it in the future. The researchers then used sophisticated statistical methods to see if the students' answers naturally grouped together into the four categories the model predicted, or if the data suggested a different structure entirely.
The results were clear and supportive of the new model. When the researchers analyzed the data, they found that the four categories they had proposed were indeed the best way to describe how these students engaged with artificial intelligence. The data did not fit a simpler model where all the answers pointed to just one general feeling, nor did it fit a model with only three categories. Instead, the four distinct dimensions stood out as separate but connected parts of the whole experience. The students' answers consistently showed that their behavior, their beliefs about usefulness, their emotions, and their future plans were all measurable and distinct aspects of their engagement. The study confirmed that these four parts work together to form a complete picture of how a learner relates to AI.
The researchers also checked to ensure that the questions they asked were reliable and that they were actually measuring what they intended to measure. They found that the questions for each of the four areas were consistent; for example, the questions about enjoyment all hung together tightly, as did the questions about future plans. This gave them confidence that the model was a solid tool for understanding student behavior. Furthermore, they verified that these four areas were different enough from one another to be treated as separate concepts, even though they were related. For instance, a student who found the AI useful also tended to enjoy using it, but the study showed that usefulness and enjoyment were not the same thing. Similarly, a student who used the AI frequently had different motivations than one who simply planned to use it in the future.
This study offers a significant step forward in understanding the human side of artificial intelligence in education. By breaking down engagement into these four specific components, the researchers provided a framework that educators and developers can use to see beyond simple usage statistics. It suggests that for students to truly benefit from AI, the technology must not only be effective but also emotionally resonant and aligned with their long-term learning goals. The findings indicate that successful integration of these tools requires attention to how students feel about the technology and how they envision using it in their future, not just whether it helps them complete a task today. While the study focused on a specific group of students in Hungary, the model it validated offers a new language for describing the complex relationship between learners and the intelligent machines that are increasingly shaping their education.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.