🤖 AI

Investigating Self-regulated Learning Sequences within a Generative AI-based Intelligent Tutoring System

This study analyzes student interaction patterns with a Generative AI-based intelligent tutoring system to identify distinct self-regulated learning sequences, revealing that while most students primarily use GenAI for information acquisition rather than transformation, these usage patterns do not significantly correlate with learning performance.

Original authors: Jie Gao, Shasha Li, Jianhua Zhang, Shan Li, Tingting Wang

Published 2026-01-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jie Gao, Shasha Li, Jianhua Zhang, Shan Li, Tingting Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex puzzle, like figuring out the perfect meal plan for a week. Now, imagine you have a super-smart, all-knowing robot assistant (Generative AI) sitting right next to you, ready to answer any question or give you a hint. This study is like a detective story that watched 114 students as they tried to solve nutrition puzzles with this robot assistant. The researchers wanted to see: How do students actually use this robot while they learn? Do they use it in a smart, organized way, or do they just bounce around randomly?

Here is the breakdown of what they found, using some simple analogies:

The Two Types of "Robot Dancers"

The researchers watched the students' every click and keystroke (their "trace data") to see how they moved through the learning process. They found that the students naturally fell into two distinct groups, or "dance styles," based on how they regulated their own learning.

The "Fast Paced" Dancers (Cluster 1):
- How they moved: These students were like runners on a track. They would set a goal, check a piece of food, evaluate it, and immediately move to the next piece. They asked the robot for help frequently (about 5 times on average).
- The Catch: They tended to ask the robot for help, get an answer, and immediately jump to checking the next item. They were very active, but they spent a lot of time looking at individual pieces of information one by one, rather than stepping back to organize the whole picture.
- The Result: They didn't perform as well as the other group. It's like trying to build a house by laying bricks one by one without ever stopping to look at the blueprint; you might get a lot of bricks down, but the house might not stand up right.
The "Deep Thinker" Dancers (Cluster 2):
- How they moved: These students were more like architects. They asked the robot for help less often (about 2 times), but when they did, they had longer, deeper conversations with it. They spent more time in "reflection," which is like pausing to look at the whole puzzle and think, "Does this piece fit with what I already know?"
- The Result: They performed better. By taking time to reflect and organize their thoughts, they built a clearer mental map of the problem.

What Did They Ask the Robot For?

The researchers also looked at why students asked the robot questions. They found two main reasons, like asking a librarian for two different things:

Information Acquisition (The "What is this?" question): This was the most common reason. Students asked things like, "What does sodium do?" or "How much caffeine is too much?" They were just gathering new facts.
Information Transformation (The "What does this mean?" question): This was less common. Students asked things like, "If this drink has carbs, is it still sugar-free?" or "How is this different from that other drink?" This is a harder task because it requires taking what they know and twisting it to solve a specific problem.

The Big Surprise: Even though asking the "What does this mean?" questions is usually a sign of a smarter, deeper learner, the study found that it didn't actually change the students' grades. Whether a student mostly asked for facts or mostly asked for deep analysis, their final scores were about the same. The way they moved through the steps (the "Fast Paced" vs. "Deep Thinker" dance) mattered much more for their success than the specific type of question they asked the robot.

The Takeaway

The study concludes that when we give students a super-smart AI tutor, we can't just assume they will use it wisely.

Some students use it like a speed-reader, skimming facts and moving fast, which might leave them confused about the big picture.
Others use it like a thinking partner, pausing to reflect and organize, which leads to better problem-solving.

The researchers suggest that if we want students to learn better, we shouldn't just give them the robot; we need to teach them how to "dance" with it. We need to build systems that encourage them to stop, reflect, and organize their thoughts, rather than just rushing to the next piece of information.

Technical Summary: Investigating Self-regulated Learning Sequences within a Generative AI-based Intelligent Tutoring System

Problem Statement
While Generative Artificial Intelligence (GenAI) is increasingly integrated into educational environments to provide adaptive content and feedback, its mere presence does not guarantee academic success. The efficacy of GenAI-assisted learning is heavily dependent on how students regulate their interactions with the technology. Although Self-Regulated Learning (SRL) is recognized as critical in these contexts, existing research often lacks the granularity to capture the temporal dynamics of student interactions with GenAI. Specifically, there is a gap in understanding the sequential patterns of SRL behaviors (e.g., planning, monitoring, reflection) as they unfold in real-time within GenAI-supported Intelligent Tutoring Systems (ITS), and how these sequences correlate with learning performance and the specific purposes of AI usage.

Methodology
The study employed a quantitative, process-oriented approach using trace data analysis within a controlled learning environment.

Participants and Context: The study involved 110 university students (out of 114 recruited) interacting with "Healthy Choice," an ITS designed to improve nutrition literacy. The system integrated ChatGPT via an "Ask AI" function to provide scaffolding. Participants completed a problem-solving task requiring them to select optimal nutrition items based on specific scenarios.
Data Extraction and Coding:
- SRL Sequences: Trace data from the platform were coded based on Zimmerman's three-phase SRL model (Forethought, Performance, Reflection). Micro-level actions included Goal-Setting (GS), Execution (EX), Evaluation (EV), Monitoring (MO), Elaboration (EL), and Reflection (RE).
- GenAI Usage Purpose: Prompts used in the "Ask AI" function were classified into two categories based on information processing theory: Information Acquisition (seeking understanding of unfamiliar terms/facts) and Information Transformation (evaluating, confirming, or resolving cognitive conflicts).
Analytical Techniques:
- Sequential Clustering: Hierarchical clustering algorithms were applied to state sequences derived from the coded actions to identify distinct groups of learners based on their SRL transition patterns.
- Transition Analysis: Transition rate heatmaps were generated to visualize and compare the frequency of specific action sequences (e.g., $EV \to EX$ , $Ask\ AI \to EV$ ) between clusters.
- Statistical Analysis: Wilson confidence intervals were used to analyze the prevalence of GenAI usage purposes, and performance differences between clusters were examined.

Key Results

Identification of Two Distinct Clusters: The analysis successfully classified the 62 participants who used the "Ask AI" function into two distinct groups based on their SRL sequences:
- Cluster 1 (n=18): Characterized by higher transition rates in specific sequences, notably $Ask\ AI \to EV$ (using AI to clarify before evaluating) and $EV \to EX$ (moving from evaluation back to execution). These students engaged in more consecutive goal-setting and execution activities but spent more time analyzing individual information pieces rather than synthesizing them.
- Cluster 2 (n=44): Exhibited repeated sequences involving self-reflection ( $RE \to RE$ ) and goal-setting ( $GS \to GS$ ). Notably, this group engaged in multiple rounds of dialogue with GenAI, suggesting deeper interaction.
GenAI Usage Patterns:
- Frequency: Students in Cluster 1 used the "Ask AI" function significantly more often (mean of 5 times) compared to Cluster 2 (mean of 2 times).
- Purpose: Across all participants, GenAI was used more frequently for Information Acquisition (53.6% of prompts) than for Information Transformation (46.4%).
Performance Correlation: The study found that the correlation between the specific purpose of using GenAI (Acquisition vs. Transformation) and learning performance was not statistically significant. However, the paper notes that Cluster 1's pattern of analyzing isolated information without coherent integration may have contributed to lower performance, though this is an inference based on the observed behavioral patterns rather than a direct statistical claim of causality in the results section.

Key Contributions

Temporal Dynamics of SRL: The study moves beyond static measures of SRL to map the sequential unfolding of student behaviors in a GenAI environment, demonstrating that the order and frequency of transitions (e.g., from AI interaction to evaluation) vary significantly between learner groups.
Differentiation of AI Usage Purposes: By distinguishing between information acquisition and transformation, the study highlights a prevalent tendency among students to use GenAI for fact-retrieval rather than higher-order cognitive processing (evaluation/synthesis).
Behavioral Clustering: The application of sequential clustering to ITS trace data successfully identified distinct behavioral profiles, revealing that high-frequency AI users do not necessarily exhibit the same regulatory patterns as low-frequency users, and that specific transition patterns (like repeated reflection) are observable.

Significance and Claims
The authors claim that their findings offer critical insights for both pedagogical design and the development of GenAI-assisted learning environments.

Pedagogical Implications: The results underscore the importance of self-reflection as a strategy for expertise development. The authors suggest instructional designs should explicitly scaffold iterative reflection cycles to help students integrate goals, actions, and outcomes.
Design Implications: The study indicates that the depth and sequencing of AI interactions influence learning. The observation that Cluster 2 (with deeper, multi-round interactions) showed different patterns suggests that adaptive AI tools should be designed to support sustained, structured interactions aligned with SRL strategies, rather than simple query-response loops.
Intervention Needs: The findings highlight a need for targeted interventions for students who rely heavily on isolated information analysis (Cluster 1 pattern) to help them construct coherent mental representations of problems. The authors propose that future systems could leverage real-time learning analytics to provide personalized guidance, steering students toward more effective SRL strategies.

The paper concludes modestly, noting that while the study identifies these patterns, future research is required to determine how real-time analytics can be effectively operationalized to provide such personalized guidance in GenAI-supported environments.

The Two Types of "Robot Dancers"

What Did They Ask the Robot For?

The Takeaway

Technical Summary: Investigating Self-regulated Learning Sequences within a Generative AI-based Intelligent Tutoring System

More like this