💬 NLP

Dialogue Act Patterns in GenAI-Mediated L2 Oral Practice: A Sequential Analysis of Learner-Chatbot Interactions

This study analyzes 10 weeks of interactions between Grade 9 Chinese EFL learners and a GenAI voice chatbot, revealing that high-progress sessions are characterized by learner-initiated questions and timely, prompting-based corrective feedback, thereby offering a pedagogy-informed framework for designing adaptive L2 educational chatbots.

Original authors: Liqun He (Cindy), Shijun (Cindy), Chen, Mutlu Cukurova, Manolis Mavrikis

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Liqun He (Cindy), Shijun (Cindy), Chen, Mutlu Cukurova, Manolis Mavrikis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to learn how to play the piano. You have a robot teacher that never gets tired, never judges you, and is available 24/7. This is what Generative AI (GenAI) voice chatbots are like for people learning a second language (like English). They offer a safe, endless space to practice speaking.

But here's the mystery: Just because the robot is there, does it actually help you get better? And if it does, how does it happen? Is it just about the robot talking, or is it about the specific dance of conversation between the human and the machine?

This paper by Liqun He and colleagues is like a detective story that investigates the "footprints" left behind in these conversations to see what leads to success.

The Setup: The 10-Week Experiment

The researchers watched 12 students (14–15 years old) practice English with a voice chatbot for 10 weeks. They recorded 70 different practice sessions.

To figure out who was "winning," they tested the students' English skills before and after the 10 weeks. They split the sessions into two groups:

The High-Progress Group: Students who got significantly better.
The Low-Progress Group: Students who improved, but not as much.

Then, the researchers acted like linguistic archaeologists. They didn't just listen to what was said; they analyzed how it was said. They broke every sentence down into its "functional purpose" (called a Dialogue Act). Did the student ask a question? Did the bot correct a grammar mistake? Did the bot just say "uh-huh"?

The Findings: What Made the Difference?

The study found two major clues that separated the "High-Progress" sessions from the "Low-Progress" ones.

1. The "Curiosity vs. Confusion" Clue

The High-Progress Students: They were like curious explorers. They asked the robot more questions of their own. Instead of just waiting for the robot to talk, they steered the conversation. They were actively trying to find things out.
The Low-Progress Students: They were like lost tourists. They asked the robot to repeat or explain things much more often ("What did you mean?", "Can you say that again?"). This didn't mean they were bad students; it just meant they were struggling to understand the robot's words, which stopped the flow of learning.

The Analogy: Imagine a hiking guide. The successful hikers ask, "Can we take that trail over there?" (Curiosity). The struggling hikers keep stopping to ask, "Which way is north?" (Confusion). The successful hikers are engaging with the journey; the others are stuck trying to figure out the map.

2. The "Correction Dance" Clue

This is the most interesting part. The researchers looked at the sequence of events—like a recipe for a perfect conversation.

The Winning Recipe:
1. The Bot asks a question.
2. The Student answers (and maybe makes a mistake).
3. The Bot pauses and says: "Hmm, try saying it this way..." (This is called Prompting). It hints at the error without giving the answer away.
4. The Student fixes it themselves.
5. The Bot asks the next question.
The Losing Recipe:
The bot either ignored the mistake, or it just blurted out the correct answer immediately ("No, you should say 'went' not 'go'").

The Analogy: Think of learning to ride a bike.

Prompting (The Winner): The coach holds the seat and says, "Pedal harder!" You figure out how to balance. You learn.
Explicit Correction (The Loser): The coach grabs the bike, fixes the wheel, and says, "There, now it's right." You didn't learn how to balance; you just watched the coach fix it.

The study found that the "High-Progress" sessions were full of that Prompting Dance. The bot would hint at an error, let the student fix it, and then keep the conversation moving. The "Low-Progress" sessions lacked this specific rhythm.

Why Does This Matter?

For a long time, people thought, "If we just put an AI in the room, learning will happen." This paper says, "No, it's not about having the AI; it's about how the AI behaves."

It's like having a very smart butler. If the butler just does everything for you, you never learn to cook. But if the butler asks, "Did you remember to add the salt?" and waits for you to fix it, you actually learn the recipe.

The Takeaway for the Future

The authors suggest that when we build these AI tutors for the future, we shouldn't just program them to be "nice" or "correct." We need to program them to:

Encourage students to ask questions (be curious, not just confused).
Use the "Hint" strategy (Prompting) instead of just giving answers.
Time their feedback perfectly (correcting a mistake right after the student speaks, but in a way that keeps the conversation flowing).

In short, the best AI tutor isn't the one that knows the most; it's the one that knows how to have a conversation that makes the student think.

1. Problem Statement

While Generative AI (GenAI) voice chatbots offer scalable solutions for Second Language (L2) oral practice, the specific interactional mechanisms that lead to learning gains remain underexplored. Previous research has established that GenAI improves proficiency metrics (fluency, complexity, pronunciation) but often relies on pre-post testing, overlooking the dialogic processes occurring during interaction.

The Gap: There is a lack of fine-grained, turn-level analysis linking specific dialogue acts (DAs) and their sequential patterns to differential learning outcomes in GenAI-mediated language learning.
The Goal: To investigate how specific dialogue act frequencies and sequential patterns differ between high-progress and low-progress learners in a GenAI voice chatbot environment.

2. Methodology

Dataset and Participants

Participants: 12 Grade 9 Chinese EFL learners (aged 14–15, CEFR A2–B1) from an international school in central China.
Intervention: 10-week program involving weekly 35-minute open-ended oral practice sessions with a purpose-built GenAI voice chatbot.
Data Volume: 70 valid sessions (35 High-Progress, 35 Low-Progress), totaling 4,922 dialogue turns and 6,957 coded Dialogue Acts (DAs).
Proficiency Measurement: Oral proficiency was assessed using TOEFL Practice Online (TPO) tasks. Gains were calculated using the Complexity–Accuracy–Fluency (CAF) framework (lexical/grammatical complexity, accuracy, and fluency metrics).
- Grouping: Sessions were split into High-Progress (HP) and Low-Progress (LP) groups based on the magnitude of pre-post learning gains.

Coding Scheme (Dialogue Acts)

A pedagogy-informed, deductive coding scheme was developed based on Communicative Language Teaching (CLT) and Focus on Form (FonF). DAs were categorized into two dimensions:

Meaning-Focused:
- Inviting: Referential Questions (Q), Greetings/Closings (G), Topic Shifting (T).
- Sustaining: Seeking Clarification (S), Agreement (A), Disagreement (D), Misinterpretation (M).
- Content Feedback: Response (R).
Form-Focused (Corrective Feedback - CF):
- Recast (Cr): Implicit reformulation.
- Prompting (Cp): Signaling error and encouraging self-correction.
- Explicit Correction (Ce): Direct error identification and correction.

Annotation and Reliability

Process: Two expert EFL teachers annotated turns at the speaker level. Up to two DA codes were assigned per turn.
Reliability: Inter-coder reliability (Cohen's $\kappa$ ) ranged from fair to almost perfect. Discrepancies in low-agreement codes (Ce, M, D) were resolved by the first author.

Data Analysis

RQ1 (DA Frequencies): Chi-square tests with Holm–Bonferroni correction compared DA frequencies between HP and LP groups.
RQ2 (Sequential Patterns):
- Mining: Used the CM-SPAM algorithm to mine frequent sequences (length 2–4, max gap 1, min support 20%).
- Filtering: Applied a support difference threshold (10) to isolate patterns with meaningful prevalence differences.
- Testing: Used permutation tests (with student-level shuffling) to handle non-independent data (multiple sessions per learner) and control for false positives.

3. Key Results

RQ1: Differences in Dialogue Act Frequencies

Overall Distribution: Interactions were predominantly chatbot-led. The top three DAs were Chatbot Questions ([t]Q, 31.4%), Learner Responses ([s]R, 27.2%), and Chatbot Agreement ([t]A, 19.0%).
Significant Learner Differences:
- High-Progress (HP): Significantly higher frequency of Learner-Initiated Questions ([s]Q) (2.1% vs. 1.0%).
- Low-Progress (LP): Significantly higher frequency of Clarification-Seeking ([s]S) (2.5% vs. 1.3%), indicating greater comprehension difficulties.

RQ2: Differences in Sequential Patterns

The "Prompting" Pattern: The most significant finding was the prevalence of Prompting Corrective Feedback ([t]Cp) in HP sessions.
- Significant Sequence: [t]Q → [s]R → [t]Cp (Chatbot asks, Student responds, Chatbot prompts for self-correction). This was significantly more frequent in HP groups ( $p_{adj} = .041$ ).
- Extended Sequence: [t]Q → [s]R → [t]Cp → [t]Q (The cycle continues with a follow-up question) was also significantly more frequent in HP groups.
Timing Matters: Effective prompting was consistently positioned immediately after learner responses ([s]R). No significant differences were found for Recasts ([t]Cr) or Explicit Corrections ([t]Ce) in isolation or sequence.
Marginal Findings: HP sessions showed marginally higher frequencies of sequences involving learner disagreement ([t]Q → [s]D), suggesting deeper meaning negotiation.

4. Key Contributions

Theoretical Framework: Proposes a systematic, pedagogy-informed DA coding scheme specifically for GenAI-mediated L2 oral practice, bridging Speech Act Theory with language learning pedagogy.
Methodological Innovation: Demonstrates the value of combining turn-level DA analysis with sequential pattern mining to uncover interactional mechanisms that pre-post testing misses.
Empirical Evidence: Provides evidence that learning gains are mediated by specific interactional sequences, not just the presence of feedback. Specifically, it highlights that how and when feedback is delivered (prompting after a response) is more critical than the frequency of feedback types.

5. Significance and Design Implications

Dialogic Lens: The study argues that GenAI chatbot design must move beyond static templates to a "dialogic lens," where the system adapts based on real-time interactional signals.
Adaptive Design Recommendations:
- Encourage Initiative: Systems should be designed to elicit and reward learner-initiated questions ([s]Q) rather than just reactive responses.
- Monitor Comprehension: High rates of clarification-seeking ([s]S) should trigger adaptive scaffolding (e.g., rephrasing, slowing speech) rather than continuing complex dialogue.
- Feedback Timing & Type: Chatbots should prioritize prompting (encouraging self-correction) over explicit correction, and crucially, deliver this feedback immediately after learner responses to maintain flow and efficacy.
- Sequence Awareness: Future GenAI tutors should be programmed to recognize specific successful sequences (e.g., Question $\to$ Response $\to$ Prompt $\to$ Follow-up) and replicate them.

Conclusion

The paper concludes that successful GenAI-mediated L2 oral practice is characterized by active learner initiation and strategically timed prompting feedback. By analyzing the sequential flow of dialogue, educators and developers can move beyond "what" the chatbot says to "how" it interacts, leading to more adaptive and effective language learning tools.