Human-Guided AI Interaction in Task-Based Language Teaching: Effects of ChatGPT and Embodied Multimodal Conversational AI in Mixed Reality on ESL Learners’ Job Interview Anxiety and Oral Performance

Technical Summary: Human-Guided AI Interaction in Task-Based Language Teaching

Problem Statement
English for Specific Purposes (ESP) learners, particularly those preparing for high-stakes scenarios like job interviews, face significant barriers in developing oral proficiency. Traditional instruction often suffers from large class sizes, a lack of personalization, and limited opportunities for authentic interaction, which exacerbate Foreign Language Anxiety (FLA). While various interventions exist, there is a critical gap in integrated research comparing how different AI-mediated interaction modalities—specifically voice-based generative AI versus embodied multimodal agents in Mixed Reality (MR)—simultaneously impact speaking performance and anxiety. Furthermore, existing studies often rely solely on self-reported anxiety measures, lacking physiological data to capture real-time affective states.

Methodology
This study employed a quasi-experimental mixed-methods design over a six-week intervention involving 75 adult ESL learners enrolled in a "Skills for Success" course. Participants were assigned to one of three conditions:

1. Control Group: Traditional instructor-led instruction.
2. ChatGPT Group: Interaction via ChatGPT voice mode, utilizing prompt engineering to simulate an ESL job-interview trainer.
3. MECAI Group (Multimodal Embodied Conversational AI): Interaction with an embodied AI agent within a Mixed Reality environment (using Meta Quest Pro headsets and Unity/Convai SDKs). The agent featured gaze, gestures, and voice interaction within a simulated office setting.

Data Collection and Analysis:

• Performance: Pre- and post-intervention mock job interviews were assessed using a standardized analytic rubric covering fluency, pronunciation, grammar, vocabulary, content, and relevance. Scores were aggregated for an overall performance metric.
• Anxiety: Measured via the Foreign Language Classroom Anxiety Scale (FLCAS) and physiological heart rate monitoring (Polar OH1) during interviews.
• Qualitative: Structured interviews with participants were analyzed thematically to understand learner perceptions.
• Statistical: A two-way repeated measures ANOVA was used to analyze differences in performance, heart rate, and FLCAS scores across groups and time.

Key Results

• Oral Performance: All groups showed significant improvement from pre- to post-test. However, the MECAI group demonstrated the greatest gains (pre: 9.36; post: 18.52), followed by the ChatGPT group (pre: 8.96; post: 15.52), and the control group (pre: 7.92; post: 12.88). Pairwise comparisons confirmed statistically significant differences between all three conditions ($p < 0.001$).
• Anxiety (FLCAS and Heart Rate): While descriptive trends suggested a reduction in anxiety for the MECAI group (both in self-reports and heart rate), neither the FLCAS nor heart rate measures reached statistical significance across the groups or over time. The study notes that performance improvements may precede measurable anxiety reductions, or that the high-stakes nature of the task maintained physiological arousal despite improved skill.
• Qualitative Findings: Thematic analysis revealed four key themes:
  1. AI's Role in Confidence: AI provided a judgment-free environment for practice, though the lack of non-verbal cues in the voice-only group was noted as a limitation.
  2. Immersion and Presence: The MECAI group reported a strong sense of "being there" and realism, with the embodied agent perceived as a "real person" or trusted advisor.
  3. Indispensable Human Instructor: All groups emphasized that AI serves as a supplement, not a replacement, for human teachers who provide emotional attunement and cultural guidance.
  4. Managing Anxiety: Learners viewed moderate anxiety as potentially productive and noted that AI reduced the fear of negative evaluation compared to human interlocutors.

Significance and Claims
The paper claims that embodied conversational AI in Mixed Reality offers meaningful advantages for ESP speaking development, outperforming both traditional instruction and voice-only AI in terms of oral performance gains. The study suggests that the "embodiment" of the AI agent—providing visual cues, gestures, and a sense of social presence—contributes to these gains more effectively than audio-only interaction.

However, the author maintains a modest stance regarding anxiety reduction. They explicitly state that the study does not provide strong evidence that AI-mediated practice reduces anxiety within the six-week period, as statistical significance was not achieved for anxiety measures. The significance of the work lies in demonstrating that while AI (particularly MR-based) can significantly enhance performance, it functions best as a pedagogical tool guided by human instructors rather than a standalone solution. The research highlights the need for further investigation into the mechanisms of social presence and the long-term transfer of skills to authentic employment contexts.