Tutorbot pilot study: learning support, feedback and assessment in a crisis intervention module
This pilot study in a South African psychology module demonstrates that while AI-powered tutorbots effectively provide multilingual, formative scaffolding for concept clarification and assessment preparation, their Socratic questioning style can cause frustration, indicating they are best used as bounded supplementary tools rather than replacements for human clinical supervision.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the training of future psychologists, there is a difficult gap between knowing a theory and knowing how to use it when a person is in immediate danger. A student might be able to define what a crisis is or list the steps for helping someone who is suicidal, yet still freeze when faced with a real, chaotic situation where safety, culture, and human emotion collide. This is not a failure of memory, but a challenge of judgment. To bridge this gap, educators have long relied on practice and feedback, but in crowded classrooms, it is often impossible for a single teacher to give every student the immediate, repeated guidance they need to build that confidence. Recently, artificial intelligence has entered this space, offering a new kind of digital companion. Unlike a general chatbot that can talk about anything, an educational tutorbot is a specialized tool designed to stick to a specific course, answer questions, and help students practice their thinking without the pressure of a human audience.
Researchers at a university in South Africa recently put this idea to the test in a high-stakes module on crisis intervention. They wanted to see how postgraduate psychology students would actually use this AI tool when studying topics like trauma, suicide, and violence. The study did not try to prove that the AI could teach students to be better therapists on its own, nor did it measure if the students' final grades improved. Instead, the researchers simply watched how the students and the bot talked to each other. They analyzed over a thousand messages exchanged between 46 students and the AI between July and October. The goal was to understand what the students were asking for, how the bot responded, and where the two sides got stuck.
The results showed that the students used the tutorbot primarily as a study partner for the hardest parts of their course. Nearly half of the messages were about specific crisis topics, such as how to handle a suicide risk or how to support a family after a violent event. The students did not just ask for simple definitions; they asked how to apply these ideas to complex, real-world scenarios. They pasted in assignment instructions and case studies, asking the bot to help them figure out what was missing from their answers or how to structure their thoughts. The bot acted as a formative scaffold, a term that describes a support structure that helps a learner reach a higher level of understanding before they can stand on their own. It explained difficult concepts, offered examples, and encouraged students to revise their thinking before they submitted their work for a grade.
However, the interaction was not always smooth. The bot was programmed with a specific teaching style known as Socratic questioning, which means it often answered a student's question with another question. Instead of giving a direct answer, it would ask, "What do you already know about this?" or "How would you apply this to a specific person?" The researchers found that while this method worked well for some students who were ready to think deeper, it frustrated others who were confused and just wanted a clear explanation. Several students explicitly told the bot to stop asking questions and just give the answer. One student even wrote that the constant questioning was overwhelming them. This tension revealed a key limitation: the bot was too rigid in its style. It could not easily tell when a student needed a direct fact versus when they needed a challenge, a skill that human teachers use instinctively.
The study also highlighted the unique challenges of teaching in a multilingual country like South Africa. While most of the conversation happened in English, a small number of students used the bot to ask for explanations in their home languages, such as isiXhosa or Setswana. This showed promise for making complex psychological ideas more accessible. However, the researchers noted that the bot sometimes misunderstood cultural nuances. In one instance, a student had to correct the bot's translation because the AI had missed the specific cultural meaning of a word related to a local ritual. This suggested that while AI can help translate words, it cannot yet fully understand the deep cultural context that is essential for effective psychological care.
Ultimately, the study concluded that this type of AI tool is valuable, but only when it is used carefully and under supervision. It can serve as a safe space for students to practice their reasoning, get immediate feedback, and rehearse for exams without fear of judgment. It can help them move from simply memorizing facts to applying them in difficult situations. Yet, the researchers were clear that the bot should never replace a human lecturer, a clinical supervisor, or the professional judgment required in real-life crisis work. The bot is a tool for learning how to think, not a substitute for the thinking itself. If used as a shortcut to get answers without doing the mental work, it could give students a false sense of competence, which would be dangerous in a field where lives are at stake. The future of this technology in education depends on designing systems that are flexible enough to meet students where they are, while keeping the human element of teaching and ethical oversight firmly in place.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.