Predicting Vocabulary Acquisition in EFL Learners Using Machine Learning Models
This study demonstrates that while generative AI significantly enhances vocabulary acquisition in EFL learners compared to conventional instruction, linear regression models relying primarily on linguistic indicators outperform complex machine learning algorithms and behavioral metrics in predicting these learning gains.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Learning a new language is a journey of accumulation, where the ability to express complex ideas depends heavily on the size and variety of a person's vocabulary. For decades, teachers have known that students learn best when they encounter new words repeatedly and use them in meaningful contexts, such as writing essays or having conversations. In recent years, a powerful new tool has entered the classroom: artificial intelligence. These systems can chat with students, offer instant explanations, and suggest better ways to phrase a sentence. But while schools are eager to adopt these tools, a critical question remains unanswered: does simply having access to an intelligent chatbot actually help students learn more words, and can we predict who will benefit the most?
Researchers at the University of Prince Mugrin in Saudi Arabia set out to answer this by observing a large group of university students learning English. They wanted to see if using AI to help with writing assignments led to better vocabulary growth compared to traditional teaching methods. More importantly, they wanted to know if they could use computer models to forecast a student's progress based on how they used the technology. The study involved 180 undergraduate students over a fifteen-week period. The students were split into two groups. One group received standard feedback from their teacher on their writing, while the other group was allowed to use advanced AI chatbots to explore word meanings, check their grammar, and revise their drafts. Both groups studied the same material and wrote the same assignments, ensuring that the only major difference was the source of their feedback.
The results showed that both groups improved their vocabulary knowledge over the course of the semester, which is expected when students are actively engaged in learning. However, the group that used the AI tools made significantly larger gains. On average, these students improved their vocabulary test scores by more than seven points, while the group relying solely on teacher feedback improved by just over three points. This difference was not just a matter of chance; the students using AI learned more words, and they used a wider variety of words in their writing. Their essays showed a greater richness of vocabulary and a higher use of academic terms, suggesting that the AI helped them not just memorize definitions, but actually incorporate new words into their active language skills.
The researchers then turned to the question of prediction. They asked whether they could look at a student's behavior—such as how long they spent talking to the AI, how often they revised their work, or how engaged they seemed—and use that information to guess how much their vocabulary would grow. They tested this using several different computer models, ranging from simple statistical methods to more complex algorithms designed to find hidden patterns. Surprisingly, the most sophisticated computer models did not perform better than the simplest ones. In fact, the most accurate way to predict a student's vocabulary gain was to look at their initial test scores and whether they were in the AI group or the teacher-only group.
Perhaps the most revealing finding was that the behavioral data collected from the AI users did not help predict learning outcomes. Knowing how many minutes a student spent with the chatbot or how many times they clicked a button to get a suggestion told the researchers very little about how much that student actually learned. The quantity of interaction was not a reliable indicator of quality. A student might spend a long time with the tool but simply accept the first suggestion without thinking, while another might use it briefly but deeply analyze the alternatives. The study suggests that the value of the AI came from the nature of the support it provided—immediate, individualized feedback that allowed students to experiment with language in real-time—rather than the sheer amount of time they spent using it.
Ultimately, the study demonstrates that artificial intelligence can be a powerful partner in language learning, but only when it is integrated carefully into a structured course. The technology did not replace the teacher; instead, it extended the teacher's reach, offering students a way to practice and refine their writing whenever they needed it. The researchers found that the best way to understand and predict vocabulary growth was not by tracking how busy a student was with a machine, but by looking at the actual changes in their writing and their test scores. This approach offers a clear path forward for educators: focus on the quality of the learning interaction and the tangible improvements in language use, rather than just counting the minutes students spend online.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.