Bias Taught: Linguistic and Model Stereotypes in AI-Based Educational Evaluations
This study reveals that large language models exhibit systematic linguistic and cultural biases in educational evaluations, generating assessments with varying difficulty and length across languages that correlate with PISA scores while reinforcing stereotypes that emphasize empathy in Western contexts versus hierarchy in Asian contexts, thereby necessitating culturally-sustaining audits before deployment.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern classroom, a quiet revolution is taking place. Teachers and students are increasingly turning to artificial intelligence, specifically large language models, to help design tests, grade papers, and offer feedback. These computer programs are trained on vast amounts of text from the internet, learning to predict what words should come next in a sentence. Because they are so good at mimicking human writing, schools are beginning to use them to create educational materials for students all over the world. However, there is a growing concern that these tools might not be as neutral as they appear. Just as a person might hold unconscious beliefs about how different groups of people learn, these computer programs might have absorbed similar biases from the data they were fed. If an AI system treats a student differently simply because they asked a question in a different language, it could create an unfair learning environment that reinforces old stereotypes rather than helping students succeed.
A team of researchers from Bowdoin College decided to test whether these digital tools were truly impartial. They set out to see if the language a student used to ask for help would change the difficulty of the work the computer assigned. To do this, they asked four of the most popular AI models to create math, reading, and science tests for fifteen-year-old students. They gave the exact same instructions to each computer, but they translated those instructions into eight different languages: Arabic, English, French, German, Hindi, Mandarin, Portuguese, and Spanish. In total, the researchers collected more than four thousand responses from the machines. They then analyzed the length of the tests, how hard the models said the tests were, and what kind of behaviors the models thought made a "good" or "bad" student and teacher.
The results revealed a striking pattern. When the researchers asked the AI to create a test in Hindi or Mandarin, the computer consistently generated much longer and more difficult assignments than when it was asked to do the same task in Arabic, German, or Spanish. For example, one model estimated that a math test written in Mandarin would take a student two hours to complete, while a test in Spanish would take less than an hour, even though the instructions were identical. The models also rated the difficulty of the Hindi and Mandarin tests as significantly higher. This was not a random glitch; the differences were so consistent that they appeared across all four different AI systems the researchers tested. The study found that the difficulty and length of the AI-generated tests strongly matched real-world international rankings of student performance. In other words, the computers seemed to believe that students from countries with high test scores deserved harder work, while students from countries with lower scores deserved easier work, regardless of the actual student sitting in front of the screen.
Beyond just creating tests, the researchers also asked the AI to describe what a "good" teacher or student looks like. Here, too, the language used changed the answer. When the prompt was in English, German, or Portuguese, the AI tended to describe good teachers as empathetic, patient, and focused on building relationships with students. However, when the prompt was in Hindi or Mandarin, the AI described good teachers as strict, disciplined, and focused on knowledge and hierarchy. Similarly, the models described "bad" students differently depending on the language. In some languages, the AI focused on a lack of discipline or procrastination, while in others, it focused on a lack of respect or general misbehavior. These differences suggest that the AI is not just translating words; it is pulling from a database of cultural stereotypes about how education works in different parts of the world.
The researchers were careful to check if these differences were simply because different countries have different school curriculums. They consulted with a group of educators from around the globe to see if the AI was just following national standards. They found that curriculum differences could only explain a tiny fraction of the results. The vast majority of the variation came from the AI itself. The study suggests that these computer programs have learned to associate certain languages with specific academic expectations, effectively acting as mirrors for global inequalities. If a student in a non-Western country uses an AI tutor, they might be given a much harder workload or a different kind of feedback than a student in a Western country, simply because of the language they speak.
This research highlights a critical problem for the future of education. As schools rely more on artificial intelligence to design lessons and evaluate students, there is a risk that these tools will quietly reinforce harmful stereotypes. The AI is not making these judgments because it understands the students; it is repeating patterns it found in its training data. The study concludes that before these tools are widely used in classrooms, they need to be audited to ensure they do not treat students unequally based on their language. Without this check, the promise of personalized, global education could instead become a system that deepens the divide between students, teaching them that their potential is defined by the language they speak rather than their own abilities.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.