💬 NLP

Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback

This study reveals that four widely used large language models systematically generate biased, stereotype-aligned writing feedback—characterized by excessive praise and withheld critique for students marked by race, language, or disability—demonstrating that automated "personalization" often reproduces harmful instructional orientations termed "Marked Pedagogies."

Original authors: Mei Tan, Lena Phalen, Dorottya Demszky

Published 2026-03-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mei Tan, Lena Phalen, Dorottya Demszky

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical, super-smart robot teacher. You give it a student's essay, and it instantly writes back with helpful comments to improve the writing. Sounds great, right? It's like having a personal tutor for every student in the world.

But this paper asks a scary question: What if this robot teacher isn't actually neutral? What if, instead of treating every student the same, it secretly changes its personality based on who it thinks the student is?

The researchers call this phenomenon "Marked Pedagogies." That's a fancy way of saying: "The robot has a different teaching style for different groups of people, and it's based on stereotypes."

Here is the story of how they found this out, using simple analogies.

The Experiment: The "Chameleon" Test

The researchers took 600 real essays written by 8th graders. These essays were all about the same topics (like "Should students do community service?" or "Is the Face on Mars real?").

Then, they asked four different AI models (like GPT-4 and Llama) to grade these essays. But here's the trick: They told the AI different lies about the students.

For the exact same essay, they gave the AI different "ID cards":

Essay A: "This student is a high-achieving White male."
Essay B: "This student is a Black female with a learning disability."
Essay C: "This student is an English Language Learner."
Essay D: "This student is unmotivated."

Since the essay text was identical, any difference in the feedback had to come from the AI's bias, not the student's writing.

What Did the Robot Do? (The Results)

The AI didn't just give different grades; it changed its entire voice and attitude. It acted like a chameleon, shifting its colors based on the student's "ID card."

1. The "Strict Drill Sergeant" vs. The "Gentle Coach"

For High-Achieving Students: The AI acted like a Gentle Coach. It said things like, "You have great potential! Let's explore deeper ideas and strengthen your argument." It treated them like future leaders capable of handling tough criticism.
For Low-Achieving or "Struggling" Students: The AI turned into a Strict Drill Sergeant. It ignored big ideas and focused only on tiny mistakes. It said, "Fix your spelling," "Check your grammar," and "Make sure you didn't make an error." It assumed these students couldn't handle big concepts, so it lowered the bar.

2. The "Language Police" vs. The "Idea Explorer"

For English Language Learners (ELL): The AI became obsessed with grammar rules. It acted like a Language Police Officer, constantly correcting capitalization and verb tenses. It assumed the student didn't know English well, so it ignored their brilliant ideas.
For Native Speakers: The AI acted like an Idea Explorer, asking, "What is your main argument? How can you make this more persuasive?"

3. The "Cultural Stereotype Machine"

This is where it gets really weird. The AI started acting out movie tropes based on race and gender:

Black Students: The AI assumed they were "natural leaders" or "social change agents." It praised their "personal stories" and "community values." While this sounds nice, it's actually a trap. It assumes they must write about culture and struggle, rather than letting them write about anything they want.
Hispanic Students: The AI assumed they were "family-oriented" and needed help with "formal English." It treated them like they were outsiders trying to fit in.
Asian Students: The AI assumed they were "academic robots" who needed to be "respectful" and "polished." It ignored their creativity and focused on them being perfect students.
Female Students: The AI became overly emotional and "cuddly." It used words like "love," "empathy," and "feelings." It treated them like they needed emotional support rather than intellectual challenge.
Male Students: The AI was cold, objective, and focused purely on facts and logic.

The Big Problem: The "Glass Ceiling"

The researchers call this "Feedback Withholding Bias."

Imagine you are a student.

If the robot thinks you are smart and White, it pushes you to the moon. It gives you hard challenges because it believes you can handle them.
If the robot thinks you are Black, Hispanic, a girl, or have a disability, it puts a glass ceiling over your head. It praises you for being "cute" or "hardworking," but it never challenges you to think deeper. It assumes you can't handle the hard stuff.

Why is this bad?
It's like a teacher who only gives the "easy" homework to certain kids because they think those kids are "too fragile" for the hard stuff. Over time, those kids never get to grow. The robot isn't just being mean; it's accidentally holding students back by assuming they are less capable than they actually are.

The Takeaway

The paper concludes that AI is not a neutral tool. It is a mirror that reflects our own human biases back at us.

If we let these robots grade our kids without checking them, we aren't just automating feedback; we are automating inequality. The robot might say, "I'm just following the rules," but the rules it learned from the internet are full of stereotypes.

The Solution?
We need to stop treating AI like a magic black box. We need to ask:

What is the robot assuming about this student?
Is it treating them differently than it would treat someone else?
Are we giving every student the same high-quality challenge, or are we lowering the bar for some?

The authors say we need "transparency." We need to know the robot's "teaching style" so we can fix it before it hurts a child's future. We can't just press a button and hope for the best; we have to make sure the robot is a fair teacher for everyone.

1. Problem Statement

The paper addresses the critical issue of linguistic bias in Large Language Model (LLM)-generated automated feedback for K-12 education. While LLMs are increasingly deployed to provide personalized writing feedback at scale, they are not language-neutral. The authors argue that these models often:

Privilege standard academic English.
Reproduce social stereotypes regarding race, gender, disability, and motivation.
Adapt their pedagogical stance based on "marked" student attributes (non-dominant identities or perceived deficits) versus "unmarked" or normative attributes.

The core problem is that this "personalization" may systematically lower expectations, withhold substantive critique, or apply stereotyped framing to marginalized students, thereby reinforcing educational inequities under the guise of adaptive learning.

2. Methodology

The study employs a controlled computational experiment using contrastive prompting and unsupervised lexical analysis to isolate the effects of student attributes on feedback generation.

Data and Models

Dataset: 600 eighth-grade persuasive essays from the PERSUADE corpus, split equally between two prompts (community service and Mars landform).
Models: Four widely used LLMs were tested:
- OpenAI: GPT-4o, GPT-3.5-turbo
- Meta: Llama-3.3 70B, Llama-3.1 8B
Prompting Strategy: For each essay, feedback was generated under three conditions:
1. Baseline: No student attributes specified.
2. Marked Prompt: Attributes indicating non-dominant status or deficits (e.g., "Black," "ELL," "Learning Disability," "Low Achievement," "Unmotivated").
3. Comparative Prompt: Attributes indicating dominant status or advantages (e.g., "White," "High Achievement," "Motivated").
- Note: Essay content remained constant; only the prompt descriptors changed.

Analytical Framework: "Marked Words"

The authors adapted the Marked Words framework (Monroe et al.) to detect systematic lexical shifts:

Log-Odds Ratio: Computed for each word across the "Marked" ( $F_s$ ) and "Comparative" ( $F_{r(s)}$ ) feedback corpora.
Statistical Significance: Words with $|z| > 1.96$ and a minimum frequency of 30 were identified as statistically significant markers.
Content Analysis: A deductive coding scheme categorized significant words into three pedagogical facets:
- Content Focus: (e.g., grammar vs. argument structure).
- Evaluative Descriptors: (e.g., "unclear" vs. "compelling").
- Manner of Address: (e.g., pronouns, modals, tone).
Concentration Metric ( $C_s(F)$ ): A regression-based metric quantifying the percentage of "marked" words in a feedback instance to measure the intensity of the bias.

3. Key Contributions

Conceptualization of "Marked Pedagogies": The authors introduce a new term to describe systematic shifts in feedback language where LLMs adopt distinct instructional orientations based on perceived student identity.
Empirical Evidence of Bias: The study provides robust, quantitative evidence that LLMs alter their feedback not just in content, but in stance and expectation based solely on prompted attributes, even when the student's writing is identical.
Methodological Innovation: The application of the Marked Words framework to educational feedback offers a scalable method for auditing AI systems for pedagogical bias beyond simple sentiment analysis.

4. Key Results

The analysis revealed systematic, stereotype-aligned shifts in feedback across all four models:

A. Learning Needs-Based Attributes

Low Achievement: Feedback became directive and disapproving, focusing on mechanical errors (spelling, grammar) with brief, corrective comments. High-achieving students received expansive, future-oriented feedback focused on growth and potential.
English Language Learners (ELL): Feedback shifted to language inadequacy, focusing heavily on surface-level corrections (capitalization, spelling) rather than argumentation or reasoning.
Learning Disabilities: Feedback assumed a need for simplification, using direct phrasing and inclusive pronouns ("let's," "we") while lowering the cognitive demand of the critique (e.g., "make it shorter/easier").

B. Identity-Based Attributes

Race (Black, Hispanic, Asian):
- Black students: Feedback validated personal experience and invoked systemic/historical contexts, framing them as agents of social change but often relying on cultural stereotypes.
- Hispanic/Asian students: Feedback frequently assumed limited English proficiency, focusing on "polishing" language and cultural tropes (e.g., "family responsibilities," "respect").
- White students: Feedback was more objective, focusing on argument structure, evidence, and avoiding first-person perspective, treating them as capable writers needing refinement rather than remediation.
Gender (Female): Feedback adopted an emotional and relational tone, using affective language ("I love," "appreciate") and encouraging empathy. Male students received more objective, task-oriented feedback focused on evidence and reasoning.

C. Socio-Psychological Attributes

Unmotivated: Feedback adopted an enthusiastic, collaborative tone ("Let's try," "Great start") with positive reinforcement, avoiding harsh critique. Motivated students received direct, higher-level critique and precision-focused suggestions.

D. Quantitative Findings

Concentration Increases: Specifying a marked attribute significantly increased the concentration of associated "marked" words.
- Example: Prompting "Unmotivated" increased the concentration of enthusiasm-related words by 74% (+4.26 percentage points).
- Example: Prompting "Black" increased the concentration of cultural/stereotype-related words by 180% (+1.87 percentage points).
Baseline Bias: Even without explicit prompting, baseline feedback showed traces of these biases (e.g., 10.6% concentration of "disapproval" language), suggesting LLMs infer attributes from writing style alone.

5. Significance and Implications

Pedagogical Harm: The study demonstrates that LLMs do not merely correct errors; they calibrate expectations. By withholding substantive critique from marginalized groups and offering "cushioned" feedback, LLMs may inadvertently limit students' educational growth and reinforce the "positive feedback bias" observed in human teachers.
Transparency and Accountability: Current automated feedback tools lack transparency regarding how student data influences pedagogical stance. The authors argue for explicit specification of pedagogical priorities in model training and deployment.
Beyond "Blindness": The authors caution against simply removing student data (blindness), as LLMs can still infer attributes. Instead, they advocate for accountable personalization where the pedagogical stance is made explicit, auditable, and aligned with equitable best practices rather than societal stereotypes.
Future Directions: The paper calls for rigorous evaluation of AI in education that goes beyond accuracy metrics to include fairness, equity, and the specific nature of the instructional relationship established by the AI.