📄 social_science

Compassionate Feedback in AI-Assisted Assessment: A Comparative Analysis of Novice and Experienced Teachers' Feedback Patterns within the Integrated AI Triad (IAT) Framework

This study compares novice and experienced teachers' use of the Integrated AI Triad (IAT) framework to generate compassionate feedback, revealing that while experienced educators provide more specific and cognitively scaffolded responses, teacher experience alone does not guarantee quality, and condensed three-session professional development workshops can effectively optimize feedback outcomes while reducing costs.

Original authors: Hossein Talebzadeh

Published 2026-08-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Hossein Talebzadeh

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the classroom as a giant, bustling kitchen where the teacher is the head chef and the students are the apprentices. For decades, the most important job of the chef has been to taste the food, spot the mistakes, and offer a gentle nudge to help the apprentice get better. This is called "feedback." But recently, a new, super-smart robot sous-chef has been hired. This robot can taste thousands of dishes in a second and write down notes instantly. The big question isn't whether the robot can write notes; it's whether those notes have a heart. Can a machine offer the kind of "compassionate feedback" that feels like a warm hug while also pointing out exactly where the salt was forgotten? This is the puzzle researchers are tackling in the world of educational science, specifically looking at how teachers and artificial intelligence (AI) work together to help struggling students.

The study you are about to read dives into this exact mix. It looks at a new idea called the "Integrated AI Triad" (IAT), which is basically a fancy map for how teachers, school leaders, and families can use AI without losing the human touch. The researchers wanted to know: Does the robot help the teacher become a better, kinder guide, or does it just make the teacher less engaged? And does it matter if the teacher is a brand-new rookie or a veteran with years of experience?

The Great Teacher vs. Robot Taste-Test

In this study, a researcher named Hossein Talebzadeh set up a cooking class for 57 teachers. Half were "novices" (newbies with 0–3 years of experience), and half were "experts" (veterans with 7+ years). They were given a tricky assignment: imagine a student who wrote a terrible answer to a question. The teachers had to use one of four different AI robots (ChatGPT, Copilot, DeepSeek, or Perplexity) to help them write a response that was both kind and helpful.

The goal was to create "compassionate feedback." Think of this as a two-part recipe:

The Warm Hug: Acknowledging the student's effort and making them feel safe.
The Sharp Knife: Giving specific, precise instructions on how to fix the mistake.

The researchers then tasted (analyzed) all 57 feedback notes to see who served up the best meal.

What the Tasters Found

Here is the delicious part of the story: Experience matters, but not in the way you might think.

The veteran teachers served up feedback that was much more specific. They didn't just say, "Good job, try again!" Instead, they pointed to the exact step where the student went wrong, like, "You added 5 to both sides, but you forgot to divide by 3 first." Their feedback was like a GPS that gave turn-by-turn directions. On a scale of 1 to 5, the experts scored a 4.0 for specificity, while the newbies only scored a 2.8.

However, the newbies had a secret weapon: they were slightly warmer. They gave out more generic hugs, saying things like, "You're doing great!" more often than the veterans. Their compassion score was a 4.0, compared to the veterans' 3.7. But here's the catch: the veterans' hugs were anchored to the work. They said, "You correctly identified the chemicals, which is a great start!" The newbies' hugs often floated in the air, disconnected from the actual mistake.

The study found that the best feedback—the kind that was both super specific and super kind—was much more common among the experienced teachers (32% of their feedback) than the newbies (only 15%).

The Robot Sous-Chefs: Not All Bots Are Created Equal

The study also tested the four different AI robots to see if they had their own "personalities."

DeepSeek turned out to be the most curious robot. It asked more "Socratic" questions, which are like a series of riddles that lead you to the answer yourself. It was the best at helping students think for themselves.
ChatGPT and Copilot were more direct, often just giving the answer or a simple question.
Perplexity was good at finding facts but less focused on the teaching style.

But there was a twist! The robots only worked well if the teacher knew how to use them. When the experienced teachers used DeepSeek, it was magic. When the newbies used it, the questions were sometimes so advanced and confusing that the students would get lost. It's like giving a race car to a driver who hasn't learned to drive yet; the car is fast, but the driver might crash. The study suggests that the tool isn't the hero; the teacher's skill in guiding the tool is what makes the difference.

The "Three-Session" Secret

One of the most practical discoveries happened over time. The teachers went through four training sessions.

Sessions 1 to 3: The teachers got significantly better at giving feedback. They learned how to mix the "warm hug" with the "sharp knife."
Session 4: The improvement stopped. It hit a wall, or a "plateau."

This suggests that schools might not need to run long, expensive four-day workshops. A shorter, intense three-day workshop might achieve the same results, saving about 25% of the cost. It's like realizing you don't need to run a marathon to get fit; sometimes, a few intense sprints are enough.

The Exceptions: When the Newbies Win and the Veterans Lose

The researchers didn't just look at the averages; they looked at the weird cases, too.

The Super-Newbies: 19% of the new teachers actually outperformed the average veteran. What made them special? They had done tutoring before, they used the smartest robot (DeepSeek), and they knew how to ask the robot very detailed questions.
The Sluggish Veterans: 16% of the experienced teachers gave feedback that was surprisingly generic and boring. Why? They were tired, they weren't good with computers, or they just got stuck in old habits.

This proves that just having years of experience doesn't guarantee you'll be good at using AI. You have to be willing to learn the new tools.

The Bottom Line

This study suggests that AI is a powerful tool, but it's not a replacement for a teacher's brain or heart. The best feedback comes when an experienced teacher uses AI to help them find the exact mistake, then wraps that correction in a genuine, specific encouragement.

The researchers found that while AI can generate words, it takes a human with "pedagogical reasoning" (the ability to think like a teacher) to turn those words into a lesson that actually helps a student grow. The future of education isn't about robots taking over; it's about teachers becoming "bilingual"—fluent in both human kindness and machine intelligence. And the good news is that with the right training (maybe just three sessions!), even new teachers can learn to serve up that perfect mix of warmth and wisdom.

Technical Summary: Compassionate Feedback in AI-Assisted Assessment

1. Problem Statement

The integration of generative AI into educational assessment has shifted focus toward technological capability, often neglecting the pedagogical intentionality required to support "compassionate teaching." While AI tools can automate feedback generation, they frequently produce responses that are linguistically encouraging but diagnostically shallow, failing to integrate emotional attunement with precise cognitive scaffolding.

A critical gap exists in understanding how teacher professional experience influences the generation of such compassionate feedback within AI-mediated contexts. Specifically, it remains unclear whether experienced teachers, possessing more sophisticated Pedagogical Content Knowledge (PCK) and pedagogical reasoning, can better critique, adapt, and integrate AI outputs to create feedback that validates student effort while guiding self-discovery. This study addresses the intersection of teacher experience, AI tool selection, and the operationalization of compassionate feedback within the Integrated AI Triad (IAT) Framework.

2. Methodology

The study employed a comparative qualitative content analysis design within a Design-Based Research (DBR) orientation, utilizing a critical-reflexive framework.

Participants: 57 teachers participated in a 16-session professional development workshop, divided into two cohorts: Novice teachers (0–3 years experience, n=26) and Experienced teachers (7+ years experience, n=31).
Data Generation: Teachers generated feedback for weak student responses using "Template 5" of the IAT framework, which explicitly required acknowledging effort, identifying areas for improvement, guiding thinking, and ending encouragingly.
AI Tools: Participants utilized four generative AI tools based on preference/access: ChatGPT, Microsoft Copilot, DeepSeek, and Perplexity.
Analytical Procedure:
- Coding: Two independent researchers coded 57 feedback entries (Inter-rater reliability $\kappa$ = 0.82). Coding focused on structural patterns (length, question types), linguistic tone (Compassion, Specificity, Cognitive Demand on 5-point Likert scales), and IAT alignment (Cognitive Partnership, Pedagogical Mediation).
- Statistical Analysis: Non-parametric tests were used due to non-normal data distribution (Shapiro-Wilk, p < .05). Analyses included Mann-Whitney U tests (Experience, Subject, Cohort), Kruskal-Wallis H tests (Tool effects), and Friedman tests with Wilcoxon post-hoc comparisons (Session progression).
- Reflexive Analysis: The study incorporated alternative hypothesis testing, negative case analysis (examining exceptional novices and underperforming experts), and overlap analysis to interrogate dominant patterns.

3. Key Results

3.1 Experience-Based Differences

Specificity and Scaffolding: Experienced teachers produced significantly more specific feedback (M=4.0/5) compared to novices (M=2.8/5) and demonstrated stronger cognitive scaffolding (M=3.8/5 vs. M=3.0/5). Their feedback contained denser content-specific references and targeted questions (42% content-specific open-ended vs. 15% for novices).
Compassion: Novices demonstrated marginally higher compassion scores (M=4.0/5 vs. M=3.7/5), but this compassion was often generic and decoupled from content challenges. Experienced teachers achieved a "balanced approach" (high compassion + high specificity) more frequently (32% vs. 15%), integrating validation with diagnostic guidance.
Cognitive Partnership: Experienced teachers were more likely to guide students toward self-discovery through strategic questioning, whereas novices more frequently provided direct corrections or generic encouragement.

3.2 Tool-Mediated Effects

DeepSeek: Generated significantly more Socratic, layered questioning compared to ChatGPT and Copilot. However, its utility was moderated by experience; while it produced exceptional feedback in the hands of experienced teachers, its outputs were sometimes "confusingly advanced" for novices.
Tool Preferences: Experienced teachers showed a preference for Copilot (52%), while novices distributed choices more evenly.
Mediation: Experienced teachers demonstrated "hermeneutic relations" with AI, interpreting and adapting outputs (editing, personalizing). Novices often exhibited "alterity relations," accepting AI outputs with limited critical evaluation.

3.3 Subject and Session Progression

Subject Area: STEM feedback was more directive and procedural, while Humanities feedback was more interpretive and open-ended. These epistemological differences persisted across experience levels.
Workshop Progression: Feedback quality improved significantly from Session 1 to Session 3 across all metrics (Compassion, Specificity, Cognitive Demand, IAT alignment). However, a plateau was observed between Session 3 and Session 4, suggesting diminishing returns for additional sessions.

3.4 Negative Cases and Exceptions

Exceptional Novices: 19% of novice entries outperformed the experienced mean. These cases were distinguished by prior tutoring/curriculum experience, the use of DeepSeek, and detailed prompting.
Underperforming Experts: 16% of experienced entries underperformed the novice mean, often due to session fatigue, limited digital literacy, or routinized thinking.
AI Errors: 5% of entries contained off-target AI responses; notably, all three teachers who encountered these errors recognized and corrected them, demonstrating appropriate pedagogical oversight.

4. Key Contributions

Operationalization of Compassionate Feedback: The study empirically defines compassionate feedback as a measurable dual-construct practice integrating emotional attunement with precise cognitive scaffolding, distinguishing it from superficial AI-generated warmth.
Extension of the IAT Framework: The findings extend the Integrated AI Triad (IAT) framework by demonstrating how teacher experience shapes the "bilingual professional" role (fluent in human pedagogy and AI). It provides evidence that experienced teachers navigate AI mediation through hermeneutic relations (critical translation), while novices often default to alterity relations (uncritical acceptance).
Tool-Experience Interaction: The research identifies distinct "toolprints" (e.g., DeepSeek's Socratic capacity) and demonstrates that tool affordances do not guarantee educational value without principled coordination between pedagogical intent and teacher expertise.
Professional Development Efficiency: The identification of a performance plateau after Session 3 suggests that condensed three-session workshops can achieve comparable outcomes to longer programs, potentially reducing implementation costs by approximately 25%.

5. Significance and Claims

The paper claims that teacher experience is a critical determinant in the quality of AI-assisted feedback, not merely in terms of volume but in the integration of affective and cognitive dimensions. It argues that while AI tools can automate feedback generation, they cannot replace the human judgment required to diagnose specific student errors and provide content-anchored compassion.

The study posits that the "bilingual professional" (AI-LDP) requires specific competencies—AI-Content Knowledge and AI-Pedagogical Knowledge—to critically evaluate and adapt AI outputs. The findings suggest that professional development should focus on:

Developing content-anchored compassion (moving beyond generic praise).
Enhancing prompt literacy and tool discernment.
Designing efficient training structures that leverage the initial learning curve without unnecessary extension.

The author concludes that AI should augment rather than replace human judgment. The most effective feedback emerges from the synthesis of AI capabilities and teacher expertise, a synthesis that requires pedagogical reasoning and the capacity to maintain ethical oversight. The study acknowledges limitations, including the sample size (N=57), the overrepresentation of science topics (~60%), and the single-country context (Iran), cautioning against broad generalization without further cross-cultural and longitudinal research.