Resident–faculty differences in global performance scores in otolaryngology workplace-based assessment: a national retrospective observational study
This national retrospective study of Taiwan's otolaryngology workplace-based assessments reveals that residents generally assign slightly lower performance scores than faculty raters, with discrepancies varying significantly by training level and case complexity, suggesting these differences can help identify specific encounters requiring targeted feedback.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of medical training, learning to be a doctor is not just about memorizing textbooks; it is about learning to judge one's own performance while working in the high-stakes environment of a hospital. This approach, known as competency-based education, relies heavily on workplace assessments. In these scenarios, a trainee performs a task, and both they and their supervising teacher rate how well it was done. The trainee's ability to look at their own work and say, "I did this well," or "I need to improve here," is considered a vital skill for lifelong learning. However, a long-standing question has lingered over these ratings: do trainees see their performance the same way their teachers do? Often, the gap between a student's self-rating and a teacher's rating has been treated as a mistake, a sign that the student simply does not know how good or bad they are. But in the complex reality of a hospital, where uncertainty and pressure are constant, that gap might tell a different story.
A recent study from Taiwan sought to understand this gap on a massive scale, moving beyond small classroom experiments to look at the real-world data of an entire national system. The researchers focused on otolaryngology, the medical specialty dealing with ears, noses, and throats. They examined nearly 47,000 specific assessment records collected over several years from residents in training and their faculty supervisors. In this system, after a clinical encounter, the resident gives themselves a score from one to ten, and the faculty member gives their own score for the same event. The study did not assume that the faculty score was the absolute truth; instead, it treated both scores as valid perspectives and looked at the difference between them to see what patterns emerged.
The researchers found that, on average, residents rated themselves slightly lower than their teachers did. The average difference was six-tenths of a point on the ten-point scale, meaning residents were generally more critical of their own work. However, this gap was not a sign of chaos or confusion. In more than 80 percent of the cases, the two scores were within one point of each other, showing a remarkable level of alignment. The most common pattern was not a huge disagreement, but a consistent, modest tendency for residents to be more conservative in their self-judgment. Only a small fraction of the time did residents rate themselves significantly higher than their teachers, while a larger, though still minority, portion of the time they rated themselves significantly lower.
What makes this finding particularly useful is that the size and direction of this gap changed depending on the situation. As residents gained more experience and moved from their first year of training to their fifth, they became less likely to overestimate their abilities. A senior resident was much less likely to claim they had done a perfect job when their teacher saw room for improvement, suggesting that with experience comes a more realistic, and perhaps more cautious, view of one's own skills. The complexity of the medical case also played a major role. When residents faced difficult, unusual, or advanced cases, the gap between their self-rating and the teacher's rating widened, with residents tending to rate themselves much lower. This suggests that in the face of uncertainty and complex challenges, trainees feel the weight of the situation more acutely than their supervisors, who may be looking at the broader picture of safety and appropriate decision-making.
The study also looked at whether the gender of the resident or the teacher influenced these ratings. The data showed that male residents were slightly more likely to rate themselves higher than their teachers compared to female residents, and male teachers were slightly less likely to give lower scores than female teachers. However, the researchers were careful to note that these are exploratory observations that need further study to understand the underlying reasons, which could range from personal confidence levels to how different pairs interact.
Ultimately, this large-scale analysis suggests that the difference between a trainee's self-assessment and a teacher's assessment is not a flaw to be fixed, but a feature to be understood. The gap is usually small and predictable, shaped by how much experience the trainee has and how difficult the case is. Rather than viewing a lower self-score as a failure of insight, educators can see it as a signal. When a resident rates themselves significantly lower than their teacher, especially in a complex case, it may indicate a moment where the resident is feeling uncertain or is being overly harsh on themselves. Conversely, when a resident rates themselves higher, it might be a moment to discuss specific gaps in performance. By treating these differences as a structured part of the learning process rather than random error, medical training can use these ratings to foster better conversations, helping doctors in training to develop not just their clinical skills, but also their ability to understand and guide their own professional growth.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.