Factors from Residency Applications Associated with Performance in Emergency Medicine: A Retrospective Study
This retrospective study of 133 emergency medicine residents found that higher USMLE Step 2 scores and younger age predict high residency performance, while application "red flags" strongly predict low performance, whereas traditional metrics like Step 1 scores and interview scores showed no significant association.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Every year, thousands of medical graduates seek a spot in emergency medicine residency programs, where they will train to become the doctors who handle the most urgent and chaotic moments in a hospital. For decades, program directors have faced a difficult puzzle: how to choose the applicants who will truly excel in this high-pressure environment. Traditionally, they have relied on a mix of tools, including scores from standardized medical exams, grades from medical school rotations, letters of recommendation, and interviews. However, the landscape is shifting. One major exam, which once served as a primary filter, has recently changed to a simple pass-or-fail system, forcing programs to look for new ways to predict success. The core question remains: do the numbers on an application actually tell us who will become a great emergency doctor, or are we looking at the wrong signs?
To answer this, a team of researchers at the University of Texas Health Science Center at Houston conducted a careful review of their own program's history. They looked back at 133 doctors who had completed their training between 2018 and 2024. The team did not just guess who was good; they gathered ratings from four senior faculty members who had overseen every resident's entire training. These mentors rated each graduate on a five-point scale, ranging from someone who should not have graduated to someone so outstanding that the faculty would choose them as their own personal doctor. The researchers then split the group into two categories: those who were rated as high performers and those who were rated as low performers. They then went back to the original application files to see which specific details from years ago matched up with these final ratings.
The study revealed a clear pattern regarding what actually predicts success. The strongest signal for a high-performing resident was a higher score on the second major medical licensing exam, known as USMLE Step 2. This test measures how well a student can apply medical knowledge to real clinical situations, rather than just recalling facts. The data showed that residents who scored higher on this exam were significantly more likely to be rated as excellent by their faculty. The researchers also found that younger residents tended to perform better than older ones, with the high-performing group having a median age of 26 compared to 28 for the lower-performing group. While the age difference was small, it was a consistent finding in their analysis.
Perhaps the most striking discovery was about what not to look for, or rather, what to look out for as a warning sign. The researchers found that if an applicant had a documented "red flag" on their record—such as a failed course, a gap in education that wasn't explained, or concerns about their professionalism—they almost certainly ended up as a low performer. In fact, every single resident in the study who had such a red flag was rated as a low performer. This suggests that while a perfect application cannot guarantee success, a specific type of failure in a candidate's history is a very strong warning that they will struggle.
Conversely, the study found that many of the tools programs traditionally rely on did not help predict who would be a star. The scores from the first medical licensing exam, the grades from core medical school rotations, membership in honor societies, and even the scores from the interview itself showed no meaningful connection to how well a resident actually performed. Surprisingly, the standardized letters of evaluation, which are meant to highlight a candidate's character and work ethic, actually showed a strange reverse trend: higher scores on these letters were slightly associated with lower performance ratings. This suggests that these subjective measures, while popular, may not be capturing the true traits that lead to success in the emergency room.
The researchers approached these findings through a specific lens, viewing a doctor's competence as a combination of two different worlds: the cognitive world of medical knowledge and reasoning, and the non-cognitive world of professionalism, resilience, and self-control. Their results suggest that the second medical exam is a good measure of the cognitive side, while red flags are a powerful indicator of problems in the non-cognitive side. The study implies that the interview and letters of recommendation, which are supposed to measure the non-cognitive side, are currently failing to distinguish between future stars and those who will struggle.
It is important to note that this study was conducted at a single hospital, and the number of doctors reviewed was relatively small. The researchers acknowledge that their findings are specific to their own program and that the ratings of performance, while done by multiple faculty members, still rely on human judgment. They did not find a magic formula that works for every situation, nor did they prove that age or exam scores are the only things that matter. Instead, they offered a clearer picture of what has worked in their specific setting. The data suggests that when selecting future emergency doctors, programs might benefit from placing more weight on the ability to apply medical knowledge and paying close attention to any history of professional lapses, while being more cautious about relying on interviews and subjective letters as the primary deciders of a candidate's potential.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.